AM69A: How to enable custom layer ?

Your question has been assigned to Chris

Hi,

The priority for this question is quite high. I would greatly appreciate it if you could provide me with an answer as soon as possible.

Thank you very much for your assistance,

Azer

Hi Azer,

Currently custom layer is only supported via the TIDL-RT interface ( i.e., config file-based interface). You should use the same in this case and set the following variable in import config file to enable custom layers:

Hi Chris,

Thank you for your answer. What do you mean by TIDL-RT interface ( i.e., config file-based interface)? Do you have any example on how to use it ?

Regards,

Azer

Hi,

I just read : https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/10_00_00_05/exports/docs/c7x-mma-tidl/ti_dl/docs/user_guide_html/itidl__rt_8h.html

"""

This file defines the public interface for TIDL Runtime API.
This same API supports a client on a different CPU, out-of-process on the same CPU, or in-process.
This interface is targeted for users developing inference application
with imported TIDL models and users developing offload APIs for Open source
runtime engines like TFlite (Delegate) and ONNX runtime (Execution provider) etc.

"""

I am using TFLITE to compile and make inference. I understand that it is supposed to work with a custom layer.

Hi Azer,

Custom layers are not fully supported in any framework except TIDL-RT.

Regards,

Chris

Hi,

This is not clear. I rephrase my question:

1 ) Can you give methe steps to perform custom layer TIDL after adding and modifying the files in the custom folder?

2 ) Can you specify if there are steps to do on the EVM?

Thank you for your answers,
Azer

Hi,

Does the compilation step is necessary to make inference on EVM with a model with custom layer? 

Then, is compilation part of TIDL-RT?

Thanks for your attention

Azer

Hi Azer,

For the EVM, you need to run the model compilation on PC and the generated artifacts can be used for inference on the device. This would be the same for the model with the custom layer.

Regards,

Christina

Hi,

Thanks  for your answer.

Could you provide me compilation logs for a model with custom layer Maxpool example? So I would know what to expect.

Thanks and regards,

Azer

The best practice would be to replicate it on your exact setup to see exactly where your issue may be. I could do this for you but I do think it would be more beneficial if you did it on yours.

Warm regards,

Christina

Hi Christina,

I already give my compilation logs on the first post. Here it is : 

tidl_tools_path                                 = /home/developer/edgeai/edgeai-tidl-tools/tidl_tools 
artifacts_folder                                = artefacts/artefacts_custom_layer_v0 
tidl_tensor_bits                                = 8 
debug_level                                     = 9 
num_tidl_subgraphs                              = 16 
tidl_denylist                                   = 
tidl_denylist_layer_name                        = 
tidl_denylist_layer_type                        = 
tidl_allowlist_layer_name                       = 
model_type                                      =  
tidl_calibration_accuracy_level                 = 7 
tidl_calibration_options:num_frames_calibration = 4 
tidl_calibration_options:bias_calibration_iterations = 3 
mixed_precision_factor = -1.000000 
model_group_id = 0 
power_of_2_quantization                         = 2 
ONNX QDQ Enabled                                = 0 
enable_high_resolution_optimization             = 0 
pre_batchnorm_fold                              = 1 
add_data_convert_ops                            = 0 
output_feature_16bit_names_list                 =  
m_params_16bit_names_list                       =  
m_single_core_layers_names_list                 =  
Inference mode                                  = 0 
Number of cores                                 = 1 
reserved_compile_constraints_flag               = 1601 
partial_init_during_compile                     = 0 
ti_internal_reserved_1                          = 

========================= [Model Compilation Started] =========================

Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning

============================== [Version Summary] ==============================

-------------------------------------------------------------------------------
|          TIDL Tools Version          |              10_00_08_00             |
-------------------------------------------------------------------------------
|         C7x Firmware Version         |              10_00_02_00             |
-------------------------------------------------------------------------------

============================== [Parsing Started] ==============================

[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- sequential_2/conv2d_3/Relu;sequential_2/conv2d_3/BiasAdd;sequential_2/conv2d_3/Conv2D;conv2d_3/bias  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 2 Tflite layer type --- 17 layer output name--- sequential_2/max_pooling2d_3/MaxPool  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 38 Tflite layer type --- 22 layer output name--- sequential_2/flatten_1/Reshape  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 6 Tflite layer type --- 9 layer output name--- sequential_2/dense_3/MatMul;sequential_2/dense_3/BiasAdd  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 7 Tflite layer type --- 25 layer output name--- StatefulPartitionedCall:0  -- [tidl_tfLiteRtImport_core.cpp, 3090]

Total Nodes = 5
-------------------------------------------------------------------------------
|          Core           |      No. of Nodes       |   Number of Subgraphs   |
-------------------------------------------------------------------------------
| C7x                     |                       5 |                       1 |
| CPU                     |                       0 |                       x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================

In TIDL_tfliteRtImportInit subgraph_id=10
Layer 0, subgraph id 10, name=StatefulPartitionedCall:0
Layer 1, subgraph id 10, name=serving_default_conv2d_3_input:0
In TIDL_tfliteRtImportNode, TIDL Layer type - 1, Tflite builtin code type - 3 
In TIDL_tfliteRtImportNode, TIDL Layer type - 2, Tflite builtin code type - 17 
In TIDL_tfliteRtImportNode, TIDL Layer type - 38, Tflite builtin code type - 22 
In TIDL_tfliteRtImportNode, TIDL Layer type - 6, Tflite builtin code type - 9 
In TIDL_tfliteRtImportNode, TIDL Layer type - 7, Tflite builtin code type - 25 
==================== [Optimization for subgraph_10 started] ====================

In TIDL_runtimesOptimizeNet: LayerIndex = 7, dataIndex = 6 
----------------------------- Optimization Summary -----------------------------
---------------------------------------------------------------------------------
|          Layer         | Nodes before optimization | Nodes after optimization |
---------------------------------------------------------------------------------
| TIDL_SoftMaxLayer      |                         1 |                        1 |
| TIDL_InnerProductLayer |                         1 |                        1 |
| TIDL_CropLayer         |                         0 |                        1 |
| TIDL_ConvolutionLayer  |                         1 |                        1 |
| TIDL_PoolingLayer      |                         1 |                        1 |
---------------------------------------------------------------------------------

=================== [Optimization for subgraph_10 completed] ===================

In TIDL_runtimesPostProcessNet 
************ in TIDL_subgraphRtCreate ************ 
 TIDL_initDebugTraceParams Done 

--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  12  0%|          | 0/4 [00:00<?, ?it/s]8, 19.67   , 0x00000000
1           , DDR Cacheable       , Persistent  ,  128, 0.65    , 0x00000000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0x00000000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0x00000000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0x00000000
5           , DDR Cacheable       , Persistent  ,  128, 258.53  , 0x00000000
6           , DDR Cacheable       , Scratch     ,  128, 178.79  , 0x00000000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0x00000000
8           , DDR Cacheable       , Scratch     ,  128, 96.12   , 0x00000000
9           , DDR Cacheable       , Scratch     ,  128, 131.00  , 0x00000000
10          , DDR Cacheable       , Persistent  ,  128, 321.81  , 0x00000000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0x00000000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x00000000
13          , DDR Cacheable       , Persistent  ,  128, 22613.75, 0x00000000
14          , DDR Cacheable       , Persistent  ,  128, 0.00    , 0x00000000
15          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x00000000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 24208.95
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
TIDL init call from ivision API 

--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  128, 19.67   , 0x2dcab000
1           , DDR Cacheable       , Persistent  ,  128, 0.65    , 0x70cf3000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0x6d601000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0x70cf2000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0x11a1a000
5           , DDR Cacheable       , Persistent  ,  128, 258.53  , 0xecc85000
6           , DDR Cacheable       , Scratch     ,  128, 178.79  , 0x0e208000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0x70cf1000
8           , DDR Cacheable       , Scratch     ,  128, 96.12   , 0x0dc85000
9           , DDR Cacheable       , Scratch     ,  128, 131.00  , 0x0c7df000
10          , DDR Cacheable       , Persistent  ,  128, 321.81  , 0xe9f5c000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0xe9edb000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x6fe03000
13          , DDR Cacheable       , Persistent  ,  128, 22613.75, 0xe578d000
14          , DDR Cacheable       , Persistent  ,  128, 0.00    , 0x6fe02000
15          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x6f401000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 24208.95
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
Alg Init for Layer # -    1
Alg Init for Layer # -    2
Alg Init for Layer # -    3
Alg Init for Layer # -    4
Alg Init for Layer # -    5
Alg Init for Layer # -    6
Alg Init for Layer # -    7
Alg Init for Layer # -    8
PREEMPTION: Adding a new priority object for targetPriority = 0, handle = 0x7c562dcab000
PREEMPTION: Now total number of priority objects = 6 at priorityId = 0,    with new memRec of base = 0x7c566fe03000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x7c562dcab000, return Addr = 0x7c55eec314b8
************ TIDL_subgraphRtCreate done ************ 
 tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
TIDL_deactivate is called with handle : 0x7c5640731000 - Copying handle of size 20144 from 0x7c5610f97080 to 0x7c5640731000 
TIDL_activate is called with handle : 0x7c562dcab000 - Copying handle of size 20144 from 0x7c562dcab000 to 0x7c5611a1a080 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 24.000000 1411.000000 280.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 1 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 7.000000 997.000000 221.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 2 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 4.000000 956.000000 8802.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 3 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 32.000000 1008.000000 247.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 4 : Running fixed point mode for calibration **************** 

In TIDL_runtimesPostProcessNet 

-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [1 / 3]: ------------------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [2 / 3]: ------------------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [3 / 3]: ------------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
100%|██████████| 4/4 [00:01<00:00,  3.78it/s]Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================


 Final number of subgraphs:1 , 5 nodes delegated to accelerator 

Running Runtimes GraphViz - /home/developer/edgeai/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser_runtimes.out artefacts/artefacts_custom_layer_v0/allowedNode.txt artefacts/artefacts_custom_layer_v0/tempDir/graphvizInfo.txt artefacts/artefacts_custom_layer_v0/tempDir/runtimes_visualization.svg 
tidl_tfLiteRtImport_delegate.cpp Invoke 647 

I just want to know how compilation logs are supposed to look like when a custom layer is detected. Here, for ex, it would be the TIDL Maxpool custom layer.

Thanks and regards,

Azer

Hi Azer,

I meant you should try to do the TIDL Maxpool custom layer on your setup to see if it works. I will try to do it on my end and get back to you with the logs.

Warm regards,

Christina

Hi,

Thanks for your answer.

That's the issue. I did it here with a model that only has one maxpool layer with the parameters taken into account by the custom layer (see files).

And we can see that, in the optimization summary, the TIDL_CustomLayer is not detected : 

----------------------------- Optimization Summary -----------------------------
----------------------------------------------------------------------------
|       Layer       | Nodes before optimization | Nodes after optimization |
----------------------------------------------------------------------------
| TIDL_PoolingLayer |                         1 |                        1 |
----------------------------------------------------------------------------

You can find all the logs in the directory attached, both for when TIDL Maxpool custom layer is setup and not, with artefacts and the model used.

Regards and thanks for your work.

custom_test.zip

Hi Azer,

Thank you for the files. Please give me some time to investigate this issue. 

Christina

Hi,

The priority for this question is quite high. I would greatly appreciate it if you could provide me with an answer as soon as possible.

Thank you very much for your assistance,

Azer

Hi Azer,

I am currently working on this and will get you a response on my status tomorrow.

Warm regards,

Christina

Hi, 

Did you manage to get compilation logs for when custom Layer Maxpool is recognized?

Thanks and regards,

Azer

Hi Azer,

I have not but Dev Team is helping in regards to this question and we should hear back soon.

Warm regards,

Christina

Hi Azer,

Just an update. Dev team is working on a fix for the issue so the custom layer will get higher precedence over the TIDL layer. They are in the process of testing it and I will update you once I know it is up.

Warm regards,

Christina

Hi,

Thank you for the update.

Do you have any news?

Regards,

Azer

Hi Azer,

Last I checked, they were still working on it. I have sent a message this morning on an update however, our Dev team (in India) is on holiday until Monday, so I will get back to you next week with an update. 

Hi Azer, 

Just an update as promised. The fix for the issue will most likely come in our next SDK release (11.0) with some additional custom layer updates. I will let you know when this is available. 

Hi Azer,

Do you have a model and config_file that you want implemented with the custom layer? Dev team wanted to test it with their custom layer fixes. If you need to send it via email due to privacy, please share yours so I can send a message.

Warm regards,

Christina

Hi,

Thanks for the offer. In the zip file, you can find a tflite model and logs with the config I want for compilation.

Do you have any info about when SDK v11 will be released?

Regards,

Azer

0574.custom_test.zip

Thanks Azer,

I have sent this over to Dev team. SDK v11 is set to release sometime end of May/June.

Warm regards,

Christina