• State Resolved
  • Locked Locked
  • Replies 27 replies
  • Subscribers 100 subscribers
  • Views 1121 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

TDA4VM: Custom layer not supported by tflite

AM69A: How to enable custom layer ?

Azer Tiop
Intellectual 350 points

Part Number: AM69A

Tool/software:

Hi,

I have a model with a custom layer that is already supported by TIDL.

I added my file for the custom layer in the SDK RTOS, and now I want to test my model.

However, when compiling, the layer is not recognized as CUSTOM_LAYER but as POOLING_LAYER:

tidl_tools_path                                 = /home/developer/edgeai/edgeai-tidl-tools/tidl_tools 
artifacts_folder                                = artefacts/artefacts_custom_layer_v0 
tidl_tensor_bits                                = 8 
debug_level                                     = 9 
num_tidl_subgraphs                              = 16 
tidl_denylist                                   = 
tidl_denylist_layer_name                        = 
tidl_denylist_layer_type                        = 
tidl_allowlist_layer_name                       = 
model_type                                      =  
tidl_calibration_accuracy_level                 = 7 
tidl_calibration_options:num_frames_calibration = 4 
tidl_calibration_options:bias_calibration_iterations = 3 
mixed_precision_factor = -1.000000 
model_group_id = 0 
power_of_2_quantization                         = 2 
ONNX QDQ Enabled                                = 0 
enable_high_resolution_optimization             = 0 
pre_batchnorm_fold                              = 1 
add_data_convert_ops                            = 0 
output_feature_16bit_names_list                 =  
m_params_16bit_names_list                       =  
m_single_core_layers_names_list                 =  
Inference mode                                  = 0 
Number of cores                                 = 1 
reserved_compile_constraints_flag               = 1601 
partial_init_during_compile                     = 0 
ti_internal_reserved_1                          = 

========================= [Model Compilation Started] =========================

Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning

============================== [Version Summary] ==============================

-------------------------------------------------------------------------------
|          TIDL Tools Version          |              10_00_08_00             |
-------------------------------------------------------------------------------
|         C7x Firmware Version         |              10_00_02_00             |
-------------------------------------------------------------------------------

============================== [Parsing Started] ==============================

[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- sequential_2/conv2d_3/Relu;sequential_2/conv2d_3/BiasAdd;sequential_2/conv2d_3/Conv2D;conv2d_3/bias  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 2 Tflite layer type --- 17 layer output name--- sequential_2/max_pooling2d_3/MaxPool  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 38 Tflite layer type --- 22 layer output name--- sequential_2/flatten_1/Reshape  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 6 Tflite layer type --- 9 layer output name--- sequential_2/dense_3/MatMul;sequential_2/dense_3/BiasAdd  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 7 Tflite layer type --- 25 layer output name--- StatefulPartitionedCall:0  -- [tidl_tfLiteRtImport_core.cpp, 3090]

Total Nodes = 5
-------------------------------------------------------------------------------
|          Core           |      No. of Nodes       |   Number of Subgraphs   |
-------------------------------------------------------------------------------
| C7x                     |                       5 |                       1 |
| CPU                     |                       0 |                       x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================

In TIDL_tfliteRtImportInit subgraph_id=10
Layer 0, subgraph id 10, name=StatefulPartitionedCall:0
Layer 1, subgraph id 10, name=serving_default_conv2d_3_input:0
In TIDL_tfliteRtImportNode, TIDL Layer type - 1, Tflite builtin code type - 3 
In TIDL_tfliteRtImportNode, TIDL Layer type - 2, Tflite builtin code type - 17 
In TIDL_tfliteRtImportNode, TIDL Layer type - 38, Tflite builtin code type - 22 
In TIDL_tfliteRtImportNode, TIDL Layer type - 6, Tflite builtin code type - 9 
In TIDL_tfliteRtImportNode, TIDL Layer type - 7, Tflite builtin code type - 25 
==================== [Optimization for subgraph_10 started] ====================

In TIDL_runtimesOptimizeNet: LayerIndex = 7, dataIndex = 6 
----------------------------- Optimization Summary -----------------------------
---------------------------------------------------------------------------------
|          Layer         | Nodes before optimization | Nodes after optimization |
---------------------------------------------------------------------------------
| TIDL_SoftMaxLayer      |                         1 |                        1 |
| TIDL_InnerProductLayer |                         1 |                        1 |
| TIDL_CropLayer         |                         0 |                        1 |
| TIDL_ConvolutionLayer  |                         1 |                        1 |
| TIDL_PoolingLayer      |                         1 |                        1 |
---------------------------------------------------------------------------------

=================== [Optimization for subgraph_10 completed] ===================

In TIDL_runtimesPostProcessNet 
************ in TIDL_subgraphRtCreate ************ 
 TIDL_initDebugTraceParams Done 

--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  12  0%|          | 0/4 [00:00<?, ?it/s]8, 19.67   , 0x00000000
1           , DDR Cacheable       , Persistent  ,  128, 0.65    , 0x00000000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0x00000000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0x00000000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0x00000000
5           , DDR Cacheable       , Persistent  ,  128, 258.53  , 0x00000000
6           , DDR Cacheable       , Scratch     ,  128, 178.79  , 0x00000000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0x00000000
8           , DDR Cacheable       , Scratch     ,  128, 96.12   , 0x00000000
9           , DDR Cacheable       , Scratch     ,  128, 131.00  , 0x00000000
10          , DDR Cacheable       , Persistent  ,  128, 321.81  , 0x00000000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0x00000000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x00000000
13          , DDR Cacheable       , Persistent  ,  128, 22613.75, 0x00000000
14          , DDR Cacheable       , Persistent  ,  128, 0.00    , 0x00000000
15          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x00000000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 24208.95
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
TIDL init call from ivision API 

--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  128, 19.67   , 0x2dcab000
1           , DDR Cacheable       , Persistent  ,  128, 0.65    , 0x70cf3000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0x6d601000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0x70cf2000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0x11a1a000
5           , DDR Cacheable       , Persistent  ,  128, 258.53  , 0xecc85000
6           , DDR Cacheable       , Scratch     ,  128, 178.79  , 0x0e208000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0x70cf1000
8           , DDR Cacheable       , Scratch     ,  128, 96.12   , 0x0dc85000
9           , DDR Cacheable       , Scratch     ,  128, 131.00  , 0x0c7df000
10          , DDR Cacheable       , Persistent  ,  128, 321.81  , 0xe9f5c000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0xe9edb000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x6fe03000
13          , DDR Cacheable       , Persistent  ,  128, 22613.75, 0xe578d000
14          , DDR Cacheable       , Persistent  ,  128, 0.00    , 0x6fe02000
15          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x6f401000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 24208.95
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
Alg Init for Layer # -    1
Alg Init for Layer # -    2
Alg Init for Layer # -    3
Alg Init for Layer # -    4
Alg Init for Layer # -    5
Alg Init for Layer # -    6
Alg Init for Layer # -    7
Alg Init for Layer # -    8
PREEMPTION: Adding a new priority object for targetPriority = 0, handle = 0x7c562dcab000
PREEMPTION: Now total number of priority objects = 6 at priorityId = 0,    with new memRec of base = 0x7c566fe03000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x7c562dcab000, return Addr = 0x7c55eec314b8
************ TIDL_subgraphRtCreate done ************ 
 tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
TIDL_deactivate is called with handle : 0x7c5640731000 - Copying handle of size 20144 from 0x7c5610f97080 to 0x7c5640731000 
TIDL_activate is called with handle : 0x7c562dcab000 - Copying handle of size 20144 from 0x7c562dcab000 to 0x7c5611a1a080 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 24.000000 1411.000000 280.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 1 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 7.000000 997.000000 221.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 2 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 4.000000 956.000000 8802.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 3 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 32.000000 1008.000000 247.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 4 : Running fixed point mode for calibration **************** 

In TIDL_runtimesPostProcessNet 

-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [1 / 3]: ------------------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [2 / 3]: ------------------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [3 / 3]: ------------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
100%|██████████| 4/4 [00:01<00:00,  3.78it/s]Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================


 Final number of subgraphs:1 , 5 nodes delegated to accelerator 

Running Runtimes GraphViz - /home/developer/edgeai/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser_runtimes.out artefacts/artefacts_custom_layer_v0/allowedNode.txt artefacts/artefacts_custom_layer_v0/tempDir/graphvizInfo.txt artefacts/artefacts_custom_layer_v0/tempDir/runtimes_visualization.svg 
tidl_tfLiteRtImport_delegate.cpp Invoke 647

To enable the custom layer, I set the following variable in the import config file: enableCustomLayers = 1

Then I compiled the custom project with `{TIDL_BASE_PATH}/make tidl`.

After that, I set up SDK RTOS with:

```
make vision_apps -jN
make tidl_rt -jN
```

I also set up edgeai_tidl_tools again.

However, the CUSTOM_LAYER is still not recognized, and the TIDL layer is chosen over my custom layer.

Could you help me enable the custom layer so it can be recognized?

Thanks,
Azer

  • 0
    TI__Genius 16500 points

    Your question has been assigned to Chris

  • 0 in reply to Chris Tsongas
    Intellectual 350 points

    Hi,

    The priority for this question is quite high. I would greatly appreciate it if you could provide me with an answer as soon as possible.

    Thank you very much for your assistance,

    Azer

  • 0
    TI__Genius 16500 points

    Hi Azer,

    Currently custom layer is only supported via the TIDL-RT interface ( i.e., config file-based interface). You should use the same in this case and set the following variable in import config file to enable custom layers:

    enableCustomLayers = 1
    Chris
  • 0 in reply to Chris Tsongas
    Intellectual 350 points

    Hi Chris,

    Thank you for your answer. What do you mean by TIDL-RT interface ( i.e., config file-based interface)? Do you have any example on how to use it ?

    Regards,

    Azer

  • 0 in reply to Azer Tiop
    Intellectual 350 points

    Hi,

    I just read : https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/10_00_00_05/exports/docs/c7x-mma-tidl/ti_dl/docs/user_guide_html/itidl__rt_8h.html

    """

    This file defines the public interface for TIDL Runtime API.
    This same API supports a client on a different CPU, out-of-process on the same CPU, or in-process.
    This interface is targeted for users developing inference application
    with imported TIDL models and users developing offload APIs for Open source
    runtime engines like TFlite (Delegate) and ONNX runtime (Execution provider) etc.

    """

    I am using TFLITE to compile and make inference. I understand that it is supposed to work with a custom layer.

  • 0 in reply to Azer Tiop
    TI__Genius 16500 points

    Hi Azer,

    Custom layers are not fully supported in any framework except TIDL-RT.

    Regards,

    Chris

  • 0 in reply to Chris Tsongas
    Intellectual 350 points

    Hi,

    This is not clear. I rephrase my question:

    1 ) Can you give methe steps to perform custom layer TIDL after adding and modifying the files in the custom folder?

    2 ) Can you specify if there are steps to do on the EVM?

    Thank you for your answers,
    Azer

  • 0 in reply to Azer Tiop
    Intellectual 350 points

    Hi,

    Does the compilation step is necessary to make inference on EVM with a model with custom layer? 

    Then, is compilation part of TIDL-RT?

    Thanks for your attention

    Azer

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    Hi Azer,

    For the EVM, you need to run the model compilation on PC and the generated artifacts can be used for inference on the device. This would be the same for the model with the custom layer.

    Regards,

    Christina

  • 0 in reply to Christina Kuruvilla
    Intellectual 350 points

    Hi,

    Thanks  for your answer.

    Could you provide me compilation logs for a model with custom layer Maxpool example? So I would know what to expect.

    Thanks and regards,

    Azer

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    The best practice would be to replicate it on your exact setup to see exactly where your issue may be. I could do this for you but I do think it would be more beneficial if you did it on yours.

    Warm regards,

    Christina

  • 0 in reply to Christina Kuruvilla
    Intellectual 350 points

    Hi Christina,

    I already give my compilation logs on the first post. Here it is : 

    tidl_tools_path                                 = /home/developer/edgeai/edgeai-tidl-tools/tidl_tools 
artifacts_folder                                = artefacts/artefacts_custom_layer_v0 
tidl_tensor_bits                                = 8 
debug_level                                     = 9 
num_tidl_subgraphs                              = 16 
tidl_denylist                                   = 
tidl_denylist_layer_name                        = 
tidl_denylist_layer_type                        = 
tidl_allowlist_layer_name                       = 
model_type                                      =  
tidl_calibration_accuracy_level                 = 7 
tidl_calibration_options:num_frames_calibration = 4 
tidl_calibration_options:bias_calibration_iterations = 3 
mixed_precision_factor = -1.000000 
model_group_id = 0 
power_of_2_quantization                         = 2 
ONNX QDQ Enabled                                = 0 
enable_high_resolution_optimization             = 0 
pre_batchnorm_fold                              = 1 
add_data_convert_ops                            = 0 
output_feature_16bit_names_list                 =  
m_params_16bit_names_list                       =  
m_single_core_layers_names_list                 =  
Inference mode                                  = 0 
Number of cores                                 = 1 
reserved_compile_constraints_flag               = 1601 
partial_init_during_compile                     = 0 
ti_internal_reserved_1                          = 

========================= [Model Compilation Started] =========================

Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning

============================== [Version Summary] ==============================

-------------------------------------------------------------------------------
|          TIDL Tools Version          |              10_00_08_00             |
-------------------------------------------------------------------------------
|         C7x Firmware Version         |              10_00_02_00             |
-------------------------------------------------------------------------------

============================== [Parsing Started] ==============================

[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- sequential_2/conv2d_3/Relu;sequential_2/conv2d_3/BiasAdd;sequential_2/conv2d_3/Conv2D;conv2d_3/bias  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 2 Tflite layer type --- 17 layer output name--- sequential_2/max_pooling2d_3/MaxPool  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 38 Tflite layer type --- 22 layer output name--- sequential_2/flatten_1/Reshape  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 6 Tflite layer type --- 9 layer output name--- sequential_2/dense_3/MatMul;sequential_2/dense_3/BiasAdd  -- [tidl_tfLiteRtImport_core.cpp, 3090]
[TIDL Import] [PARSER] SUPPORTED: Supported TIDL layer type --- 7 Tflite layer type --- 25 layer output name--- StatefulPartitionedCall:0  -- [tidl_tfLiteRtImport_core.cpp, 3090]

Total Nodes = 5
-------------------------------------------------------------------------------
|          Core           |      No. of Nodes       |   Number of Subgraphs   |
-------------------------------------------------------------------------------
| C7x                     |                       5 |                       1 |
| CPU                     |                       0 |                       x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================

In TIDL_tfliteRtImportInit subgraph_id=10
Layer 0, subgraph id 10, name=StatefulPartitionedCall:0
Layer 1, subgraph id 10, name=serving_default_conv2d_3_input:0
In TIDL_tfliteRtImportNode, TIDL Layer type - 1, Tflite builtin code type - 3 
In TIDL_tfliteRtImportNode, TIDL Layer type - 2, Tflite builtin code type - 17 
In TIDL_tfliteRtImportNode, TIDL Layer type - 38, Tflite builtin code type - 22 
In TIDL_tfliteRtImportNode, TIDL Layer type - 6, Tflite builtin code type - 9 
In TIDL_tfliteRtImportNode, TIDL Layer type - 7, Tflite builtin code type - 25 
==================== [Optimization for subgraph_10 started] ====================

In TIDL_runtimesOptimizeNet: LayerIndex = 7, dataIndex = 6 
----------------------------- Optimization Summary -----------------------------
---------------------------------------------------------------------------------
|          Layer         | Nodes before optimization | Nodes after optimization |
---------------------------------------------------------------------------------
| TIDL_SoftMaxLayer      |                         1 |                        1 |
| TIDL_InnerProductLayer |                         1 |                        1 |
| TIDL_CropLayer         |                         0 |                        1 |
| TIDL_ConvolutionLayer  |                         1 |                        1 |
| TIDL_PoolingLayer      |                         1 |                        1 |
---------------------------------------------------------------------------------

=================== [Optimization for subgraph_10 completed] ===================

In TIDL_runtimesPostProcessNet 
************ in TIDL_subgraphRtCreate ************ 
 TIDL_initDebugTraceParams Done 

--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  12  0%|          | 0/4 [00:00<?, ?it/s]8, 19.67   , 0x00000000
1           , DDR Cacheable       , Persistent  ,  128, 0.65    , 0x00000000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0x00000000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0x00000000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0x00000000
5           , DDR Cacheable       , Persistent  ,  128, 258.53  , 0x00000000
6           , DDR Cacheable       , Scratch     ,  128, 178.79  , 0x00000000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0x00000000
8           , DDR Cacheable       , Scratch     ,  128, 96.12   , 0x00000000
9           , DDR Cacheable       , Scratch     ,  128, 131.00  , 0x00000000
10          , DDR Cacheable       , Persistent  ,  128, 321.81  , 0x00000000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0x00000000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x00000000
13          , DDR Cacheable       , Persistent  ,  128, 22613.75, 0x00000000
14          , DDR Cacheable       , Persistent  ,  128, 0.00    , 0x00000000
15          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x00000000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 24208.95
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
TIDL init call from ivision API 

--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  128, 19.67   , 0x2dcab000
1           , DDR Cacheable       , Persistent  ,  128, 0.65    , 0x70cf3000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0x6d601000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0x70cf2000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0x11a1a000
5           , DDR Cacheable       , Persistent  ,  128, 258.53  , 0xecc85000
6           , DDR Cacheable       , Scratch     ,  128, 178.79  , 0x0e208000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0x70cf1000
8           , DDR Cacheable       , Scratch     ,  128, 96.12   , 0x0dc85000
9           , DDR Cacheable       , Scratch     ,  128, 131.00  , 0x0c7df000
10          , DDR Cacheable       , Persistent  ,  128, 321.81  , 0xe9f5c000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0xe9edb000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x6fe03000
13          , DDR Cacheable       , Persistent  ,  128, 22613.75, 0xe578d000
14          , DDR Cacheable       , Persistent  ,  128, 0.00    , 0x6fe02000
15          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x6f401000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 24208.95
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
Alg Init for Layer # -    1
Alg Init for Layer # -    2
Alg Init for Layer # -    3
Alg Init for Layer # -    4
Alg Init for Layer # -    5
Alg Init for Layer # -    6
Alg Init for Layer # -    7
Alg Init for Layer # -    8
PREEMPTION: Adding a new priority object for targetPriority = 0, handle = 0x7c562dcab000
PREEMPTION: Now total number of priority objects = 6 at priorityId = 0,    with new memRec of base = 0x7c566fe03000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x7c562dcab000, return Addr = 0x7c55eec314b8
************ TIDL_subgraphRtCreate done ************ 
 tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
TIDL_deactivate is called with handle : 0x7c5640731000 - Copying handle of size 20144 from 0x7c5610f97080 to 0x7c5640731000 
TIDL_activate is called with handle : 0x7c562dcab000 - Copying handle of size 20144 from 0x7c562dcab000 to 0x7c5611a1a080 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 24.000000 1411.000000 280.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 1 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 7.000000 997.000000 221.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 2 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 4.000000 956.000000 8802.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 3 : Running float inference **************** 

tidl_tfLiteRtImport_delegate.cpp Invoke 647 
tidl_tfLiteRtImport_delegate.cpp Invoke 526 
*******   In TIDL_subgraphRtInvoke  ******** 
TIDL_process is started with handle : 0x7c562dcab000 
Coreid 0 Layerid to execute = 0 
Core 0 Alg Process for Layer # -    0, layer type 0
Coreid 0 Layerid to execute = 1 
Core 0 Alg Process for Layer # -    1, layer type 1
Processing Layer # -    1
Core 0 End of Layer # -    1 with outPtrs[0] = 0x7c560e208000
Coreid 0 Layerid to execute = 2 
Core 0 Alg Process for Layer # -    2, layer type 15
Processing Layer # -    2
Core 0 End of Layer # -    2 with outPtrs[0] = 0x7c560e218080
Coreid 0 Layerid to execute = 3 
Core 0 Alg Process for Layer # -    3, layer type 2
Processing Layer # -    3
Core 0 End of Layer # -    3 with outPtrs[0] = 0x7c560e226200
Coreid 0 Layerid to execute = 4 
Core 0 Alg Process for Layer # -    4, layer type 29
Processing Layer # -    4
Core 0 End of Layer # -    4 with outPtrs[0] = 0x7c560e229b00
Coreid 0 Layerid to execute = 5 
Core 0 Alg Process for Layer # -    5, layer type 38
Processing Layer # -    5
Core 0 End of Layer # -    5 with outPtrs[0] = 0x7c560e22d400
Coreid 0 Layerid to execute = 6 
Core 0 Alg Process for Layer # -    6, layer type 29
Processing Layer # -    6
Core 0 End of Layer # -    6 with outPtrs[0] = 0x7c560e230d00
Coreid 0 Layerid to execute = 7 
Core 0 Alg Process for Layer # -    7, layer type 6
Processing Layer # -    7
Executing reference flow for inner product layer 
Core 0 End of Layer # -    7 with outPtrs[0] = 0x7c560e234600
Coreid 0 Layerid to execute = 8 
Core 0 Alg Process for Layer # -    8, layer type 7
Processing Layer # -    8
Core 0 End of Layer # -    8 with outPtrs[0] = 0x7c5670d46000
Coreid 0 Layerid to execute = 9 
Core 0 Alg Process for Layer # -    9, layer type 0
Coreid 0 Layerid to execute = -1 
TIDL_process is completed with handle : 0x7c562dcab000 
 Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
     1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
 Sum of Layer Cycles 0 
Sub Graph Stats 32.000000 1008.000000 247.000000 
*******  TIDL_subgraphRtInvoke done  ******** 

 ************ Frame index 4 : Running fixed point mode for calibration **************** 

In TIDL_runtimesPostProcessNet 

-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [1 / 3]: ------------------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [2 / 3]: ------------------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [3 / 3]: ------------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
100%|██████████| 4/4 [00:01<00:00,  3.78it/s]Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================


 Final number of subgraphs:1 , 5 nodes delegated to accelerator 

Running Runtimes GraphViz - /home/developer/edgeai/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser_runtimes.out artefacts/artefacts_custom_layer_v0/allowedNode.txt artefacts/artefacts_custom_layer_v0/tempDir/graphvizInfo.txt artefacts/artefacts_custom_layer_v0/tempDir/runtimes_visualization.svg 
tidl_tfLiteRtImport_delegate.cpp Invoke 647

    I just want to know how compilation logs are supposed to look like when a custom layer is detected. Here, for ex, it would be the TIDL Maxpool custom layer.

    Thanks and regards,

    Azer

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    Hi Azer,

    I meant you should try to do the TIDL Maxpool custom layer on your setup to see if it works. I will try to do it on my end and get back to you with the logs.

    Warm regards,

    Christina

  • 0 in reply to Christina Kuruvilla
    Intellectual 350 points

    Hi,

    Thanks for your answer.

    That's the issue. I did it here with a model that only has one maxpool layer with the parameters taken into account by the custom layer (see files).

    And we can see that, in the optimization summary, the TIDL_CustomLayer is not detected : 

    ----------------------------- Optimization Summary -----------------------------
----------------------------------------------------------------------------
|       Layer       | Nodes before optimization | Nodes after optimization |
----------------------------------------------------------------------------
| TIDL_PoolingLayer |                         1 |                        1 |
----------------------------------------------------------------------------

    You can find all the logs in the directory attached, both for when TIDL Maxpool custom layer is setup and not, with artefacts and the model used.

    Regards and thanks for your work.

    custom_test.zip

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    Hi Azer,

    Thank you for the files. Please give me some time to investigate this issue. 

    Christina

  • 0 in reply to Christina Kuruvilla
    Intellectual 350 points

    Hi,

    The priority for this question is quite high. I would greatly appreciate it if you could provide me with an answer as soon as possible.

    Thank you very much for your assistance,

    Azer

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    Hi Azer,

    I am currently working on this and will get you a response on my status tomorrow.

    Warm regards,

    Christina

  • 0 in reply to Christina Kuruvilla
    Intellectual 350 points

    Hi, 

    Did you manage to get compilation logs for when custom Layer Maxpool is recognized?

    Thanks and regards,

    Azer

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    Hi Azer,

    I have not but Dev Team is helping in regards to this question and we should hear back soon.

    Warm regards,

    Christina

  • 0 in reply to Christina Kuruvilla
    TI__Expert 7080 points

    Hi Azer,

    Just an update. Dev team is working on a fix for the issue so the custom layer will get higher precedence over the TIDL layer. They are in the process of testing it and I will update you once I know it is up.

    Warm regards,

    Christina

  • 0 in reply to Christina Kuruvilla
    Intellectual 350 points

    Hi,

    Thank you for the update.

    Do you have any news?

    Regards,

    Azer

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    Hi Azer,

    Last I checked, they were still working on it. I have sent a message this morning on an update however, our Dev team (in India) is on holiday until Monday, so I will get back to you next week with an update. 

  • 0 in reply to Christina Kuruvilla
    TI__Expert 7080 points

    Hi Azer, 

    Just an update as promised. The fix for the issue will most likely come in our next SDK release (11.0) with some additional custom layer updates. I will let you know when this is available. 

  • 0 in reply to Christina Kuruvilla
    TI__Expert 7080 points

    Hi Azer,

    Do you have a model and config_file that you want implemented with the custom layer? Dev team wanted to test it with their custom layer fixes. If you need to send it via email due to privacy, please share yours so I can send a message.

    Warm regards,

    Christina

  • 0 in reply to Christina Kuruvilla
    Intellectual 350 points

    Hi,

    Thanks for the offer. In the zip file, you can find a tflite model and logs with the config I want for compilation.

    Do you have any info about when SDK v11 will be released?

    Regards,

    Azer


    0574.custom_test.zip

  • 0 in reply to Azer Tiop
    TI__Expert 7080 points

    Thanks Azer,

    I have sent this over to Dev team. SDK v11 is set to release sometime end of May/June.

    Warm regards,

    Christina

  • +1 in reply to Christina Kuruvilla
    TI__Expert 7080 points

    Hi Azer,

    TIDL 11.0 just got released. Here is the github: https://github.com/TexasInstruments/edgeai-tidl-tools/releases

    This should have the fixes for your custom layer issues. Please reopen this thread if you continue to have holdups.

    Warm regards,

    Christina