I'm trying to build the custom model from ONNX using edgeai-tidl-tools, but encountered the problem.
Here is the visualization of ONNX layers:
Here is the log message with max debug level:
tidl_tools_path = /home/root/tidl_tools
artifacts_folder = /home/root/model-artifacts/search/
tidl_tensor_bits = 8
debug_level = 6
num_tidl_subgraphs = 16
tidl_denylist =
tidl_denylist_layer_name =
tidl_denylist_layer_type =
tidl_allowlist_layer_name =
model_type =
tidl_calibration_accuracy_level = 7
tidl_calibration_options:num_frames_calibration = 20
tidl_calibration_options:bias_calibration_iterations = 50
mixed_precision_factor = -1.000000
model_group_id = 0
power_of_2_quantization = 2
enable_high_resolution_optimization = 0
pre_batchnorm_fold = 1
add_data_convert_ops = 0
output_feature_16bit_names_list =
m_params_16bit_names_list =
reserved_compile_constraints_flag = 1601
ti_internal_reserved_1 =
****** WARNING : Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options******
Supported TIDL layer type --- Conv -- /Conv
Supported TIDL layer type --- Reshape -- /connect_model/cls_encode/Reshape
Unsupported (TIDL check) TIDL layer type --- Transpose
Preliminary subgraphs created = 1
Final number of subgraphs created are : 1, - Offloaded Nodes - 2, Total Nodes - 3
TIDL ALLOWLISTING LAYER CHECK -- [TIDL_TransposeLayer] should be removed in import process. If not, this model will not work!
/home/root/tidl_tools/tidl_graphVisualiser_runtimes.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
Running runtimes graphviz - /home/root/tidl_tools/tidl_graphVisualiser_runtimes.out /home/root/model-artifacts/search//allowedNode.txt /home/root/model-artifacts/search//tempDir/graphvizInfo.txt /home/root/model-artifacts/search//tempDir/runtimes_visualization.svg
*** In TIDL_createStateImportFunc ***
Compute on node : TIDLExecutionProvider_TIDL_0_0
0, Conv, 3, 1, template_features, /Conv_output_0
1, Reshape, 2, 1, /Conv_output_0, /connect_model/cls_encode/Reshape_output_0
Input tensor name - template_features
Output tensor name - /connect_model/cls_encode/Reshape_output_0
Initialized
In TIDL_onnxRtImportInit subgraph_name=_connect_model_cls_encode_Reshape_output_0
Layer 0, subgraph id _connect_model_cls_encode_Reshape_output_0, name=/connect_model/cls_encode/Reshape_output_0
Layer 1, subgraph id _connect_model_cls_encode_Reshape_output_0, name=template_features
In TIDL_runtimesOptimizeNet: LayerIndex = 4, dataIndex = 3
************** Frame index 1 : Running float import *************
In TIDL_runtimesPostProcessNet
/home/root/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
****************************************************
** ALL MODEL CHECK PASSED **
****************************************************
************ in TIDL_subgraphRtCreate ************
The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
0.0s: VX_ZONE_INIT:Enabled
0.7s: VX_ZONE_ERROR:Enabled
0.12s: VX_ZONE_WARNING:Enabled
0.2171s: VX_ZONE_INIT:[tivxInit:184] Initialization Done !!!
TIDL_initDebugTraceParams Done
Alg Alloc for Layer # - 0
Alg Alloc for Layer # - 1
Alg Alloc for Layer # - 2
Alg Alloc for Layer # - 3
Alg Alloc for Layer # - 4
Alg Alloc for Layer # - 5
--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum , Space , Attribute , Size(KBytes)
0 , DDR Cacheable, Persistent , 14.84
1 , DDR Cacheable, Persistent , 0.14
2 , DDR Cacheable, Scratch , 16.00
3 , DDR Cacheable, Scratch , 4.00
4 , DDR Cacheable, Scratch , 56.00
5 , DDR Cacheable, Persistent , 258.00
6 , DDR Cacheable, Scratch , 448.62
7 , DDR Cacheable, Scratch , 0.12
8 , DDR Cacheable, Scratch , 96.12
9 , DDR Cacheable, Scratch , 515.00
10 , DDR Cacheable, Scratch , 0.12
11 , DDR Cacheable, Persistent , 201.12
12 , DDR Cacheable, Scratch , 512.25
13 , DDR Cacheable, Persistent , 0.12
14 , DDR Cacheable, Persistent , 675.81
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 2798.29
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
To get the actual TIDL memory requirement make sure to run on EVM with
debugTraceLevel = 2
--------------------------------------------
Alg Init for Layer # - 0 out of 5
Alg Init for Layer # - 1 out of 5
Alg Init for Layer # - 2 out of 5
Alg Init for Layer # - 3 out of 5
Alg Init for Layer # - 4 out of 5
Alg Init for Layer # - 5 out of 5
PREEMPTION: Adding a new priority object for targetPriority = 2, handle = 0x7feee47a3000
PREEMPTION: Now total number of priority objects = 1 at priorityId = 2, with new memRec of base = 0x7fee30017000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x7feee47a3000, return Addr = 0x7fee30017000
************ TIDL_subgraphRtCreate done ************
Warning : Couldn't find corresponding ioBuf tensor for onnx tensor with matching name
******* In TIDL_subgraphRtInvoke ********
TIDL_activate is called with handle : e47a3000
Starting Layer # - 1
Processing Layer # - 1
End of Layer # - 1 with outPtrs[0] = 0x7fee30167000
Starting Layer # - 2
Processing Layer # - 2
End of Layer # - 2 with outPtrs[0] = 0x7fee30177000
Starting Layer # - 3
Processing Layer # - 3
End of Layer # - 3 with outPtrs[0] = 0x7fee30187000
Starting Layer # - 4
Processing Layer # - 4
End of Layer # - 4 with outPtrs[0] = 0x7fee302e0000
TIDL_process is completed with handle : e47a3000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 12.000000 38971.000000 164.000000
******* TIDL_subgraphRtInvoke done ********
********** Frame Index 1 : Running float inference **********
Segmentation fault (core dumped)
Importantly, that if Transpose layer is removed from the onnx model, tidl compilation is successful. Here is the log message of succesfull compilation when "Transpose" layer is removed from model:
tidl_tools_path = /home/root/tidl_tools
artifacts_folder = /home/root/model-artifacts/search/
tidl_tensor_bits = 8
debug_level = 6
num_tidl_subgraphs = 16
tidl_denylist =
tidl_denylist_layer_name =
tidl_denylist_layer_type =
tidl_allowlist_layer_name =
model_type =
tidl_calibration_accuracy_level = 7
tidl_calibration_options:num_frames_calibration = 20
tidl_calibration_options:bias_calibration_iterations = 50
mixed_precision_factor = -1.000000
model_group_id = 0
power_of_2_quantization = 2
enable_high_resolution_optimization = 0
pre_batchnorm_fold = 1
add_data_convert_ops = 0
output_feature_16bit_names_list =
m_params_16bit_names_list =
reserved_compile_constraints_flag = 1601
ti_internal_reserved_1 =
****** WARNING : Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options******
Supported TIDL layer type --- Conv -- /Conv
Supported TIDL layer type --- Reshape -- /connect_model/cls_encode/Reshape
Preliminary subgraphs created = 1
Final number of subgraphs created are : 1, - Offloaded Nodes - 2, Total Nodes - 2
/home/root/tidl_tools/tidl_graphVisualiser_runtimes.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
Running runtimes graphviz - /home/root/tidl_tools/tidl_graphVisualiser_runtimes.out /home/root/model-artifacts/search//allowedNode.txt /home/root/model-artifacts/search//tempDir/graphvizInfo.txt /home/root/model-artifacts/search//tempDir/runtimes_visualization.svg
*** In TIDL_createStateImportFunc ***
Compute on node : TIDLExecutionProvider_TIDL_0_0
0, Conv, 3, 1, template_features, /Conv_output_0
1, Reshape, 2, 1, /Conv_output_0, /connect_model/cls_encode/Reshape_output_0
Input tensor name - template_features
Output tensor name - /connect_model/cls_encode/Reshape_output_0
Initialized
In TIDL_onnxRtImportInit subgraph_name=_connect_model_cls_encode_Reshape_output_0
Layer 0, subgraph id _connect_model_cls_encode_Reshape_output_0, name=/connect_model/cls_encode/Reshape_output_0
Layer 1, subgraph id _connect_model_cls_encode_Reshape_output_0, name=template_features
In TIDL_runtimesOptimizeNet: LayerIndex = 4, dataIndex = 3
************** Frame index 1 : Running float import *************
In TIDL_runtimesPostProcessNet
/home/root/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
****************************************************
** ALL MODEL CHECK PASSED **
****************************************************
************ in TIDL_subgraphRtCreate ************
The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
0.0s: VX_ZONE_INIT:Enabled
0.6s: VX_ZONE_ERROR:Enabled
0.10s: VX_ZONE_WARNING:Enabled
0.1829s: VX_ZONE_INIT:[tivxInit:184] Initialization Done !!!
TIDL_initDebugTraceParams Done
Alg Alloc for Layer # - 0
Alg Alloc for Layer # - 1
Alg Alloc for Layer # - 2
Alg Alloc for Layer # - 3
Alg Alloc for Layer # - 4
Alg Alloc for Layer # - 5
--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum , Space , Attribute , Size(KBytes)
0 , DDR Cacheable, Persistent , 14.84
1 , DDR Cacheable, Persistent , 0.14
2 , DDR Cacheable, Scratch , 16.00
3 , DDR Cacheable, Scratch , 4.00
4 , DDR Cacheable, Scratch , 56.00
5 , DDR Cacheable, Persistent , 258.00
6 , DDR Cacheable, Scratch , 448.62
7 , DDR Cacheable, Scratch , 0.12
8 , DDR Cacheable, Scratch , 96.12
9 , DDR Cacheable, Scratch , 515.00
10 , DDR Cacheable, Scratch , 0.12
11 , DDR Cacheable, Persistent , 201.12
12 , DDR Cacheable, Scratch , 512.25
13 , DDR Cacheable, Persistent , 0.12
14 , DDR Cacheable, Persistent , 675.81
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 2798.29
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
To get the actual TIDL memory requirement make sure to run on EVM with
debugTraceLevel = 2
--------------------------------------------
Alg Init for Layer # - 0 out of 5
Alg Init for Layer # - 1 out of 5
Alg Init for Layer # - 2 out of 5
Alg Init for Layer # - 3 out of 5
Alg Init for Layer # - 4 out of 5
Alg Init for Layer # - 5 out of 5
PREEMPTION: Adding a new priority object for targetPriority = 2, handle = 0x7f8a4b00d000
PREEMPTION: Now total number of priority objects = 1 at priorityId = 2, with new memRec of base = 0x7f8998084000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x7f8a4b00d000, return Addr = 0x7f8998084000
************ TIDL_subgraphRtCreate done ************
Warning : Couldn't find corresponding ioBuf tensor for onnx tensor with matching name
******* In TIDL_subgraphRtInvoke ********
TIDL_activate is called with handle : 4b00d000
Starting Layer # - 1
Processing Layer # - 1
End of Layer # - 1 with outPtrs[0] = 0x7f89981d4000
Starting Layer # - 2
Processing Layer # - 2
End of Layer # - 2 with outPtrs[0] = 0x7f89981e4000
Starting Layer # - 3
Processing Layer # - 3
End of Layer # - 3 with outPtrs[0] = 0x7f89981f4000
Starting Layer # - 4
Processing Layer # - 4
End of Layer # - 4 with outPtrs[0] = 0x7f899834d000
TIDL_process is completed with handle : 4b00d000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 12.000000 38914.000000 179.000000
******* TIDL_subgraphRtInvoke done ********
********** Frame Index 1 : Running float inference **********
[array(0.646822, dtype=float32)]
Video processed and saved to outputs/test.mp4
************ in TIDL_subgraphRtDelete ************
TIDL_deactivate is called with handle : 4b00d000
PREEMPTION: Removing priroty object with handle = 0x7f8a4b00d000 and targetPriority = 2, Number of obejcts left are = 0, removed object with base = 0x7f8998084000 and size =128
MEM: Deinit ... !!!
MEM: Alloc's: 26 alloc's of 4658204 bytes
MEM: Free's : 26 free's of 4658204 bytes
MEM: Open's : 0 allocs of 0 bytes
MEM: Deinit ... Done !!!During the import process next two steps caught my attention:
Unsupported (TIDL check) TIDL layer type --- Transpose
TIDL ALLOWLISTING LAYER CHECK -- [TIDL_TransposeLayer] should be removed in import process. If not, this model will not work!
And I am wondering how the compilation process should be changed ?
