TDA4VM: DL model compile failure using edgeai_tidl_tools

Hi,

Could you try open debug_level=3 to output much more log? Thanks~

Thank you for your reply, here is the logs output by -> *debug_level=3*, I still have no any clue about how to solve this problem, TAT

The size of my onnxfile is 53.1MB, 

=====

input_names: [input]
output_names: [output]
Initing onnxrt ...
libtidl_onnxrt_EP loaded 0x56441328cf30
artifacts_folder = ./artifacts
debug_level = 3
target_priority = 0
max_pre_empt_delay = 340282346638528859811704183484516925440.000000

******** WARNING ******* : Could not open ./artifacts/allowedNode.txt for reading... Entire model will run on ARM without any delegation to TIDL !
Final number of subgraphs created are : 1, - Offloaded Nodes - 0, Total Nodes - 0
Running onnxrt ...
input_names: [input]
tidl_tools_path = ${my_local_folder}/edgeai-tidl-tools/tidl_tools
artifacts_folder = ./artifacts
tidl_tensor_bits = 8
debug_level = 3
num_tidl_subgraphs = 16
tidl_denylist =
tidl_denylist_layer_name =
tidl_denylist_layer_type =
model_type = -1
tidl_calibration_accuracy_level = 7
tidl_calibration_options:num_frames_calibration = 1
tidl_calibration_options:bias_calibration_iterations = 5
mixed_precision_factor = -1.000000
model_group_id = 0
power_of_2_quantization = 2
enable_high_resolution_optimization = 1
pre_batchnorm_fold = 1
add_data_convert_ops = 0
output_feature_16bit_names_list =
m_params_16bit_names_list =
reserved_compile_constraints_flag = 1601
ti_internal_reserved_1 =

****** WARNING : Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options******

Supported TIDL layer type --- Conv -- /model/backbone/input/input.0/Conv
Supported TIDL layer type --- Relu -- /model/backbone/input/input.2/Relu
Supported TIDL layer type --- MaxPool -- /model/backbone/input/input.3/MaxPool
Supported TIDL layer type --- Conv -- /model/backbone/layer1/layer1.0/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer1/layer1.0/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer1/layer1.0/conv2/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer1/layer1.0/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer1/layer1.0/relu_1/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer1/layer1.1/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer1/layer1.1/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer1/layer1.1/conv2/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer1/layer1.1/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer1/layer1.1/relu_1/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer2/layer2.0/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer2/layer2.0/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer2/layer2.0/conv2/Conv
Supported TIDL layer type --- Conv -- /model/backbone/layer2/layer2.0/downsample/downsample.0/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer2/layer2.0/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer2/layer2.0/relu_1/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer2/layer2.1/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer2/layer2.1/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer2/layer2.1/conv2/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer2/layer2.1/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer2/layer2.1/relu_1/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer3/layer3.0/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer3/layer3.0/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer3/layer3.0/conv2/Conv
Supported TIDL layer type --- Conv -- /model/backbone/layer3/layer3.0/downsample/downsample.0/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer3/layer3.0/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer3/layer3.0/relu_1/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer3/layer3.1/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer3/layer3.1/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer3/layer3.1/conv2/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer3/layer3.1/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer3/layer3.1/relu_1/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer4/layer4.0/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer4/layer4.0/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer4/layer4.0/conv2/Conv
Supported TIDL layer type --- Conv -- /model/backbone/layer4/layer4.0/downsample/downsample.0/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer4/layer4.0/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer4/layer4.0/relu_1/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer4/layer4.1/conv1/Conv
Supported TIDL layer type --- Relu -- /model/backbone/layer4/layer4.1/relu/Relu
Supported TIDL layer type --- Conv -- /model/backbone/layer4/layer4.1/conv2/Conv
Supported TIDL layer type --- Add -- /model/backbone/layer4/layer4.1/Add
Supported TIDL layer type --- Relu -- /model/backbone/layer4/layer4.1/relu_1/Relu
Supported TIDL layer type --- Relu -- /model/backbone/relu/Relu
Unsupported (import) TIDL layer type for ONNX op type --- ReduceMean
Supported TIDL layer type --- Conv -- /model/head/cp/conv_avg/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/cp/conv_avg/relu/Relu
Supported TIDL layer type --- Conv -- /model/head/cp/arm32/conv/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/cp/arm32/conv/relu/Relu
Unsupported (import) TIDL layer type for ONNX op type --- ReduceMean
Supported TIDL layer type --- Conv -- /model/head/cp/arm32/conv_atten/Conv
Supported TIDL layer type --- Sigmoid -- /model/head/cp/arm32/Sigmoid
Unsupported (TIDL check) TIDL layer type --- Mul
Unsupported (TIDL check) TIDL layer type --- Add
Supported TIDL layer type --- Resize -- /model/head/cp/up32/Resize
Supported TIDL layer type --- Conv -- /model/head/cp/conv_head32/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/cp/conv_head32/relu/Relu
Supported TIDL layer type --- Conv -- /model/head/cp/arm16/conv/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/cp/arm16/conv/relu/Relu
Unsupported (import) TIDL layer type for ONNX op type --- ReduceMean
Supported TIDL layer type --- Conv -- /model/head/cp/arm16/conv_atten/Conv
Supported TIDL layer type --- Sigmoid -- /model/head/cp/arm16/Sigmoid
Unsupported (TIDL check) TIDL layer type --- Mul
Supported TIDL layer type --- Add -- /model/head/cp/Add_1
Supported TIDL layer type --- Resize -- /model/head/cp/up16/Resize
Supported TIDL layer type --- Conv -- /model/head/cp/conv_head16/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/cp/conv_head16/relu/Relu
Supported TIDL layer type --- Conv -- /model/head/sp/conv1/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/sp/conv1/relu/Relu
Supported TIDL layer type --- Conv -- /model/head/sp/conv2/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/sp/conv2/relu/Relu
Supported TIDL layer type --- Conv -- /model/head/sp/conv3/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/sp/conv3/relu/Relu
Supported TIDL layer type --- Conv -- /model/head/sp/conv_out/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/sp/conv_out/relu/Relu
Supported TIDL layer type --- Concat -- /model/head/ffm/Concat
Supported TIDL layer type --- Conv -- /model/head/ffm/convblk/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/ffm/convblk/relu/Relu
Unsupported (import) TIDL layer type for ONNX op type --- ReduceMean
Supported TIDL layer type --- Conv -- /model/head/ffm/conv/Conv
Supported TIDL layer type --- Sigmoid -- /model/head/ffm/Sigmoid
Unsupported (TIDL check) TIDL layer type --- Mul
Supported TIDL layer type --- Add -- /model/head/ffm/Add
Supported TIDL layer type --- Conv -- /model/head/conv_out/conv/conv/Conv
Supported TIDL layer type --- Relu -- /model/head/conv_out/conv/relu/Relu
Supported TIDL layer type --- Conv -- /model/head/conv_out/conv_out/Conv
Supported TIDL layer type --- Resize -- /model/head/conv_out/up/Resize
Supported TIDL layer type --- Sigmoid -- /sigmoid/Sigmoid

Preliminary subgraphs created = 8
Final number of subgraphs created are : 8, - Offloaded Nodes - 83, Total Nodes - 91
Layer type not supported by TIDL --- layer type - ReduceMean, Node name -/model/head/cp/ReduceMean
Layer type not supported by TIDL --- layer type - ReduceMean, Node name -/model/head/cp/arm32/ReduceMean
ALLOWLISTING : ADD layer : Only elementwise add operator supported if none of the inputs to add layer is a constant -- file info - tidl_import_common_model_check.cpp , TIDL_checkAddAndMulTensorProperties , 236
ALLOWLISTING : ADD layer : Only elementwise add operator supported if none of the inputs to add layer is a constant -- file info - tidl_import_common_model_check.cpp , TIDL_checkAddAndMulTensorProperties , 236
INFORMATION -- [TIDL_ResizeLayer] Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
Layer type not supported by TIDL --- layer type - ReduceMean, Node name -/model/head/cp/arm16/ReduceMean
ALLOWLISTING : ADD layer : Only elementwise add operator supported if none of the inputs to add layer is a constant -- file info - tidl_import_common_model_check.cpp , TIDL_checkAddAndMulTensorProperties , 236
INFORMATION -- [TIDL_ResizeLayer] Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
Layer type not supported by TIDL --- layer type - ReduceMean, Node name -/model/head/ffm/ReduceMean
ALLOWLISTING : ADD layer : Only elementwise add operator supported if none of the inputs to add layer is a constant -- file info - tidl_import_common_model_check.cpp , TIDL_checkAddAndMulTensorProperties , 236
SUGGESTION -- [TIDL_ResizeLayer] Resize kernel with non-power of 2 resize ratio is not optimal.
Running runtimes graphviz - ${my_local_folder}/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser_runtimes.out ./artifacts/allowedNode.txt ./artifacts/tempDir/graphvizInfo.txt ./artifacts/tempDir/runtimes_visualization.svg
*** In TIDL_createStateImportFunc ***
Compute on node : TIDLExecutionProvider_TIDL_0_0
0, Conv, 3, 1, input, /model/backbone/input/input.0/Conv_output_0
1, Relu, 1, 1, /model/backbone/input/input.0/Conv_output_0, /model/backbone/input/input.2/Relu_output_0
2, MaxPool, 1, 1, /model/backbone/input/input.2/Relu_output_0, /model/backbone/input/input.3/MaxPool_output_0
3, Conv, 3, 1, /model/backbone/input/input.3/MaxPool_output_0, /model/backbone/layer1/layer1.0/conv1/Conv_output_0
4, Relu, 1, 1, /model/backbone/layer1/layer1.0/conv1/Conv_output_0, /model/backbone/layer1/layer1.0/relu/Relu_output_0
5, Conv, 3, 1, /model/backbone/layer1/layer1.0/relu/Relu_output_0, /model/backbone/layer1/layer1.0/conv2/Conv_output_0
6, Add, 2, 1, /model/backbone/layer1/layer1.0/conv2/Conv_output_0, /model/backbone/layer1/layer1.0/Add_output_0
7, Relu, 1, 1, /model/backbone/layer1/layer1.0/Add_output_0, /model/backbone/layer1/layer1.0/relu_1/Relu_output_0
8, Conv, 3, 1, /model/backbone/layer1/layer1.0/relu_1/Relu_output_0, /model/backbone/layer1/layer1.1/conv1/Conv_output_0
9, Relu, 1, 1, /model/backbone/layer1/layer1.1/conv1/Conv_output_0, /model/backbone/layer1/layer1.1/relu/Relu_output_0
10, Conv, 3, 1, /model/backbone/layer1/layer1.1/relu/Relu_output_0, /model/backbone/layer1/layer1.1/conv2/Conv_output_0
11, Add, 2, 1, /model/backbone/layer1/layer1.1/conv2/Conv_output_0, /model/backbone/layer1/layer1.1/Add_output_0
12, Relu, 1, 1, /model/backbone/layer1/layer1.1/Add_output_0, /model/backbone/layer1/layer1.1/relu_1/Relu_output_0
13, Conv, 3, 1, /model/backbone/layer1/layer1.1/relu_1/Relu_output_0, /model/backbone/layer2/layer2.0/conv1/Conv_output_0
14, Relu, 1, 1, /model/backbone/layer2/layer2.0/conv1/Conv_output_0, /model/backbone/layer2/layer2.0/relu/Relu_output_0
15, Conv, 3, 1, /model/backbone/layer2/layer2.0/relu/Relu_output_0, /model/backbone/layer2/layer2.0/conv2/Conv_output_0
16, Conv, 3, 1, /model/backbone/layer1/layer1.1/relu_1/Relu_output_0, /model/backbone/layer2/layer2.0/downsample/downsample.0/Conv_output_0
17, Add, 2, 1, /model/backbone/layer2/layer2.0/conv2/Conv_output_0, /model/backbone/layer2/layer2.0/Add_output_0
18, Relu, 1, 1, /model/backbone/layer2/layer2.0/Add_output_0, /model/backbone/layer2/layer2.0/relu_1/Relu_output_0
19, Conv, 3, 1, /model/backbone/layer2/layer2.0/relu_1/Relu_output_0, /model/backbone/layer2/layer2.1/conv1/Conv_output_0
20, Relu, 1, 1, /model/backbone/layer2/layer2.1/conv1/Conv_output_0, /model/backbone/layer2/layer2.1/relu/Relu_output_0
21, Conv, 3, 1, /model/backbone/layer2/layer2.1/relu/Relu_output_0, /model/backbone/layer2/layer2.1/conv2/Conv_output_0
22, Add, 2, 1, /model/backbone/layer2/layer2.1/conv2/Conv_output_0, /model/backbone/layer2/layer2.1/Add_output_0
23, Relu, 1, 1, /model/backbone/layer2/layer2.1/Add_output_0, /model/backbone/layer2/layer2.1/relu_1/Relu_output_0
24, Conv, 3, 1, /model/backbone/layer2/layer2.1/relu_1/Relu_output_0, /model/backbone/layer3/layer3.0/conv1/Conv_output_0
25, Relu, 1, 1, /model/backbone/layer3/layer3.0/conv1/Conv_output_0, /model/backbone/layer3/layer3.0/relu/Relu_output_0
26, Conv, 3, 1, /model/backbone/layer3/layer3.0/relu/Relu_output_0, /model/backbone/layer3/layer3.0/conv2/Conv_output_0
27, Conv, 3, 1, /model/backbone/layer2/layer2.1/relu_1/Relu_output_0, /model/backbone/layer3/layer3.0/downsample/downsample.0/Conv_output_0
28, Add, 2, 1, /model/backbone/layer3/layer3.0/conv2/Conv_output_0, /model/backbone/layer3/layer3.0/Add_output_0
29, Relu, 1, 1, /model/backbone/layer3/layer3.0/Add_output_0, /model/backbone/layer3/layer3.0/relu_1/Relu_output_0
30, Conv, 3, 1, /model/backbone/layer3/layer3.0/relu_1/Relu_output_0, /model/backbone/layer3/layer3.1/conv1/Conv_output_0
31, Relu, 1, 1, /model/backbone/layer3/layer3.1/conv1/Conv_output_0, /model/backbone/layer3/layer3.1/relu/Relu_output_0
32, Conv, 3, 1, /model/backbone/layer3/layer3.1/relu/Relu_output_0, /model/backbone/layer3/layer3.1/conv2/Conv_output_0
33, Add, 2, 1, /model/backbone/layer3/layer3.1/conv2/Conv_output_0, /model/backbone/layer3/layer3.1/Add_output_0
34, Relu, 1, 1, /model/backbone/layer3/layer3.1/Add_output_0, /model/backbone/layer3/layer3.1/relu_1/Relu_output_0
35, Conv, 3, 1, /model/backbone/layer3/layer3.1/relu_1/Relu_output_0, /model/head/cp/arm16/conv/conv/Conv_output_0
36, Relu, 1, 1, /model/head/cp/arm16/conv/conv/Conv_output_0, /model/head/cp/arm16/conv/relu/Relu_output_0
37, ReduceMean, 1, 1, /model/head/cp/arm16/conv/relu/Relu_output_0, /model/head/cp/arm16/ReduceMean_output_0
38, Conv, 3, 1, /model/head/cp/arm16/ReduceMean_output_0, /model/head/cp/arm16/conv_atten/Conv_output_0
39, Conv, 3, 1, input, /model/head/sp/conv1/conv/Conv_output_0
40, Relu, 1, 1, /model/head/sp/conv1/conv/Conv_output_0, /model/head/sp/conv1/relu/Relu_output_0
41, Conv, 3, 1, /model/head/sp/conv1/relu/Relu_output_0, /model/head/sp/conv2/conv/Conv_output_0
42, Relu, 1, 1, /model/head/sp/conv2/conv/Conv_output_0, /model/head/sp/conv2/relu/Relu_output_0
43, Conv, 3, 1, /model/head/sp/conv2/relu/Relu_output_0, /model/head/sp/conv3/conv/Conv_output_0
44, Relu, 1, 1, /model/head/sp/conv3/conv/Conv_output_0, /model/head/sp/conv3/relu/Relu_output_0
45, Conv, 3, 1, /model/head/sp/conv3/relu/Relu_output_0, /model/head/sp/conv_out/conv/Conv_output_0
46, Relu, 1, 1, /model/head/sp/conv_out/conv/Conv_output_0, /model/head/sp/conv_out/relu/Relu_output_0

Input tensor name - input
Output tensor name - /model/head/sp/conv_out/relu/Relu_output_0
Output tensor name - /model/backbone/layer3/layer3.1/relu_1/Relu_output_0
Output tensor name - /model/head/cp/arm16/conv/relu/Relu_output_0
Output tensor name - /model/head/cp/arm16/conv_atten/Conv_output_0
*** In TIDL_createStateImportFunc ***
Compute on node : TIDLExecutionProvider_TIDL_1_1
0, Conv, 3, 1, /model/backbone/layer3/layer3.1/relu_1/Relu_output_0, /model/backbone/layer4/layer4.0/conv1/Conv_output_0
1, Relu, 1, 1, /model/backbone/layer4/layer4.0/conv1/Conv_output_0, /model/backbone/layer4/layer4.0/relu/Relu_output_0
2, Conv, 3, 1, /model/backbone/layer3/layer3.1/relu_1/Relu_output_0, /model/backbone/layer4/layer4.0/downsample/downsample.0/Conv_output_0
3, Mul, 2, 1, /model/head/cp/arm16/conv/relu/Relu_output_0, /model/head/cp/arm16/Mul_output_0
*** buffer overflow detected ***: python terminated
Aborted (core dumped)

=====

Hi 
Seems you are follow https://github.com/TexasInstruments/edgeai-tidl-tools to import model. This tools only support TDA4VM&AM62a now. According to your log, there is a waring need be solved "Could not open ./artifacts/allowedNode.txt for reading... Entire model will run on ARM without any delegation to TIDL !". It indicate all of layer will not accelerate on C7X&MMA. So in order to accelerate CNN inference, we need use stand alone TIDL to deploy your model. Reference below guide: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_02_00_05/exports/docs/tidl_j721e_08_02_00_11/ti_dl/docs/user_guide_html/md_tidl_model_import.html

Hi

I convert and run the other model on TDA4 successfully by using edgeai-tidl-tools. But BiSeNet is still failed to convert, Thank you!

Hi, Songhua,

Please use the TDA4 related tools for your development. There is no guarantee for a tool developed for one SOC can be directly used for another SOC. Will close this ticket and if there are any issue when using TDA4 tools, please submit a new one. Thanks.

Br, Tommy