• State Resolved
  • Locked Locked
  • Replies 5 replies
  • Subscribers 98 subscribers
  • Views 371 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SK-AM62A-LP: edgeai-tidl-tool compile error

Takayuki Okamoto
Prodigy 240 points
Part Number: SK-AM62A-LP

We are using a SK-AM62A and are trying out the compile functionality of edgeai-tidl-tools.

The version of edgeai-tidl-tools is 09_01_03_00.

When I converted the attached model using tflrt_delegate.py, it failed as shown in the log below.

How can I avoid this error?

Running 1 Models - ['cl-tfl-wideresnet50_v2']


Running_Model :  cl-tfl-wideresnet50_v2

 Preliminary number of subgraphs:18 , 109 nodes delegated out of 127 nodes 
 
convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting 
WARNING: Conv Layer onnx_tf_prefix_/relu/Relu;Add;convolution;Const_106's coeff cannot be found(or not match) in coef file, Random coeff will be generated! Only for evaluation usage! Results are all random!

 ************** Frame index 1 : Running float import ************* 
****************************************************
**   All the Tensor Dimensions has to be greater then Zero 
**   DIM Error - For Tensor 2, Dim 4 is -1
****************************************************
****************************************************
**                ALL MODEL CHECK PASSED          **
****************************************************

The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INIT:Enabled
 0.32s:  VX_ZONE_ERROR:Enabled
 0.37s:  VX_ZONE_WARNING:Enabled
 0.3102s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!
 0.11113s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:130] size is 0
 0.11133s:  VX_ZONE_ERROR:[ownAllocReferenceBufferGeneric:347] Memory allocation failed
 0.11136s:  VX_ZONE_ERROR:[ownGraphAllocateDataObject:923] Memory allocation for data reference failed
 0.11140s:  VX_ZONE_ERROR:[vxVerifyGraph:2000] Memory alloc for data objects failed
 0.11145s:  VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting 
WARNING: Conv Layer Add_4;convolution_22;convolution_4;Const_98's coeff cannot be found(or not match) in coef file, Random coeff will be generated! Only for evaluation usage! Results are all random!
convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting 
WARNING: Conv Layer onnx_tf_prefix_/layer1/layer1.0/relu/Relu;Add_1;convolution_9;convolution_1;Const_104's coeff cannot be found(or not match) in coef file, Random coeff will be generated! Only for evaluation usage! Results are all random!

 ************** Frame index 1 : Running float import ************* 
****************************************************
**                ALL MODEL CHECK PASSED          **
****************************************************

convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting

1581.model.zip

  • 0
    TI__Genius 15616 points

    Hello Okamoto-san,

    Thank you for your query. I see that you are having difficulty compiling this custom model.

    For starters, would you please pull the more recent version of the 9.1 SDK tidl-tools repo? There is a tagged commit for "09_01_06_00" that includes a few bug fixes for this SDK.

    I will take a look at the model you have supplied. In the meantime, could I ask that you rerun this with more logging enabled on your host machine?

    • In linux environment, run `export TIDL_RT_DEBUG=1` to enable a few more logs
    • In the common_utils.py file in the parent directory, please set debug_level to 2

    And then supply the logs resulting from this, please.

    Before looking at the model, I can see several warnings about model architecture parameters. Please review the list of supported operators to see if there are any configurations in your network that will be unsupported.

    If you cannot change your model architecture, you may use the deny_list (defined in common_utils as well) to designate some layers to run on the Arm A53 CPU. I must note that this will have a performance penalty.

    Best regards,
    Reese

  • 0 in reply to Reese Grimsley
    TI__Genius 15616 points

    On first look, I see some strangeness in that there are Transpose->pad->transpose blocks replicated throughout the network. Is this intentional? I should think the padding can be done without transpose on either end.

    I see layer names that seem to indicate that you exported from ONNX->Tensorflow-lite. Sometimes this adds operations or makes necessary changes. You can directly compile with an ONNX model. Have you tried this? I find ONNX easier to work with in many cases since the graph is more easily parsed and modified, but that's a personal preference.

    Would you please also provide me any SVG files that are within the artifacts directory? Depending on where the compilation exited, these may be produced even if compilation did not complete. You can load these files in a browser to visualize how TIDL parsed your model and produced subgraphs.

    BR,
    Reese

  • 0 in reply to Reese Grimsley
    Prodigy 240 points 
    Hello Grimsley-san,

    Thank you for your quick reply!

    I tried it with the "09_01_06_00" tag, but the same problem occurred.
    I am attaching a log file with "export TIDL_RT_DEBUG=1" and debug_level set to 2.
    This was tried with the "09_01_06_00" tag.
    /content/tidl-tools/edgeai-tidl-tools/examples/osrt_python/tfl
Running 1 Models - ['cl-tfl-wideresnet50_v2']


Running_Model :  cl-tfl-wideresnet50_v2
tidl_tools_path                                 = /content/tidl-tools/edgeai-tidl-tools/tidl_tools 
artifacts_folder                                = ../../../model-artifacts//cl-tfl-wideresnet50_v2/ 
tidl_tensor_bits                                = 8 
debug_level                                     = 2 
num_tidl_subgraphs                              = 16 
tidl_denylist                                   = 
tidl_denylist_layer_name                        = 
tidl_denylist_layer_type                         = 
tidl_allowlist_layer_name                        = 
model_type                                      =  
tidl_calibration_accuracy_level                 = 7 
tidl_calibration_options:num_frames_calibration = 2 
tidl_calibration_options:bias_calibration_iterations = 5 
mixed_precision_factor = -1.000000 
model_group_id = 0 
power_of_2_quantization                         = 2 
ONNX QDQ Enabled                                = 0 
enable_high_resolution_optimization             = 0 
pre_batchnorm_fold                              = 1 
add_data_convert_ops                          = 3 
output_feature_16bit_names_list                 =  
m_params_16bit_names_list                       =  
reserved_compile_constraints_flag               = 1601 
ti_internal_reserved_1                          = 


 ****** WARNING : Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options******

Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---             Pad 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---     transpose_1 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/relu/Relu;Add;convolution;Const_106 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---     transpose_2 
Unsupported (import) TIDL layer type for Tflite layer type --- 60  layer output name---           PadV2 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---     transpose_3 
Supported TIDL layer type --- 2 Tflite layer type --- 17 layer output name---       MaxPool2d 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_4;convolution_22;convolution_4;Const_98 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer1/layer1.0/relu/Relu;Add_1;convolution_9;convolution_1;Const_104 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---     transpose_7 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_1 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---     transpose_9 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer1/layer1.0/relu_1/Relu;Add_2;convolution_9;convolution_2;Const_102 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_3;convolution_22;convolution_3;Const_100 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer1/layer1.0/relu_2/Relu;onnx_tf_prefix_/layer1/layer1.0/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer1/layer1.1/relu/Relu;Add_5;convolution_9;convolution_5;Const_96 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_19 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_2 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_21 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer1/layer1.1/relu_1/Relu;Add_6;convolution_9;convolution_6;Const_94 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_7;convolution_22;convolution_7;Const_92 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer1/layer1.1/relu_2/Relu;onnx_tf_prefix_/layer1/layer1.1/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer1/layer1.2/relu/Relu;Add_8;convolution_9;convolution_8;Const_90 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_28 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_3 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_30 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer1/layer1.2/relu_1/Relu;Add_9;convolution_9;Const_88 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_10;convolution_22;convolution_10;Const_86 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer1/layer1.2/relu_2/Relu;onnx_tf_prefix_/layer1/layer1.2/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.0/relu/Relu;Add_11;convolution_22;convolution_11;Const_84 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_37 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_4 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_39 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.0/relu_1/Relu;Add_12;convolution_22;convolution_12;Const_82 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_13;convolution_41;convolution_13;Const_80 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_14;convolution_41;convolution_14;Const_78 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer2/layer2.0/relu_2/Relu;onnx_tf_prefix_/layer2/layer2.0/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.1/relu/Relu;Add_15;convolution_22;convolution_15;Const_76 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_49 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_5 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_51 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.1/relu_1/Relu;Add_16;convolution_22;convolution_16;Const_74 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_17;convolution_41;convolution_17;Const_72 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer2/layer2.1/relu_2/Relu;onnx_tf_prefix_/layer2/layer2.1/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.2/relu/Relu;Add_18;convolution_22;convolution_18;Const_70 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_58 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_6 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_60 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.2/relu_1/Relu;Add_19;convolution_22;convolution_19;Const_68 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_20;convolution_41;convolution_20;Const_66 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer2/layer2.2/relu_2/Relu;onnx_tf_prefix_/layer2/layer2.2/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.3/relu/Relu;Add_21;convolution_22;convolution_21;Const_64 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_67 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_7 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_69 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer2/layer2.3/relu_1/Relu;Add_22;convolution_22;Const_62 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_23;convolution_41;convolution_23;Const_60 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer2/layer2.3/relu_2/Relu;onnx_tf_prefix_/layer2/layer2.3/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.0/relu/Relu;Add_24;convolution_41;convolution_24;Const_58 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_76 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_8 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_78 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.0/relu_1/Relu;Add_25;convolution_41;convolution_25;Const_56 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_26;convolution_51;convolution_26;Const_54 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_27;convolution_51;convolution_27;Const_52 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer3/layer3.0/relu_2/Relu;onnx_tf_prefix_/layer3/layer3.0/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.1/relu/Relu;Add_28;convolution_41;convolution_28;Const_50 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_88 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---           Pad_9 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_90 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.1/relu_1/Relu;Add_29;convolution_41;convolution_29;Const_48 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_30;convolution_51;convolution_30;Const_46 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer3/layer3.1/relu_2/Relu;onnx_tf_prefix_/layer3/layer3.1/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.2/relu/Relu;Add_31;convolution_41;convolution_31;Const_44 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_97 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---          Pad_10 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---    transpose_99 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.2/relu_1/Relu;Add_32;convolution_41;convolution_32;Const_42 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_33;convolution_51;convolution_33;Const_40 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer3/layer3.2/relu_2/Relu;onnx_tf_prefix_/layer3/layer3.2/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.3/relu/Relu;Add_34;convolution_41;convolution_34;Const_38 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_106 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---          Pad_11 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_108 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.3/relu_1/Relu;Add_35;convolution_41;convolution_35;Const_36 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_36;convolution_51;convolution_36;Const_34 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer3/layer3.3/relu_2/Relu;onnx_tf_prefix_/layer3/layer3.3/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.4/relu/Relu;Add_37;convolution_41;convolution_37;Const_32 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_115 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---          Pad_12 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_117 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.4/relu_1/Relu;Add_38;convolution_41;convolution_38;Const_30 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_39;convolution_51;convolution_39;Const_28 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer3/layer3.4/relu_2/Relu;onnx_tf_prefix_/layer3/layer3.4/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.5/relu/Relu;Add_40;convolution_41;convolution_40;Const_26 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_124 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---          Pad_13 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_126 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer3/layer3.5/relu_1/Relu;Add_41;convolution_41;Const_24 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_42;convolution_51;convolution_42;Const_22 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer3/layer3.5/relu_2/Relu;onnx_tf_prefix_/layer3/layer3.5/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer4/layer4.0/relu/Relu;Add_43;convolution_51;convolution_43;Const_20 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_133 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---          Pad_14 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_135 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer4/layer4.0/relu_1/Relu;Add_44;convolution_51;convolution_44;Const_18 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_45;convolution_52;convolution_45;Const_16 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_46;convolution_52;convolution_46;Const_14 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer4/layer4.0/relu_2/Relu;onnx_tf_prefix_/layer4/layer4.0/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer4/layer4.1/relu/Relu;Add_47;convolution_51;convolution_47;Const_12 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_145 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---          Pad_15 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_147 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer4/layer4.1/relu_1/Relu;Add_48;convolution_51;convolution_48;Const_10 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_49;convolution_52;convolution_49;Const_8 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer4/layer4.1/relu_2/Relu;onnx_tf_prefix_/layer4/layer4.1/Add 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer4/layer4.2/relu/Relu;Add_50;convolution_51;convolution_50;Const_6 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_154 
Unsupported (TIDL check) TIDL layer type --- 26 Tflite layer type --- 34 layer output name---          Pad_16 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_156 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- onnx_tf_prefix_/layer4/layer4.2/relu_1/Relu;Add_51;convolution_51;Const_4 
Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- Add_52;convolution_52;Const_2 
Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- onnx_tf_prefix_/layer4/layer4.2/relu_2/Relu;onnx_tf_prefix_/layer4/layer4.2/Add 
Supported TIDL layer type --- 41 Tflite layer type --- 39 layer output name---   transpose_160 
Supported TIDL layer type --- 2 Tflite layer type --- 40 layer output name---            Mean 
Supported TIDL layer type --- 38 Tflite layer type --- 22 layer output name--- flatten/Reshape;Reshape 
Supported TIDL layer type --- 6 Tflite layer type --- 9 layer output name--- PartitionedCall:0 

 Preliminary number of subgraphs:18 , 109 nodes delegated out of 127 nodes 
 
Padding is only supported in Width/height axis -layer name - Pad -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Layer type not supported by TIDL ----  tflite layer code - 60, tensor name - PadV2  
Padding is only supported in Width/height axis -layer name - Pad_1 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_2 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_3 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_4 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_5 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_6 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_7 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_8 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_9 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_10 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_11 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_12 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_13 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_14 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_15 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
Padding is only supported in Width/height axis -layer name - Pad_16 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410  
In TIDL_tfliteRtImportInit subgraph_id=119
Layer 0, subgraph id 119, name=transpose_2
Layer 1, subgraph id 119, name=Pad
In TIDL_tfliteRtImportNode  TIDL Layer type 41   Tflite builtin code type 39 
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3 
In TIDL_tfliteRtImportNode  TIDL Layer type 41   Tflite builtin code type 39 
In TIDL_runtimesOptimizeNet: LayerIndex = 5, dataIndex = 4 
convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting 
WARNING: Conv Layer onnx_tf_prefix_/relu/Relu;Add;convolution;Const_106's coeff cannot be found(or not match) in coef file, Random coeff will be generated! Only for evaluation usage! Results are all random!

 ************** Frame index 1 : Running float import ************* 
In TIDL_runtimesPostProcessNet 
In TIDL_runtimesPostProcessNet 1
In TIDL_runtimesPostProcessNet 2
In TIDL_runtimesPostProcessNet 3
****************************************************
**   All the Tensor Dimensions has to be greater then Zero 
**   DIM Error - For Tensor 2, Dim 4 is -1
****************************************************
****************************************************
**                ALL MODEL CHECK PASSED          **
****************************************************

In TIDL_runtimesPostProcessNet 4
************ in TIDL_subgraphRtCreate ************ 
 The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INIT:Enabled
 0.13s:  VX_ZONE_ERROR:Enabled
 0.16s:  VX_ZONE_WARNING:Enabled
 0.3509s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!
 0.9839s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:130] size is 0
 0.9855s:  VX_ZONE_ERROR:[ownAllocReferenceBufferGeneric:347] Memory allocation failed
 0.9860s:  VX_ZONE_ERROR:[ownGraphAllocateDataObject:923] Memory allocation for data reference failed
 0.9864s:  VX_ZONE_ERROR:[vxVerifyGraph:2000] Memory alloc for data objects failed
 0.9867s:  VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
************ TIDL_subgraphRtCreate done ************ 
 In TIDL_tfliteRtImportInit subgraph_id=125
Layer 0, subgraph id 125, name=Add_4;convolution_22;convolution_4;Const_98
Layer 1, subgraph id 125, name=transpose_7
Layer 2, subgraph id 125, name=PadV2
In TIDL_tfliteRtImportNode  TIDL Layer type 41   Tflite builtin code type 39 
In TIDL_tfliteRtImportNode  TIDL Layer type 2   Tflite builtin code type 17 
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3 
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3 
In TIDL_tfliteRtImportNode  TIDL Layer type 41   Tflite builtin code type 39 
In TIDL_runtimesOptimizeNet: LayerIndex = 8, dataIndex = 6 
convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting 
WARNING: Conv Layer Add_4;convolution_22;convolution_4;Const_98's coeff cannot be found(or not match) in coef file, Random coeff will be generated! Only for evaluation usage! Results are all random!
convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting 
WARNING: Conv Layer onnx_tf_prefix_/layer1/layer1.0/relu/Relu;Add_1;convolution_9;convolution_1;Const_104's coeff cannot be found(or not match) in coef file, Random coeff will be generated! Only for evaluation usage! Results are all random!

 ************** Frame index 1 : Running float import ************* 
In TIDL_runtimesPostProcessNet 
In TIDL_runtimesPostProcessNet 1
In TIDL_runtimesPostProcessNet 2
In TIDL_runtimesPostProcessNet 3
****************************************************
**                ALL MODEL CHECK PASSED          **
****************************************************

In TIDL_runtimesPostProcessNet 4
************ in TIDL_subgraphRtCreate ************ 
 
--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  128, 15.25   , 0x00000000
1           , DDR Cacheable       , Persistent  ,  128, 0.64    , 0x00000000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0x00000000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0x00000000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0x00000000
5           , DDR Cacheable       , Persistent  ,  128, 261.75  , 0x00000000
6           , DDR Cacheable       , Scratch     ,  128, 10282.95, 0x00000000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0x00000000
8           , DDR Cacheable       , Scratch     ,  128, 2604.12 , 0x00000000
9           , DDR Cacheable       , Scratch     ,  128, 6865.27 , 0x00000000
10          , DDR Cacheable       , Persistent  ,  128, 258.58  , 0x00000000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0x00000000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0x00000000
13          , DDR Cacheable       , Persistent  ,  128, 3074.44 , 0x00000000
14          , DDR Cacheable       , Persistent  ,  128, 0.01    , 0x00000000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 23951.51
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
TIDL init call from ivision API 

--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum   , Space               , Attribute   , Alignment   , Size(KBytes), BasePtr     
0           , DDR Cacheable       , Persistent  ,  128, 15.25   , 0xf0443000
1           , DDR Cacheable       , Persistent  ,  128, 0.64    , 0xf0442000
2           , DDR Cacheable       , Scratch     ,  128, 16.00   , 0xf043e000
3           , DDR Cacheable       , Scratch     ,  128, 4.00    , 0xf043d000
4           , DDR Cacheable       , Scratch     ,  128, 56.00   , 0xf042f000
5           , DDR Cacheable       , Persistent  ,  128, 261.75  , 0xedcbe000
6           , DDR Cacheable       , Scratch     ,  128, 10282.95, 0xb35f5000
7           , DDR Cacheable       , Scratch     ,  128, 0.12    , 0xf042e000
8           , DDR Cacheable       , Scratch     ,  128, 2604.12 , 0xb86d0000
9           , DDR Cacheable       , Scratch     ,  128, 6865.27 , 0xb801b000
10          , DDR Cacheable       , Persistent  ,  128, 258.58  , 0xedc7d000
11          , DDR Cacheable       , Scratch     ,  128, 512.25  , 0xb3574000
12          , DDR Cacheable       , Persistent  ,  128, 0.12    , 0xf042d000
13          , DDR Cacheable       , Persistent  ,  128, 3074.44 , 0xb3273000
14          , DDR Cacheable       , Persistent  ,  128, 0.01    , 0xef432000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 23951.51
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
      To get the actual TIDL memory requirement make sure to run on EVM with 
      debugTraceLevel = 2

--------------------------------------------
Alg Init for Layer # -    1
Alg Init for Layer # -    2
Alg Init for Layer # -    3
Alg Init for Layer # -    4
Alg Init for Layer # -    5
Alg Init for Layer # -    7
Alg Init for Layer # -    8
Alg Init for Layer # -    9
Alg Init for Layer # -   10
Alg Init for Layer # -   11
PREEMPTION: Adding a new priority object for targetPriority = 0, handle = 0x7ebef0443000
PREEMPTION: Now total number of priority objects = 1 at priorityId = 0,    with new memRec of base = 0x7ebef042d000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x7ebef0443000, return Addr = 0x7ebee5cb4438
************ TIDL_subgraphRtCreate done ************ 
 In TIDL_tfliteRtImportInit subgraph_id=132
Layer 0, subgraph id 132, name=onnx_tf_prefix_/layer1/layer1.0/relu_2/Relu;onnx_tf_prefix_/layer1/layer1.0/Add
Layer 1, subgraph id 132, name=transpose_19
Layer 2, subgraph id 132, name=Add_4;convolution_22;convolution_4;Const_98
Layer 3, subgraph id 132, name=Pad_1
In TIDL_tfliteRtImportNode  TIDL Layer type 41   Tflite builtin code type 39 
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3 
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3 
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0 
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3 
In TIDL_tfliteRtImportNode  TIDL Layer type 41   Tflite builtin code type 39 
In TIDL_runtimesOptimizeNet: LayerIndex = 10, dataIndex = 8 
convParams.numInChannels Is not multiple of convParams.numGroups -  Exiting 


    I understand that by using deny_list, it is possible to avoid the error.
    As you said, this model is exported from an ONNX format to Tensorflow-lite format.
    Since a transpose was unintentionally added during conversion, we will review the conversion process.
    I would have liked to check it in Tensorflow-lite format for comparison, but I agree that the ONNX format is more convenient.

    I am attaching the files in tempDir that are generated during compilation.
    tempDir.tar.gz

    Best Regards,
    Takayuki
  • +1 in reply to Takayuki Okamoto
    TI__Genius 15616 points

    Hello,

    Thank you for including this information. I can better see the issue now. It looks like all the Pad layers are marked as unsupported, most likely due to some parameters for that layer, since they are otherwise supported (although I think Padv2 is not supported in general).

    By typical convention, Onnx formats tensors as NCHW and Tensorflow-lite uses NHWC for it's dimensions. Some of our model-parsing code makes this assumption.

    What I believe is happening, is that the transpose causes the tensor dimensions to be in an unexpected order.

    Padding is only supported in Width/height axis -layer name - Pad_2 -- file info - tidl_tfLiteRtImport_core.cpp , TIDL_tfliteRtMapPadParams , 1410

    So when the transpose happens and our import tool assumes NHWC tensor ordering, it assumes that this Pad  layer is operating on the width and channel dimensions. Per the message above, padding is only supported along the height and width channels. Technically, that's what this network actually does, but the import tool doesn't recognize this due to the Transpose layer.

    I also see some messages that indicate some tensor weights could not be found (e.g. Add_4;convolution_22;convolution_4;Const_98). I'm not sure why this is. I am curious if this is also perhaps caused by export to TFLite. I assume the TFLite model runs on its own (can test with tflrt_delegate.py -d. The -d tag disables TIDL delegates).

    My recommendation here is to try compling the ONNX model (and perhaps you already have) itself. I am concerned that the results of ONNX->TF conversion are creating corner cases that our import tool is not accustomed to, even if the model is technically valid.

    BR,
    Reese

  • 0 in reply to Reese Grimsley
    Prodigy 240 points

    Hello,

    Thank you for your confirmation.

    I understand the problem is that the trasnpose and pad layers inserted during the conversion to TFLite are not supported with compilation.

    Regarding the message indicating that tensor weights cannot be found, I am not sure what the cause is, as I have confirmed that the TFLite model can run on its own.

    The ONNX model was compiled successfully, so we will proceed with the ONNX model for now.

    Thank you for your assistance.

    Best Regards,

    Takayuki