TDA4VM: Edgeai-tidl-tools fails to compile the yolov8n-obb model

Part Number: TDA4VM

Tool/software:

Compiler: edgeai-tidl-tools (version 09_02_07_00)

Steps:

1. Place the model in the $edgeai-tidl-tools$/models/public folder

2.Modify $edgeai-tidl-tools$/examples/osrt_python/model_config.py

'yolov8n_20240715_best_test_sim' : {
        'model_path' : os.path.join(models_base_path, 'best_sim.onnx'),
        'mean': [0, 0, 0],
        'scale' : [0.003921568627,0.003921568627,0.003921568627],
        'num_images' : numImages,
        'num_classes': 11,
        'model_type': 'od',
        'od_type' : 'YoloV5',//The relevant post-processing for yolov8 has not yet been written
        'framework' : '',
        'session_name' : 'onnxrt' ,
        # 'meta_arch_type' :-1
    },

3.Modify $edgeai-tidl-tools$/examples/osrt_python/ort/onnxrt_ep.py

models = ["yolov8n_20240715_best_test_sim"]

4.Start the docker virtual machine and cd it to /home/root/

5.Execute the script:

CURDIR=`pwd`
export SOC=am68pa
export TIDL_TOOLS_PATH=$(pwd)/tidl_tools
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TIDL_TOOLS_PATH
export ARM64_GCC_PATH=$(pwd)/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu
cd $CURDIR/examples/osrt_python/ort
python3 onnxrt_ep.py -c

Screenshot of the onnx model:

Screenshot of the run:

  • Hi,

    Would you be able to attach the compilation logs with debug_level = 2 as a txt file so I can see all of the debug information? 

    If you are also able to attached the model as an ONNX file, that would be helpful. Unfortunately the screenshot is not readable on my end.

    Best,

    Asha

  • Hello, how to set the log level?  

  • Hi,

    This is part of settings that you can adjust in common_utils.py

    Best,

    Asha

  • Hi,I changed edgeai-tidl-tools to version 09_02_09_00. But there are still other problems.I set the log level=4.

    root@d0b9e9e2fd62:/home/root# source ./scripts/run.sh 
    Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
    
    Running_Model :  yolov8n  
    
    
    Running shape inference on model ../../../models/public/yolov8n.onnx 
    
    tidl_tools_path                                 = /home/root/tidl_tools 
    artifacts_folder                                = ../../../model-artifacts//yolov8n/ 
    tidl_tensor_bits                                = 8 
    debug_level                                     = 4 
    num_tidl_subgraphs                              = 1 
    tidl_denylist                                   = 
    tidl_denylist_layer_name                        = 
    tidl_denylist_layer_type                         = 
    tidl_allowlist_layer_name                        = 
    model_type                                      =  
    tidl_calibration_accuracy_level                 = 7 
    tidl_calibration_options:num_frames_calibration = 2 
    tidl_calibration_options:bias_calibration_iterations = 5 
    mixed_precision_factor = -1.000000 
    model_group_id = 0 
    power_of_2_quantization                         = 2 
    ONNX QDQ Enabled                                = 0 
    enable_high_resolution_optimization             = 0 
    pre_batchnorm_fold                              = 1 
    add_data_convert_ops                            = 3 
    output_feature_16bit_names_list                 =  
    m_params_16bit_names_list                       =  
    m_single_core_layers_names_list                    =  
    reserved_compile_constraints_flag               = 1601 
    ti_internal_reserved_1                          = 
    
    
     ****** WARNING : Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options******
    
    Supported TIDL layer type ---            Conv -- Conv_0 
    Supported TIDL layer type ---            Relu -- Relu_1 
    Supported TIDL layer type ---            Conv -- Conv_2 
    Supported TIDL layer type ---            Relu -- Relu_3 
    Supported TIDL layer type ---            Conv -- Conv_4 
    Supported TIDL layer type ---            Relu -- Relu_5 
    Supported TIDL layer type ---           Split -- Split_6 
    Supported TIDL layer type ---            Conv -- Conv_7 
    Supported TIDL layer type ---            Relu -- Relu_8 
    Supported TIDL layer type ---            Conv -- Conv_9 
    Supported TIDL layer type ---            Relu -- Relu_10 
    Supported TIDL layer type ---             Add -- Add_11 
    Supported TIDL layer type ---          Concat -- Concat_12 
    Supported TIDL layer type ---            Conv -- Conv_13 
    Supported TIDL layer type ---            Relu -- Relu_14 
    Supported TIDL layer type ---            Conv -- Conv_15 
    Supported TIDL layer type ---            Relu -- Relu_16 
    Supported TIDL layer type ---            Conv -- Conv_17 
    Supported TIDL layer type ---            Relu -- Relu_18 
    Supported TIDL layer type ---           Split -- Split_19 
    Supported TIDL layer type ---            Conv -- Conv_20 
    Supported TIDL layer type ---            Relu -- Relu_21 
    Supported TIDL layer type ---            Conv -- Conv_22 
    Supported TIDL layer type ---            Relu -- Relu_23 
    Supported TIDL layer type ---             Add -- Add_24 
    Supported TIDL layer type ---            Conv -- Conv_25 
    Supported TIDL layer type ---            Relu -- Relu_26 
    Supported TIDL layer type ---            Conv -- Conv_27 
    Supported TIDL layer type ---            Relu -- Relu_28 
    Supported TIDL layer type ---             Add -- Add_29 
    Supported TIDL layer type ---          Concat -- Concat_30 
    Supported TIDL layer type ---            Conv -- Conv_31 
    Supported TIDL layer type ---            Relu -- Relu_32 
    Supported TIDL layer type ---            Conv -- Conv_33 
    Supported TIDL layer type ---            Relu -- Relu_34 
    Supported TIDL layer type ---            Conv -- Conv_35 
    Supported TIDL layer type ---            Relu -- Relu_36 
    Supported TIDL layer type ---           Split -- Split_37 
    Supported TIDL layer type ---            Conv -- Conv_38 
    Supported TIDL layer type ---            Relu -- Relu_39 
    Supported TIDL layer type ---            Conv -- Conv_40 
    Supported TIDL layer type ---            Relu -- Relu_41 
    Supported TIDL layer type ---             Add -- Add_42 
    Supported TIDL layer type ---            Conv -- Conv_43 
    Supported TIDL layer type ---            Relu -- Relu_44 
    Supported TIDL layer type ---            Conv -- Conv_45 
    Supported TIDL layer type ---            Relu -- Relu_46 
    Supported TIDL layer type ---             Add -- Add_47 
    Supported TIDL layer type ---          Concat -- Concat_48 
    Supported TIDL layer type ---            Conv -- Conv_49 
    Supported TIDL layer type ---            Relu -- Relu_50 
    Supported TIDL layer type ---            Conv -- Conv_51 
    Supported TIDL layer type ---            Relu -- Relu_52 
    Supported TIDL layer type ---            Conv -- Conv_53 
    Supported TIDL layer type ---            Relu -- Relu_54 
    Supported TIDL layer type ---           Split -- Split_55 
    Supported TIDL layer type ---            Conv -- Conv_56 
    Supported TIDL layer type ---            Relu -- Relu_57 
    Supported TIDL layer type ---            Conv -- Conv_58 
    Supported TIDL layer type ---            Relu -- Relu_59 
    Supported TIDL layer type ---             Add -- Add_60 
    Supported TIDL layer type ---          Concat -- Concat_61 
    Supported TIDL layer type ---            Conv -- Conv_62 
    Supported TIDL layer type ---            Relu -- Relu_63 
    Supported TIDL layer type ---            Conv -- Conv_64 
    Supported TIDL layer type ---            Relu -- Relu_65 
    Unsupported (TIDL check) TIDL layer type ---         MaxPool 
    Unsupported (TIDL check) TIDL layer type ---         MaxPool 
    Unsupported (TIDL check) TIDL layer type ---         MaxPool 
    Supported TIDL layer type ---          Concat -- Concat_69 
    Supported TIDL layer type ---            Conv -- Conv_70 
    Supported TIDL layer type ---            Relu -- Relu_71 
    Supported TIDL layer type ---          Resize -- Resize_73 
    Supported TIDL layer type ---          Concat -- Concat_74 
    Supported TIDL layer type ---            Conv -- Conv_75 
    Supported TIDL layer type ---            Relu -- Relu_76 
    Supported TIDL layer type ---           Split -- Split_77 
    Supported TIDL layer type ---            Conv -- Conv_78 
    Supported TIDL layer type ---            Relu -- Relu_79 
    Supported TIDL layer type ---            Conv -- Conv_80 
    Supported TIDL layer type ---            Relu -- Relu_81 
    Supported TIDL layer type ---          Concat -- Concat_82 
    Supported TIDL layer type ---            Conv -- Conv_83 
    Supported TIDL layer type ---            Relu -- Relu_84 
    Supported TIDL layer type ---          Resize -- Resize_86 
    Supported TIDL layer type ---          Concat -- Concat_87 
    Supported TIDL layer type ---            Conv -- Conv_88 
    Supported TIDL layer type ---            Relu -- Relu_89 
    Supported TIDL layer type ---           Split -- Split_90 
    Supported TIDL layer type ---            Conv -- Conv_91 
    Supported TIDL layer type ---            Relu -- Relu_92 
    Supported TIDL layer type ---            Conv -- Conv_93 
    Supported TIDL layer type ---            Relu -- Relu_94 
    Supported TIDL layer type ---          Concat -- Concat_95 
    Supported TIDL layer type ---            Conv -- Conv_96 
    Supported TIDL layer type ---            Relu -- Relu_97 
    Supported TIDL layer type ---            Conv -- Conv_98 
    Supported TIDL layer type ---            Relu -- Relu_99 
    Supported TIDL layer type ---          Concat -- Concat_100 
    Supported TIDL layer type ---            Conv -- Conv_101 
    Supported TIDL layer type ---            Relu -- Relu_102 
    Supported TIDL layer type ---           Split -- Split_103 
    Supported TIDL layer type ---            Conv -- Conv_104 
    Supported TIDL layer type ---            Relu -- Relu_105 
    Supported TIDL layer type ---            Conv -- Conv_106 
    Supported TIDL layer type ---            Relu -- Relu_107 
    Supported TIDL layer type ---          Concat -- Concat_108 
    Supported TIDL layer type ---            Conv -- Conv_109 
    Supported TIDL layer type ---            Relu -- Relu_110 
    Supported TIDL layer type ---            Conv -- Conv_111 
    Supported TIDL layer type ---            Relu -- Relu_112 
    Supported TIDL layer type ---          Concat -- Concat_113 
    Supported TIDL layer type ---            Conv -- Conv_114 
    Supported TIDL layer type ---            Relu -- Relu_115 
    Supported TIDL layer type ---           Split -- Split_116 
    Supported TIDL layer type ---            Conv -- Conv_117 
    Supported TIDL layer type ---            Relu -- Relu_118 
    Supported TIDL layer type ---            Conv -- Conv_119 
    Supported TIDL layer type ---            Relu -- Relu_120 
    Supported TIDL layer type ---          Concat -- Concat_121 
    Supported TIDL layer type ---            Conv -- Conv_122 
    Supported TIDL layer type ---            Relu -- Relu_123 
    Supported TIDL layer type ---            Conv -- Conv_184 
    Supported TIDL layer type ---            Relu -- Relu_185 
    Supported TIDL layer type ---            Conv -- Conv_186 
    Supported TIDL layer type ---            Relu -- Relu_187 
    Supported TIDL layer type ---            Conv -- Conv_188 
    Supported TIDL layer type ---            Conv -- Conv_179 
    Supported TIDL layer type ---            Relu -- Relu_180 
    Supported TIDL layer type ---            Conv -- Conv_181 
    Supported TIDL layer type ---            Relu -- Relu_182 
    Supported TIDL layer type ---            Conv -- Conv_183 
    Supported TIDL layer type ---          Concat -- Concat_189 
    Supported TIDL layer type ---         Reshape -- Reshape_201 
    Supported TIDL layer type ---            Conv -- Conv_173 
    Supported TIDL layer type ---            Relu -- Relu_174 
    Supported TIDL layer type ---            Conv -- Conv_175 
    Supported TIDL layer type ---            Relu -- Relu_176 
    Supported TIDL layer type ---            Conv -- Conv_177 
    Supported TIDL layer type ---            Conv -- Conv_168 
    Supported TIDL layer type ---            Relu -- Relu_169 
    Supported TIDL layer type ---            Conv -- Conv_170 
    Supported TIDL layer type ---            Relu -- Relu_171 
    Supported TIDL layer type ---            Conv -- Conv_172 
    Supported TIDL layer type ---          Concat -- Concat_178 
    Supported TIDL layer type ---         Reshape -- Reshape_198 
    Supported TIDL layer type ---            Conv -- Conv_162 
    Supported TIDL layer type ---            Relu -- Relu_163 
    Supported TIDL layer type ---            Conv -- Conv_164 
    Supported TIDL layer type ---            Relu -- Relu_165 
    Supported TIDL layer type ---            Conv -- Conv_166 
    Supported TIDL layer type ---            Conv -- Conv_157 
    Supported TIDL layer type ---            Relu -- Relu_158 
    Supported TIDL layer type ---            Conv -- Conv_159 
    Supported TIDL layer type ---            Relu -- Relu_160 
    Supported TIDL layer type ---            Conv -- Conv_161 
    Supported TIDL layer type ---          Concat -- Concat_167 
    Supported TIDL layer type ---         Reshape -- Reshape_195 
    Supported TIDL layer type ---          Concat -- Concat_202 
    Supported TIDL layer type ---           Split -- Split_203 
    Supported TIDL layer type ---         Sigmoid -- Sigmoid_241 
    Supported TIDL layer type ---         Reshape -- Reshape_213 
    Supported TIDL layer type ---       Transpose -- Transpose_214 
    Supported TIDL layer type ---         Softmax -- Softmax_215 
    Supported TIDL layer type ---       Transpose -- Transpose_216 
    Supported TIDL layer type ---            Conv -- Conv_217 
    Supported TIDL layer type ---         Reshape -- Reshape_221 
    Supported TIDL layer type ---           Split -- Split_222 
    Supported TIDL layer type ---             Add -- Add_237 
    Supported TIDL layer type ---             Sub -- Sub_225 
    Supported TIDL layer type ---             Div -- Div_226 
    Supported TIDL layer type ---           Split -- Split_227 
    Supported TIDL layer type ---            Conv -- Conv_143 
    Supported TIDL layer type ---            Relu -- Relu_144 
    Supported TIDL layer type ---            Conv -- Conv_145 
    Supported TIDL layer type ---            Relu -- Relu_146 
    Supported TIDL layer type ---            Conv -- Conv_147 
    Supported TIDL layer type ---         Reshape -- Reshape_150 
    Supported TIDL layer type ---            Conv -- Conv_135 
    Supported TIDL layer type ---            Relu -- Relu_136 
    Supported TIDL layer type ---            Conv -- Conv_137 
    Supported TIDL layer type ---            Relu -- Relu_138 
    Supported TIDL layer type ---            Conv -- Conv_139 
    Supported TIDL layer type ---         Reshape -- Reshape_142 
    Supported TIDL layer type ---            Conv -- Conv_127 
    Supported TIDL layer type ---            Relu -- Relu_128 
    Supported TIDL layer type ---            Conv -- Conv_129 
    Supported TIDL layer type ---            Relu -- Relu_130 
    Supported TIDL layer type ---            Conv -- Conv_131 
    Supported TIDL layer type ---         Reshape -- Reshape_134 
    Supported TIDL layer type ---          Concat -- Concat_151 
    Supported TIDL layer type ---         Sigmoid -- Sigmoid_152 
    Supported TIDL layer type ---             Sub -- Sub_154 
    Supported TIDL layer type ---             Mul -- Mul_156 
    Unsupported (imports) TIDL layer type for ONNX op type --- Cos 
    Supported TIDL layer type ---             Mul -- Mul_232 
    Unsupported (imports) TIDL layer type for ONNX op type --- Sin 
    Supported TIDL layer type ---             Mul -- Mul_231 
    Supported TIDL layer type ---             Add -- Add_233 
    Supported TIDL layer type ---             Mul -- Mul_229 
    Supported TIDL layer type ---             Mul -- Mul_228 
    Supported TIDL layer type ---             Sub -- Sub_230 
    Supported TIDL layer type ---          Concat -- Concat_234 
    Supported TIDL layer type ---             Add -- Add_236 
    Supported TIDL layer type ---          Concat -- Concat_238 
    Supported TIDL layer type ---             Mul -- Mul_240 
    Supported TIDL layer type ---          Concat -- Concat_243 
    
    Preliminary subgraphs created = 4 
    
    *** WARNING : Number of subgraphs generated > max_num_subgraphs provided in options - additional subgraphs are delegated to ARM *** 
    Final number of subgraphs created are : 1, - Offloaded Nodes - 66, Total Nodes - 207 
    TIDL ALLOWLISTING LAYER CHECK -- TIDL_PoolingLayer '': kernel size 5x5 with stride 1x1 not supported  
    TIDL ALLOWLISTING LAYER CHECK -- TIDL_PoolingLayer '': kernel size 5x5 with stride 1x1 not supported  
    TIDL ALLOWLISTING LAYER CHECK -- TIDL_PoolingLayer '': kernel size 5x5 with stride 1x1 not supported  
    INFORMATION -- [TIDL_ResizeLayer]  Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.  
    INFORMATION -- [TIDL_ResizeLayer]  Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.  
    Layers type not supported by TIDL --- layer type - Cos,  Node name -Cos_223  
    Layers type not supported by TIDL --- layer type - Sin,  Node name -Sin_224  
    Running runtimes graphviz - /home/root/tidl_tools/tidl_graphVisualiser_runtimes.out ../../../model-artifacts//yolov8n//allowedNode.txt ../../../model-artifacts//yolov8n//tempDir/graphvizInfo.txt ../../../model-artifacts//yolov8n//tempDir/runtimes_visualization.svg 
    *** In TIDL_createStateImportFunc *** 
    Compute on node : TIDLExecutionProvider_TIDL_0_0
      0,            Conv, 3, 1, images, 146
      1,            Relu, 1, 1, 146, 147
      2,            Conv, 3, 1, 147, 148
      3,            Relu, 1, 1, 148, 149
      4,            Conv, 3, 1, 149, 150
      5,            Relu, 1, 1, 150, 151
      6,           Split, 1, 2, 151, 152
      7,            Conv, 3, 1, 153, 154
      8,            Relu, 1, 1, 154, 155
      9,            Conv, 3, 1, 155, 156
     10,            Relu, 1, 1, 156, 157
     11,             Add, 2, 1, 153, 158
     12,          Concat, 3, 1, 152, 159
     13,            Conv, 3, 1, 159, 160
     14,            Relu, 1, 1, 160, 161
     15,            Conv, 3, 1, 161, 162
     16,            Relu, 1, 1, 162, 163
     17,            Conv, 3, 1, 163, 164
     18,            Relu, 1, 1, 164, 165
     19,           Split, 1, 2, 165, 166
     20,            Conv, 3, 1, 167, 168
     21,            Relu, 1, 1, 168, 169
     22,            Conv, 3, 1, 169, 170
     23,            Relu, 1, 1, 170, 171
     24,             Add, 2, 1, 167, 172
     25,            Conv, 3, 1, 172, 173
     26,            Relu, 1, 1, 173, 174
     27,            Conv, 3, 1, 174, 175
     28,            Relu, 1, 1, 175, 176
     29,             Add, 2, 1, 172, 177
     30,          Concat, 4, 1, 166, 178
     31,            Conv, 3, 1, 178, 179
     32,            Relu, 1, 1, 179, 180
     33,            Conv, 3, 1, 180, 181
     34,            Relu, 1, 1, 181, 182
     35,            Conv, 3, 1, 182, 183
     36,            Relu, 1, 1, 183, 184
     37,           Split, 1, 2, 184, 185
     38,            Conv, 3, 1, 186, 187
     39,            Relu, 1, 1, 187, 188
     40,            Conv, 3, 1, 188, 189
     41,            Relu, 1, 1, 189, 190
     42,             Add, 2, 1, 186, 191
     43,            Conv, 3, 1, 191, 192
     44,            Relu, 1, 1, 192, 193
     45,            Conv, 3, 1, 193, 194
     46,            Relu, 1, 1, 194, 195
     47,             Add, 2, 1, 191, 196
     48,          Concat, 4, 1, 185, 197
     49,            Conv, 3, 1, 197, 198
     50,            Relu, 1, 1, 198, 199
     51,            Conv, 3, 1, 199, 200
     52,            Relu, 1, 1, 200, 201
     53,            Conv, 3, 1, 201, 202
     54,            Relu, 1, 1, 202, 203
     55,           Split, 1, 2, 203, 204
     56,            Conv, 3, 1, 205, 206
     57,            Relu, 1, 1, 206, 207
     58,            Conv, 3, 1, 207, 208
     59,            Relu, 1, 1, 208, 209
     60,             Add, 2, 1, 205, 210
     61,          Concat, 3, 1, 204, 211
     62,            Conv, 3, 1, 211, 212
     63,            Relu, 1, 1, 212, 213
     64,            Conv, 3, 1, 213, 214
     65,            Relu, 1, 1, 214, 215
    
    Input tensor name -  images 
    Output tensor name - 180 
    Output tensor name - 199 
    Output tensor name - 215 
     Graph Domain TO version : 11In TIDL_onnxRtImportInit subgraph_name=subgraph_0
    Layer 0, subgraph id subgraph_0, name=180
    Layer 1, subgraph id subgraph_0, name=199
    Layer 2, subgraph id subgraph_0, name=215
    Layer 3, subgraph id subgraph_0, name=images
    In TIDL_runtimesOptimizeNet: LayerIndex = 70, dataIndex = 71 
    
     ************** Frame index 1 : Running float import ************* 
    In TIDL_runtimesPostProcessNet 
    In TIDL_runtimesPostProcessNet 1
    In TIDL_runtimesPostProcessNet 2
    In TIDL_runtimesPostProcessNet 3
    ****************************************************
    **                ALL MODEL CHECK PASSED          **
    ****************************************************
    
    In TIDL_runtimesPostProcessNet 4
    ************ in TIDL_subgraphRtCreate ************ 
     The soft limit is 2048
    The hard limit is 2048
    MEM: Init ... !!!
    MEM: Init ... Done !!!
     0.0s:  VX_ZONE_INIT:Enabled
     0.22s:  VX_ZONE_ERROR:Enabled
     0.27s:  VX_ZONE_WARNING:Enabled
     0.2955s:  VX_ZONE_INIT:[tivxInit:190] Initialization Done !!!
    ************ TIDL_subgraphRtCreate done ************ 
     *******   In TIDL_subgraphRtInvoke  ******** 
       0         1.00000         0.05490         1.00000 6
       1         1.00000         0.05490         1.00000 6
       2         1.00000         0.00000        36.73758 6
       3         1.00000         0.00000        55.12258 6
       4         1.00000         0.00000        14.97529 6
       5         1.00000         0.00000        14.97529 6
       6         1.00000         0.00000         8.84293 6
       7         1.00000         0.00000        18.59137 6
       8         1.00000         0.00000        14.98748 6
       9         1.00000         0.00000        17.30734 6
      10         1.00000         0.00000        17.30734 6
      11         1.00000         0.00000        10.18376 6
      12         1.00000         0.00000         5.28234 6
      13         1.00000         0.00000         5.64779 6
      14         1.00000         0.00000         5.64779 6
      15         1.00000         0.00000         3.24116 6
      16         1.00000         0.00000         3.93958 6
      17         1.00000         0.00000         3.82232 6
      18         1.00000         0.00000         5.06502 6
      19         1.00000         0.00000         3.04332 6
      20         1.00000         0.00000         6.82933 6
      21         1.00000         0.00000         8.95797 6
      22         1.00000         0.00000         8.95797 6
      23         1.00000         0.00000         4.80396 6
      24         1.00000         0.00000         4.80396 6
      25         1.00000         0.00000         4.65725 6
      26         1.00000         0.00000         5.14755 6
      28         1.00000         0.00000         5.14755 6
      29         1.00000         0.00000         4.13653 6
      30         1.00000         0.00000         3.39918 6
      31         1.00000         0.00000         3.85458 6
      32         1.00000         0.00000         4.95134 6
      33         1.00000         0.00000         3.11164 6
      34         1.00000         0.00000         5.61543 6
      35         1.00000         0.00000         6.68253 6
      36         1.00000         0.00000         6.68253 6
      37         1.00000         0.00000         4.45978 6
      38         1.00000         0.00000         4.45978 6
      39         1.00000         0.00000         3.45519 6
      40         1.00000         0.00000         5.69489 6
      42         1.00000         0.00000         5.69489 6
      43         1.00000         0.00000         4.11026 6
      44         1.00000         0.00000         2.87815 6
      45         1.00000         0.00000         5.11256 6
      46         1.00000         0.00000         5.20790 6
      47         1.00000         0.00000         5.69489 6
      48         1.00000         0.00000         4.16965 6
      49         1.00000         0.00000         4.97699 6
      50         1.00000         0.00000         4.97699 6
     Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger,    paddingWait,LayerWithoutPad,LayerHandleCopy,   BackupCycles,  RestoreCycles,Multic7xContextCopyCycles,
         0,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         1,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         2,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         3,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         4,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         5,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         6,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         7,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         8,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         9,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        10,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        11,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        12,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        13,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        14,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        15,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        16,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        17,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        18,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        19,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        20,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        21,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        22,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        23,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        24,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        25,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        26,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         0,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        28,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        29,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        30,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        31,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        32,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        33,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        34,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        35,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        36,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        37,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        38,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        39,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        40,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         0,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        42,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        43,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        44,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        45,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        46,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        47,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        48,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        49,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
        50,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
         0,              0,              0,              0,              0,              0,                 0,              0,                 0,              0,              0,              0,              0,              0,              0,              0,              0,              0,
     Sum of Layer Cycles 0 
    Sub Graph Stats 395.000000 760336.000000 1218.000000 
    *******  TIDL_subgraphRtInvoke done  ******** 
    
    **********  Frame Index 1 : Running float inference **********
    2024-08-07 06:00:46.847105905 [E:onnxruntime:, sequential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running ReorderInput node. Name:'ReorderInput' Status Message: /onnxruntime/onnxruntime/contrib_ops/cpu/nchwc_ops.cc:17 virtual onnxruntime::common::Status onnxruntime::contrib::ReorderInput::Compute(onnxruntime::OpKernelContext*) const X_rank == 4 was false. 
    
    Process Process-1:
    Traceback (most recent call last):
      File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
        self.run()
      File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
        self._target(*self._args, **self._kwargs)
      File "/home/root/examples/osrt_python/ort/onnxrt_ep.py", line 242, in run_model
        imgs, output, proc_time, sub_graph_time, height, width  = infer_image(sess, input_images, config)
      File "/home/root/examples/osrt_python/ort/onnxrt_ep.py", line 132, in infer_image
        output = list(sess.run(None, {input_name: input_data}))
      File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 200, in run
        return self._sess.run(output_names, input_feed, run_options)
    onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running ReorderInput node. Name:'ReorderInput' Status Message: /onnxruntime/onnxruntime/contrib_ops/cpu/nchwc_ops.cc:17 virtual onnxruntime::common::Status onnxruntime::contrib::ReorderInput::Compute(onnxruntime::OpKernelContext*) const X_rank == 4 was false. 
    
    ************ in TIDL_subgraphRtDelete ************ 
     MEM: Deinit ... !!!
    MEM: Alloc's: 28 alloc's of 143916457 bytes 
    MEM: Free's : 28 free's  of 143916457 bytes 
    MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    

  • Hi,

    Looking at this warning -

    Preliminary subgraphs created = 4

    *** WARNING : Number of subgraphs generated > max_num_subgraphs provided in options - additional subgraphs are delegated to ARM ***
    Final number of subgraphs created are : 1, - Offloaded Nodes - 66, Total Nodes - 207

    Can you ensure that the max_num_subgraphs is greater that the subgraphs that are needed (4)?

    num_tidl_subgraphs                              = 1 

    Right now you have it set to 1. Can you try the default 16 max subgraphs? This is changeable in common_utils.py

    Best,

    Asha