TDA4VM: Support for YOLOPv2 compile on TDA4VM

Shubham Sharma

Prodigy 90 points

Part Number: TDA4VM

Tool/software:

Hello Devs,

I am trying to compile YOLOPv2 [LINK] for TDA4VM.

There seems to be no official support for YOLOPv2 as the model has three heads.

Please help with the compilation steps.

Thank you

Best

Shubham

over 2 years ago

0 Pratik Kedar over 2 years ago

TI__Mastermind 24041 points

Hi,

Our analytics expert will reach out to you.

Thanks for your patience.

0 Asha Bhandarkar over 2 years ago in reply to Pratik Kedar

TI__Genius 10170 points

Hi Shubham,

You are right in that we do not supply YOLOPv2 as part of our model zoo.

If you are looking to compile the model with edgeai-tidl-tools, I would recommend looking at our documentation on custom model evaluation (https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/custom_model_evaluation.md) as well as the OSRT compilation and inference flow described here (https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/examples/osrt_python/README.md)

Is there a specific step within the compilation process you need help with?

Best,

Asha

0 Shubham Sharma over 2 years ago in reply to Asha Bhandarkar

Prodigy 90 points

Hi Asha,

Thanks for the reply.

I am working on it, will get back to you in case needed.

Please do not close this thread.

Thank you

Best

Shubham

0 Asha Bhandarkar over 2 years ago in reply to Shubham Sharma

TI__Genius 10170 points

Hi Shubham,

Sounds good. I will wait for your update.

Best,

Asha

0 Shubham Sharma over 2 years ago in reply to Asha Bhandarkar

Prodigy 90 points

Hi Asha,

I have been following this [LINK].

The yolo.zip (in the link) file contains import and infer config files.

However, as per documentation [LINK], it does not support parameters provided in import_config files.

Please guide, if I am going wrong somewhere.

I also followed the steps you suggested, I am encountering this error:

model_config :

'yolop' : {
        'model_path' : os.path.join(models_base_path, 'yolop.onnx'),
        'source' : {'model_url': '', 'opt': True,  'infer_shape' : True, \
                    'meta_arch_url' : ''},
        'mean': [0, 0, 0],
        'std' : [0.003921568627, 0.003921568627, 0.003921568627],
        'num_images' : numImages,
        'num_classes': 1000,
        'model_type': 'classification',
        'meta_layers_names_list' : os.path.join(models_base_path, 'yolop.prototxt'),
        'session_name' : 'onnxrt' ,
        'meta_arch_type' : 6
    }

_______________________________________________________________________
ERROR :
root@81731c42f150:/home/examples/osrt_python/ort# python3 onnxrt_ep.py -c
Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['yolop']


Running_Model :  yolop

-111-------------> ['../../../test_data/airshow.jpg', '../../../test_data/ADE_val_00001801.jpg']

WARNING : 'meta_layers_names_list' is not provided - running OD post processing in ARM mode

TIDL Meta PipeLine (Proto) File  :

Number of OD backbone nodes = 0
Size of odBackboneNodeIds = 0

Preliminary subgraphs created = 4
Final number of subgraphs created are : 1, - Offloaded Nodes - 252, Total Nodes - 356
@@@@@@@@@@@@@@@@@@@@@@@@
--222----------> 0 0
------4444-----> {'images': array([[[[0.7137255 , 0.7137255 , 0.7137255 , ..., 0.65882355,
          0.65882355, 0.65882355],
         [0.7137255 , 0.7137255 , 0.7137255 , ..., 0.65882355,
          0.65882355, 0.65882355],
         [0.7137255 , 0.7137255 , 0.7137255 , ..., 0.65882355,
          0.65882355, 0.65882355],
         ...,
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.49803925,
          0.49411768, 0.49411768],
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.49803925,
          0.49803925, 0.49803925],
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.49803925,
          0.49803925, 0.49803925]],

        [[0.7058824 , 0.7058824 , 0.7058824 , ..., 0.64705884,
          0.64705884, 0.64705884],
         [0.7058824 , 0.7058824 , 0.7058824 , ..., 0.64705884,
          0.64705884, 0.64705884],
         [0.7058824 , 0.7058824 , 0.7058824 , ..., 0.64705884,
          0.64705884, 0.64705884],
         ...,
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.47450984,
          0.47450984, 0.47450984],
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.4784314 ,
          0.4784314 , 0.4784314 ],
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.4784314 ,
          0.4784314 , 0.4784314 ]],

        [[0.7254902 , 0.7254902 , 0.7254902 , ..., 0.6745098 ,
          0.6745098 , 0.6745098 ],
         [0.7254902 , 0.7254902 , 0.7254902 , ..., 0.6745098 ,
          0.6745098 , 0.6745098 ],
         [0.7254902 , 0.7254902 , 0.7254902 , ..., 0.6745098 ,
          0.6745098 , 0.6745098 ],
         ...,
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.4039216 ,
          0.40000004, 0.39607847],
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.4039216 ,
          0.4039216 , 0.4039216 ],
         [0.1764706 , 0.1764706 , 0.17254902, ..., 0.4039216 ,
          0.40784317, 0.40784317]]]], dtype=float32)}

 ************** Frame index 1 : Running float import *************
INFORMATION: [TIDL_ResizeLayer] Resize_107 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_125 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1175 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1192 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1209 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1216 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1233 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1250 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          9 WARNINGS          0 ERRORS          **
****************************************************
 0.0s:  VX_ZONE_INIT:Enabled
 0.25s:  VX_ZONE_ERROR:Enabled
 0.29s:  VX_ZONE_WARNING:Enabled
 0.1657s:  VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!
 0.88408s:  VX_ZONE_ERROR:[tivxAlgiVisionCreate:344] Calling ialg.algAlloc failed with status = -1110
 0.88434s:  VX_ZONE_ERROR:[tivxKernelTIDLCreate:659] tivxAlgiVisionCreate returned NULL
 0.88639s:  VX_ZONE_ERROR:[ownContextSendCmd:817] Command ack message returned failure cmd_status: -1
 0.88660s:  VX_ZONE_ERROR:[ownContextSendCmd:851] tivxEventWait() failed.
 0.88666s:  VX_ZONE_ERROR:[ownNodeKernelInit:538] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
 0.88670s:  VX_ZONE_ERROR:[ownNodeKernelInit:539] Please be sure the target callbacks have been registered for this core
 0.88674s:  VX_ZONE_ERROR:[ownNodeKernelInit:540] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
 0.88681s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl ... failed !!!
 0.88688s:  VX_ZONE_ERROR:[vxVerifyGraph:2055] Node kernel init failed
 0.88692s:  VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
 0.101669s:  VX_ZONE_ERROR:[tivxAlgiVisionCreate:344] Calling ialg.algAlloc failed with status = -1110
 0.101694s:  VX_ZONE_ERROR:[tivxKernelTIDLCreate:659] tivxAlgiVisionCreate returned NULL
 0.101919s:  VX_ZONE_ERROR:[ownContextSendCmd:817] Command ack message returned failure cmd_status: -1
 0.101956s:  VX_ZONE_ERROR:[ownContextSendCmd:851] tivxEventWait() failed.
 0.101964s:  VX_ZONE_ERROR:[ownNodeKernelInit:538] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
 0.101970s:  VX_ZONE_ERROR:[ownNodeKernelInit:539] Please be sure the target callbacks have been registered for this core
 0.101975s:  VX_ZONE_ERROR:[ownNodeKernelInit:540] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
 0.101984s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl ... failed !!!
 0.101992s:  VX_ZONE_ERROR:[vxVerifyGraph:2055] Node kernel init failed
 0.101998s:  VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
 0.102119s:  VX_ZONE_ERROR:[ownGraphScheduleGraphWrapper:820] graph is not in a state required to be scheduled
 0.102126s:  VX_ZONE_ERROR:[vxProcessGraph:755] schedule graph failed
 0.102131s:  VX_ZONE_ERROR:[vxProcessGraph:760] wait graph failed
ERROR: Running TIDL graph ... Failed !!!

**********  Frame Index 1 : Running float inference **********
2024-06-14 13:48:42.507268027 [E:onnxruntime:, sequential_executor.cc:339 Execute] Non-zero status code returned while running Slice node. Name:'Slice_896' Status Message: 'axes' has an axis outside of the tensor dimension count
Traceback (most recent call last):
  File "onnxrt_ep.py", line 261, in <module>
    run_model(model, mIdx)
  File "onnxrt_ep.py", line 186, in run_model
    imgs, output, proc_time, sub_graph_time, height, width  = infer_image(sess, input_images, config)
  File "onnxrt_ep.py", line 100, in infer_image
    output = list(sess.run(None, {input_name: input_data}))
  File "/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 188, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Slice node. Name:'Slice_896' Status Message: 'axes' has an axis outside of the tensor dimension count
root@81731c42f150:/home/examples/osrt_python/ort# python3 onnxrt_ep.py -c
Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['yolop']


Running_Model :  yolop

-111-------------> ['../../../test_data/airshow.jpg', '../../../test_data/ADE_val_00001801.jpg']

WARNING : 'meta_layers_names_list' is not provided - running OD post processing in ARM mode

TIDL Meta PipeLine (Proto) File  :

Number of OD backbone nodes = 0
Size of odBackboneNodeIds = 0

Preliminary subgraphs created = 4
Final number of subgraphs created are : 1, - Offloaded Nodes - 252, Total Nodes - 356
@@@@@@@@@@@@@@@@@@@@@@@@
--222----------> 0 0
------4444-----> {'images': (1, 3, 640, 640)}

 ************** Frame index 1 : Running float import *************
INFORMATION: [TIDL_ResizeLayer] Resize_107 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_125 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1175 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1192 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1209 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1216 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1233 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_1250 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          9 WARNINGS          0 ERRORS          **
****************************************************
 0.0s:  VX_ZONE_INIT:Enabled
 0.30s:  VX_ZONE_ERROR:Enabled
 0.32s:  VX_ZONE_WARNING:Enabled
 0.1384s:  VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!
 0.86043s:  VX_ZONE_ERROR:[tivxAlgiVisionCreate:344] Calling ialg.algAlloc failed with status = -1110
 0.86078s:  VX_ZONE_ERROR:[tivxKernelTIDLCreate:659] tivxAlgiVisionCreate returned NULL
 0.86294s:  VX_ZONE_ERROR:[ownContextSendCmd:817] Command ack message returned failure cmd_status: -1
 0.86331s:  VX_ZONE_ERROR:[ownContextSendCmd:851] tivxEventWait() failed.
 0.86339s:  VX_ZONE_ERROR:[ownNodeKernelInit:538] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
 0.86344s:  VX_ZONE_ERROR:[ownNodeKernelInit:539] Please be sure the target callbacks have been registered for this core
 0.86349s:  VX_ZONE_ERROR:[ownNodeKernelInit:540] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
 0.86358s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl ... failed !!!
 0.86367s:  VX_ZONE_ERROR:[vxVerifyGraph:2055] Node kernel init failed
 0.86372s:  VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
 0.100158s:  VX_ZONE_ERROR:[tivxAlgiVisionCreate:344] Calling ialg.algAlloc failed with status = -1110
 0.100183s:  VX_ZONE_ERROR:[tivxKernelTIDLCreate:659] tivxAlgiVisionCreate returned NULL
 0.100367s:  VX_ZONE_ERROR:[ownContextSendCmd:817] Command ack message returned failure cmd_status: -1
 0.100397s:  VX_ZONE_ERROR:[ownContextSendCmd:851] tivxEventWait() failed.
 0.100404s:  VX_ZONE_ERROR:[ownNodeKernelInit:538] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
 0.100410s:  VX_ZONE_ERROR:[ownNodeKernelInit:539] Please be sure the target callbacks have been registered for this core
 0.100415s:  VX_ZONE_ERROR:[ownNodeKernelInit:540] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
 0.100424s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl ... failed !!!
 0.100432s:  VX_ZONE_ERROR:[vxVerifyGraph:2055] Node kernel init failed
 0.100440s:  VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
 0.100597s:  VX_ZONE_ERROR:[ownGraphScheduleGraphWrapper:820] graph is not in a state required to be scheduled
 0.100608s:  VX_ZONE_ERROR:[vxProcessGraph:755] schedule graph failed
 0.100613s:  VX_ZONE_ERROR:[vxProcessGraph:760] wait graph failed
ERROR: Running TIDL graph ... Failed !!!

**********  Frame Index 1 : Running float inference **********
2024-06-14 13:49:24.034944762 [E:onnxruntime:, sequential_executor.cc:339 Execute] Non-zero status code returned while running Slice node. Name:'Slice_896' Status Message: 'axes' has an axis outside of the tensor dimension count
Traceback (most recent call last):
  File "onnxrt_ep.py", line 261, in <module>
    run_model(model, mIdx)
  File "onnxrt_ep.py", line 186, in run_model
    imgs, output, proc_time, sub_graph_time, height, width  = infer_image(sess, input_images, config)
  File "onnxrt_ep.py", line 100, in infer_image
    output = list(sess.run(None, {input_name: input_data}))
  File "/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 188, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Slice node. Name:'Slice_896' Status Message: 'axes' has an axis outside of the tensor dimension count

Thank you

Best

Shubham

0 Asha Bhandarkar over 2 years ago in reply to Shubham Sharma

TI__Genius 10170 points

Hi Shubham,

Very sorry for the delay in getting back to you.

Based on the log you've provided (thank you!) I'm seeing two immediate potential issues. One is the the warning on line 170 regarding memory allocation. The other looks to be the ONNX runtime on line 206 which indicates to me some issue with the properties of that specific Slice layer.

I'm basing my observations on what I believe is the model you are running which I believe your colleague has linked here: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1374336/sk-tda4vm-sk-tda4vm-yolop-compile
I think the attachment of this in your response did not work.

Have you tried running this model without C7x offload (passing the -d option) to see if the layers are passing here? Can you try this and provide the same debug logs?

Best,

Asha

Processors

Processors forum

TDA4VM: Support for YOLOPv2 compile on TDA4VM