TDA4VM: TDA4VM: Need support on YOLOP source compilation using TIDL and deployment on TDA platform

Nandhakumar Shanmugam

Prodigy 10 points

Part Number: TDA4VM

Hi,

I am working on deploying a custom YOLOP model on TDA4VM using Processor SDK Linux Edge AI version 11_00_06_00.

https://github.com/hustvl/YOLOP

I would like guidance on the deployment workflow for this model on TDA4VM.

My questions are:

Is YOLOP supported with TIDL on SDK 11_00_06_00?

Can all three output heads (detection, drivable area segmentation, lane segmentation) be accelerated on TDA4VM?

What is the recommended procedure to compile the YOLOP sources using edgeai-tidl-tools?

Are there any reference examples for deploying multi-output ONNX models containing both detection and segmentation outputs?

If some layers are unsupported by TIDL, will they automatically execute on ARM through ONNX Runtime + TIDL Execution Provider?

I have already exported the model to ONNX and would appreciate any deployment guidance, sample configurations, or reference examples.

SDK Version: 11_00_06_00
Platform: SK-TDA4VM / TDA4VM

Looking forward to hear from you ASAP.

Thank you.

Regards
Nandhakumar

3 months ago

0 Vaibhav Kumar 3 months ago

TI__Intellectual 2690 points

Nandhakumar Shanmugam said:
Is YOLOP supported with TIDL on SDK 11_00_06_00?

I tried quick compilation of model from the github link, it is not fully supported as in not all layers are going to be accelerated. For e.g. some slice layers, some pooling layers are not matching TIDL contraint, but you can tweak the model to get complete model accelerated.

Nandhakumar Shanmugam said:
Can all three output heads (detection, drivable area segmentation, lane segmentation) be accelerated on TDA4VM?

Yes, layers which are running on c7x will give accelerated performance while other layers are not supported will run in cpu.

Nandhakumar Shanmugam said:
What is the recommended procedure to compile the YOLOP sources using edgeai-tidl-tools?

You can go through https://github.com/TexasInstruments/edgeai-tidl-tools/tree/11_02_04_00/docs to understand tidl, compilation, inference etc.

Nandhakumar Shanmugam said:
If some layers are unsupported by TIDL, will they automatically execute on ARM through ONNX Runtime + TIDL Execution Provider?

Yes.

Nandhakumar Shanmugam said:
Are there any reference examples for deploying multi-output ONNX models containing both detection and segmentation outputs?

There are examples from tidl which does multiple tasks. But there are some individual task compiled models available at github.com/.../edgeai-modelzoo

Regards,
Vaibhav

0 Nandhakumar Shanmugam 3 months ago in reply to Vaibhav Kumar

Prodigy 10 points

Hi Vaibhav,

I compiled the YOLOP model and observed the same layer error during compilation. However, the compilation completed successfully. I have attached the compilation log for reference.

I am currently modifying the unsupported layer. Could you please let me know what exact changes are required for the model to run on the TDA4VM board?

I also have a few questions:

1. Should NMS be included inside the ONNX model, or is it recommended to handle NMS as post-processing?
2. For YOLOP (Detection + Drivable Area Segmentation + Lane Line Segmentation), is any special configuration required during compilation? In SDK 11_00_06_00, the examples appear to use a single task type during compilation. How should a multi-head model such as YOLOP be compiled?
3. What is the recommended approach to compile and deploy a multi-head model with TIDL?
4. Does YOLOP deployment require a prototxt file, or is the ONNX model sufficient for compilation and deployment? If a prototxt file is required, what is the recommended way to generate it for YOLOP?
5. If possible, could you please share a reference compilation script and a sample model_config for a multi-head model such as YOLOP?
6. In the E2E thread "SK-TDA4VM: YOLOP compile", a YOLOP model was shared and validated on an older SDK version. Can the same model and deployment approach be used with SDK 11_00_06_00, or are any modifications required for the current SDK?
  
  https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1161201/sk-tda4vm-yolop-compile/4374851
7. What exact layer change and modification need to be done for YOLOP to make it compatible with TDA4VM?

I have attached the compilation log for reference.

yolop-s.log

Thanks,
Nandhakumar

0 Vaibhav Kumar 3 months ago in reply to Nandhakumar Shanmugam

TI__Intellectual 2690 points

Hi, I think file is not attached properly, can you reattach?

0 Nandhakumar Shanmugam 3 months ago in reply to Vaibhav Kumar

Prodigy 10 points

Fullscreen yolop-s.log Download

root@887e96b8ab27:/home/root# cd examples/
root@887e96b8ab27:/home/root/examples# cd osrt_python/
root@887e96b8ab27:/home/root/examples/osrt_python# cd ort/
root@887e96b8ab27:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py -c -m yolop_s
Skipping import of model optimizer
Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['yolop_s']


Running_Model :  yolop_s  

✅ Using local model file - skipping download

Running shape inference on model /home/root/yolop-model-s/yolop-640-640.onnx 

========================= [Model Compilation Started] =========================

Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning

============================== [Version Summary] ==============================

-------------------------------------------------------------------------------
|          TIDL Tools Version          |              11_00_06_00             |
-------------------------------------------------------------------------------
|         C7x Firmware Version         |              11_00_00_00             |
-------------------------------------------------------------------------------
|            Runtime Version           |                1.15.0                |
-------------------------------------------------------------------------------
|          Model Opset Version         |                  12                  |
-------------------------------------------------------------------------------

============================== [Parsing Started] ==============================

[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options

------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
|          Core           |      No. of Nodes       |   Number of Subgraphs   |
-------------------------------------------------------------------------------
| C7x                     |                     522 |                       8 |
| CPU                     |                      15 |                       x |
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
|  Node   |  Node Name  |                       Reason                        |
-------------------------------------------------------------------------------
| Slice   | Slice_34    | Non-singular stride are not supported individually  |
| Slice   | Slice_39    | Non-singular stride are not supported individually  |
| Slice   | Slice_19    | Non-singular stride are not supported individually  |
| Slice   | Slice_24    | Non-singular stride are not supported individually  |
| Slice   | Slice_29    | Non-singular stride are not supported individually  |
| Slice   | Slice_9     | Non-singular stride are not supported individually  |
| MaxPool | MaxPool_325 | Kernel size (13x13) with stride (1x1) not supported |
| MaxPool | MaxPool_324 | Kernel size (9x9) with stride (1x1) not supported   |
| MaxPool | MaxPool_323 | Kernel size (5x5) with stride (1x1) not supported   |
| Expand  | Expand_686  | Layer type not supported by TIDL                    |
| Expand  | Expand_810  | Layer type not supported by TIDL                    |
| Expand  | Expand_994  | Layer type not supported by TIDL                    |
| Expand  | Expand_1118 | Layer type not supported by TIDL                    |
| Expand  | Expand_1302 | Layer type not supported by TIDL                    |
| Expand  | Expand_1426 | Layer type not supported by TIDL                    |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================

==================== [Optimization for subgraph_0 Started] ====================

----------------------------- Optimization Summary -----------------------------
--------------------------------------------------------------------------------
|         Layer         | Nodes before optimization | Nodes after optimization |
--------------------------------------------------------------------------------
| TIDL_BatchNormLayer   |                         3 |                       45 |
| TIDL_CLipLayer        |                        26 |                        0 |
| TIDL_EltWiseLayer     |                        85 |                       33 |
| TIDL_ConvolutionLayer |                        32 |                       32 |
| TIDL_ConcatLayer      |                         4 |                        4 |
| TIDL_LeakyReluLayer   |                         3 |                        0 |
--------------------------------------------------------------------------------

Total nodes in subgraph: 128

=================== [Optimization for subgraph_0 Completed] ===================

The soft limit is 10240
The hard limit is 10240
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
 0.9s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
 0.11s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
 0.177s:  VX_ZONE_INFO: [ownAddTargetKernelInternal:189] registered kernel vx_tutorial_graph.phase_rgb on target DSP-1
 0.201s:  VX_ZONE_INFO: [ownAddTargetKernelInternal:189] registered kernel vx_tutorial_graph.phase_rgb on target DSP-2
 0.1423s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 
 0.1482s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 
 0.1549s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 
 0.1597s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 
 0.1658s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1 
 0.1707s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_2 
 0.1756s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_3 
 0.1807s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_4 
 0.1859s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_5 
 0.1906s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_6 
 0.1954s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_7 
 0.2010s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_8 
 0.2076s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU2-0 
 0.2122s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_NF 
 0.2168s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_LDC1 
 0.2218s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_MSC1 
 0.2266s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_MSC2 
 0.2318s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_VISS1 
 0.2363s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE1 
 0.2407s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE2 
 0.2442s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE3 
 0.2486s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE4 
 0.2523s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE5 
 0.2564s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE6 
 0.2601s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE7 
 0.2638s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE8 
 0.2679s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DISPLAY1 
 0.2721s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DISPLAY2 
 0.2758s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CSITX 
 0.2797s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSS_M2M1 
 0.2840s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSS_M2M2 
 0.2879s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSS_M2M3 
 0.2923s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSS_M2M4 
 0.2964s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_FC 
 0.3011s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU2-1 
 0.3059s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DMPAC_SDE 
 0.3101s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DMPAC_DOF 
 0.3148s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU3-0 
 0.3190s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU3-1 
 0.3232s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP-1 
 0.3278s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP-2 
 0.3283s:  VX_ZONE_INFO: [tivxInit:152] Initialization Done !!!
 0.3289s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
============= [Quantization & Calibration for subgraph_0 Started] =============

==================== [Optimization for subgraph_1 Started] ====================

----------------------------- Optimization Summary -----------------------------
--------------------------------------------------------------------------------
|         Layer         | Nodes before optimization | Nodes after optimization |
--------------------------------------------------------------------------------
| TIDL_LeakyReluLayer   |                         5 |                        0 |
| TIDL_ConvolutionLayer |                        42 |                       42 |
| TIDL_TransposeLayer   |                         1 |                        1 |
| TIDL_SliceLayer       |                         1 |                        1 |
| TIDL_ConstDataLayer   |                         0 |                        1 |
| TIDL_EltWiseLayer     |                        91 |                       30 |
| TIDL_ConcatLayer      |                         9 |                        9 |
| TIDL_CLipLayer        |                        29 |                        0 |
| TIDL_ResizeLayer      |                         6 |                        6 |
| TIDL_BatchNormLayer   |                         5 |                       46 |
| TIDL_SigmoidLayer     |                         1 |                        0 |
--------------------------------------------------------------------------------

Total nodes in subgraph: 164

=================== [Optimization for subgraph_1 Completed] ===================

============= [Quantization & Calibration for subgraph_1 Started] =============

==================== [Optimization for subgraph_2 Started] ====================

----------------------------- Optimization Summary -----------------------------
------------------------------------------------------------------------------------
|           Layer           | Nodes before optimization | Nodes after optimization |
------------------------------------------------------------------------------------
| TIDL_BatchNormLayer       |                         0 |                       10 |
| TIDL_ConstDataLayer       |                         0 |                        2 |
| TIDL_SliceLayer           |                         1 |                        1 |
| TIDL_ConvolutionLayer     |                         7 |                        7 |
| TIDL_PowLayer             |                         1 |                        0 |
| TIDL_ScatterElementsLayer |                         1 |                        1 |
| TIDL_CLipLayer            |                         6 |                        0 |
| TIDL_EltWiseLayer         |                        15 |                        5 |
------------------------------------------------------------------------------------

Total nodes in subgraph: 56

=================== [Optimization for subgraph_2 Completed] ===================

============= [Quantization & Calibration for subgraph_2 Started] =============

==================== [Optimization for subgraph_3 Started] ====================

----------------------------- Optimization Summary -----------------------------
------------------------------------------------------------------------------------
|           Layer           | Nodes before optimization | Nodes after optimization |
------------------------------------------------------------------------------------
| TIDL_ConstDataLayer       |                         0 |                        2 |
| TIDL_LeakyReluLayer       |                         3 |                        0 |
| TIDL_EltWiseLayer         |                        36 |                       12 |
| TIDL_ConvolutionLayer     |                        15 |                       15 |
| TIDL_TransposeLayer       |                         1 |                        1 |
| TIDL_SliceLayer           |                         1 |                        1 |
| TIDL_ConcatLayer          |                         4 |                        4 |
| TIDL_CLipLayer            |                         9 |                        0 |
| TIDL_ScatterElementsLayer |                         1 |                        1 |
| TIDL_ResizeLayer          |                         2 |                        2 |
| TIDL_BatchNormLayer       |                         3 |                       22 |
| TIDL_SigmoidLayer         |                         1 |                        0 |
------------------------------------------------------------------------------------

Total nodes in subgraph: 101

=================== [Optimization for subgraph_3 Completed] ===================

============= [Quantization & Calibration for subgraph_3 Started] =============

==================== [Optimization for subgraph_4 Started] ====================

----------------------------- Optimization Summary -----------------------------
------------------------------------------------------------------------------------
|           Layer           | Nodes before optimization | Nodes after optimization |
------------------------------------------------------------------------------------
| TIDL_BatchNormLayer       |                         0 |                       10 |
| TIDL_SigmoidLayer         |                         2 |                        0 |
| TIDL_ConstDataLayer       |                         0 |                        2 |
| TIDL_SliceLayer           |                         1 |                        1 |
| TIDL_ConvolutionLayer     |                         3 |                        3 |
| TIDL_PowLayer             |                         1 |                        0 |
| TIDL_ScatterElementsLayer |                         1 |                        1 |
| TIDL_CLipLayer            |                         4 |                        0 |
| TIDL_EltWiseLayer         |                        13 |                        5 |
------------------------------------------------------------------------------------

Total nodes in subgraph: 44

=================== [Optimization for subgraph_4 Completed] ===================

============= [Quantization & Calibration for subgraph_4 Started] =============

==================== [Optimization for subgraph_5 Started] ====================

----------------------------- Optimization Summary -----------------------------
------------------------------------------------------------------------------------
|           Layer           | Nodes before optimization | Nodes after optimization |
------------------------------------------------------------------------------------
| TIDL_LeakyReluLayer       |                         1 |                        0 |
| TIDL_EltWiseLayer         |                         8 |                        2 |
| TIDL_ConvolutionLayer     |                         3 |                        3 |
| TIDL_TransposeLayer       |                         1 |                        1 |
| TIDL_SliceLayer           |                         1 |                        1 |
| TIDL_ConstDataLayer       |                         0 |                        2 |
| TIDL_ConcatLayer          |                         1 |                        1 |
| TIDL_CLipLayer            |                         1 |                        0 |
| TIDL_ScatterElementsLayer |                         1 |                        1 |
| TIDL_BatchNormLayer       |                         1 |                        6 |
| TIDL_SigmoidLayer         |                         1 |                        0 |
------------------------------------------------------------------------------------

Total nodes in subgraph: 36

=================== [Optimization for subgraph_5 Completed] ===================

============= [Quantization & Calibration for subgraph_5 Started] =============

==================== [Optimization for subgraph_6 Started] ====================

----------------------------- Optimization Summary -----------------------------
------------------------------------------------------------------------------------
|           Layer           | Nodes before optimization | Nodes after optimization |
------------------------------------------------------------------------------------
| TIDL_BatchNormLayer       |                         0 |                        2 |
| TIDL_EltWiseLayer         |                         2 |                        1 |
| TIDL_ConstDataLayer       |                         0 |                        2 |
| TIDL_SliceLayer           |                         1 |                        1 |
| TIDL_PowLayer             |                         1 |                        0 |
| TIDL_ScatterElementsLayer |                         1 |                        1 |
------------------------------------------------------------------------------------

Total nodes in subgraph: 17

=================== [Optimization for subgraph_6 Completed] ===================

============= [Quantization & Calibration for subgraph_6 Started] =============

==================== [Optimization for subgraph_7 Started] ====================

----------------------------- Optimization Summary -----------------------------
------------------------------------------------------------------------------------
|           Layer           | Nodes before optimization | Nodes after optimization |
------------------------------------------------------------------------------------
| TIDL_ConstDataLayer       |                         0 |                        1 |
| TIDL_ConcatLayer          |                         1 |                        1 |
| TIDL_ScatterElementsLayer |                         1 |                        1 |
------------------------------------------------------------------------------------

Total nodes in subgraph: 16

=================== [Optimization for subgraph_7 Completed] ===================

============= [Quantization & Calibration for subgraph_7 Started] =============


-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [1 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [2 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [3 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [4 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [5 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [6 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [7 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [8 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [9 / 10]: -----------------
[=============================================================================] 100 %

----------------- Fixed-point Calibration Iteration [10 / 10]: -----------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================




 
Completed_Model :     1, Name : yolop_s                                           , Total time :  159158.56, Offload Time :   16726.41 , DDR RW MBs : 0, Output Image File : py_out_yolop_s_ADE_val_00001801.jpg, Output Bin File : py_out_yolop_s_ADE_val_00001801.bin
 
 
MEM: Deinit ... !!!
MEM: Alloc's: 270 alloc's of 994288512 bytes 
MEM: Free's : 270 free's  of 994288512 bytes 
MEM: Open's : 0 allocs  of 0 bytes 
MEM: Deinit ... Done !!!
root@887e96b8ab27:/home/root/examples/osrt_python/ort#

0 Vaibhav Kumar 3 months ago in reply to Nandhakumar Shanmugam

TI__Intellectual 2690 points

Are you using https://github.com/hustvl/YOLOP/blob/main/export_onnx.py model?

0 Nandhakumar Shanmugam 3 months ago in reply to Vaibhav Kumar

Prodigy 10 points

Yes, I am using the exported YOLOP model from the weights directory with a 640×640 input resolution. Could you please provide your feedback on the above questions at your earliest convenience, as it will help me proceed with the deployment?

0 Vaibhav Kumar 3 months ago in reply to Nandhakumar Shanmugam

TI__Intellectual 2690 points

Hi,

Nandhakumar Shanmugam said:
For YOLOP (Detection + Drivable Area Segmentation + Lane Line Segmentation), is any special configuration required during compilation? In SDK 11_00_06_00, the examples appear to use a single task type during compilation. How should a multi-head model such as YOLOP be compiled?

There is no difference in TIDL compilation wrt number of inputs, outputs. When there are multiple inputs, onnx returns dict as output and you can access your outputs (this is standard in onnx). You can refer examples/osrt_python/advanced_examples/unit_tests_validation/ort/onnxrt_ep.py

Nandhakumar Shanmugam said:
What is the recommended approach to compile and deploy a multi-head model with TIDL?

No difference in compilation and inference. I am not sure about how are you going to deploy your model with some application but simple device inference has no difference wrt multiple outputs.

Nandhakumar Shanmugam said:
Does YOLOP deployment require a prototxt file, or is the ONNX model sufficient for compilation and deployment? If a prototxt file is required, what is the recommended way to generate it for YOLOP?

Prototxt file is optional in object detection models because tidl has optimised standard post processing backbones like yolo (https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/od_meta_arch.md) which will give better performance but it is optional.

Nandhakumar Shanmugam said:
If possible, could you please share a reference compilation script and a sample model_config for a multi-head model such as YOLOP?

There is nothing special about multi-head model. Just for reference about handing multiple inputs, you can refer You can refer examples/osrt_python/advanced_examples/unit_tests_validation/ort/onnxrt_ep.py

Nandhakumar Shanmugam said:
Should NMS be included inside the ONNX model, or is it recommended to handle NMS as post-processing?

If you are using prototxt then you can set in compilation options itself otherwise you may do that in post processing. https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/model_compilation.md#object-detection-specific-options

Nandhakumar Shanmugam said:
n the E2E thread "SK-TDA4VM: YOLOP compile", a YOLOP model was shared and validated on an older SDK version. Can the same model and deployment approach be used with SDK 11_00_06_00, or are any modifications required for the current SDK?

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1161201/sk-tda4vm-yolop-compile/4374851

Yes, it can be used (that method is old file based method TIDLRT). Since you are using osrt so i would suggest you use this onnxrt_ep.py file itself.

Nandhakumar Shanmugam said:
What exact layer change and modification need to be done for YOLOP to make it compatible with TDA4VM?

This is mentioned in compilation logs. For some layers like maxpool where kernel size issue is present you may use https://github.com/TexasInstruments/edgeai-tidl-tools/tree/master/osrt-model-tools/osrt_model_tools/onnx_tools/tidl_onnx_model_optimizer . For other layers, you can check the constraints here https://github.com/TexasInstruments/edgeai-tidl-tools/blob/11_00_06_00/docs/supported_ops_rts_versions.md .

Processors

Processors forum

TDA4VM: TDA4VM: Need support on YOLOP source compilation using TIDL and deployment on TDA platform