J721EXCPXEVM: Using edgeai-tidl-tools to convert onnx(edgeai-yolov5) into bin does not work

Hi,

Thanks for the question.

Due to limited bandwidth am not able to take look at it, i will try to loop back next week.

Hi Pratik Kedar,

I once copied edgeai_tidl_tools to the SD card and successfully ran "/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/scripts/scripts/run_python_examples.sh" on the TDA4VM EVB board.

root@j721e-evm:~# cd /edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools
root@j721e-evm:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools# export SOC=am68pa
root@j721e-evm:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools# export LD_LIBRARY_PATH=:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/tidl_tools:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/tidl_tools/osrt_deps:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/tidl_tools/osrt_deps/opencv/
root@j721e-evm:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools# export ARM64_GCC_PATH=/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu
root@j721e-evm:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools# export CGT7X_ROOT=/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/ti-cgt-c7000_3.1.0.LTS
root@j721e-evm:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools# export TIDL_TOOLS_PATH=/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/tidl_tools
root@j721e-evm:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools# source ./scripts/run_python_examples.sh
ARM Architecture
run python3 onnxrt_ep.py
Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['yolov5-ti-pld-20240521']


Running_Model :  yolov5-ti-pld-20240521  

libtidl_onnxrt_EP loaded 0xaf2cc70
Final number of subgraphs created are : 1, - Offloaded Nodes - 293, Total Nodes - 293
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  1081.603541 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  1081.606716 s:  VX_ZONE_INIT:Enabled
  1081.606725 s:  VX_ZONE_ERROR:Enabled
  1081.606731 s:  VX_ZONE_WARNING:Enabled
  1081.610638 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
  1081.611043 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
  1081.611142 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
  1081.611214 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
  1081.611223 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
  1081.615559 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!

Saving image to  ../../../output_images/

 
Completed_Model :     1, Name : yolov5-ti-pld-20240521                            , Total time :       7.78, Offload Time :       7.55 , DDR RW MBs : 0, Output File : py_out_yolov5-ti-pld-20240521_ADE_val_00000000.jpg
 
 
  1085.545489 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
  1085.549895 s:  VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
IPC: Deinit ... !!!
IPC: DeInit ... Done !!!
MEM: Deinit ... !!!
DDR_SHARED_MEM: Alloc's: 7 alloc's of 17622080 bytes
DDR_SHARED_MEM: Free's : 7 free's  of 17622080 bytes
DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes
MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
root@j721e-evm:/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools#

Checking "/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/output_images/py_out_yolov5-ti-pld-20240521_ADE_val_00000000.jpg" I can find my expected result.
So I assume "/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/model-artifacts/yolov5-ti-pld-20240521/subgraph_0_tidl_net.bin" and "/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/model-artifacts/yolov5-ti-pld-20240521/subgraph_0_tidl_io_1.bin" are working fine.
Then I copy the "/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/model-artifacts/yolov5-ti-pld-20240521/subgraph_0_tidl_net.bin" and "/edgeai-tidl-tools-09_02_07_00/edgeai-tidl-tools/model-artifacts/yolov5-ti-pld-20240521/subgraph_0_tidl_io_1.bin" to "/opt/model_zoo/yolov5-ti-pld-20240521/artifacts/subgraph_0_tidl_net.bin" and "/opt/model_zoo/yolov5-ti-pld-20240521/artifacts/subgraph_0_tidl_io_1.bin".
Modify "/opt/vision_apps/app_od.cfg" so that vx_app_tidl_od.out can load net and io bin.
When vx_app_tidl_od.out is executed, EVB will not be able to display the screen normally.

root@j721e-evm:~# cd /opt/vision_apps
root@j721e-evm:/opt/vision_apps# source ./vision_apps_init.sh
root@j721e-evm:/opt/vision_apps# cd /opt/vision_apps
root@j721e-evm:/opt/vision_apps# ./vx_app_tidl_od.out --cfg app_od.cfg

If you restore the settings of tidl_config and tidl_network. That is, when set to tidl_io_peele_300_1.bin and tidl_net_peele_300.bin, the screen will display normally.

So I guess there should be some problem with subgraph_0_tidl_io_1.bin and subgraph_0_tidl_net.bin but I don't know how to rule it out.

Please help me.

Besr regard,
George Lin

Hi Pratik Kedar,

Can I give you my onnx and then you help me try to transfer onnx to bin using your tool?
I want to check if my edgeai-tidl-tools is working properly.

Best regard,
George Lin

Hi,

Thank You for your continues interest in this, i will circle back coming week to update the thread, due to my limited bandwidth.

Thanks 

Hi Pratik Kedar,

I have been waiting for your helpful reply for over 11 days.
If you are still unable to return to the office at this time, please forward this issue to another of your colleagues who can take responsibility for the issue.

Because my department manager has begun to be concerned about this progress, I cannot solve this problem just by relying on my own analysis. I need experts from TI to help me fix this problem.
The DEBUG work our company has done so far is as follows:

1. Clarify whether the onnx is self-made by our company.
Test method: Use yolov5s6_640_ti_lite_37p4_56p0.onnx and yolov5s6_640_ti_lite_metaarch.prototxt of edgeai-yolov5 in github to create detections_tidl_io_1.bin and detections_tidl_net.bin, and put them into EVB Board for testing.
Result: The problem still exists.

2. Clarify whether it is a problem with the edgeai-tidl-tools execution environment.
Test method: Find two computers, and two people execute edgeai-tidl-tools to make a BIN file and put it into the EVB Board for testing.
Result: The problem still exists.

Note: It is also possible that there is a problem with the execution of edgeai-tidl-tools on both computers, so it is not completely clear that it is a problem with the edgeai-tidl-tools environment and execution method!
I need TI to help convert the ONNX provided by my company into BIN, and then provide BIN to my company for testing to clarify the problem.
PLC ONNX:yolov5-ti-pld-20240521.onnx

PLC PROTOTXT:yolov5-ti-pld-20240521.prototxt

output_images:py_out_yolov5-ti-pld-20240521_ADE_val_00000000.jpg

3. Debug code to find problem points.

It was found that the code in drawBox() in /ti-processor-sdk-rtos-j721e-evm-09_02_00_05/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c was commented out.
Not letting it do its original job doesn't have this problem.
So the problem is caused by Draw Box.
However, the parameters substituted by drawBox() are all the results of AI recognition.
But where and how to troubleshoot these results requires the assistance of TI experts. grateful!

Best regard,
George Lin

Hi Pratik Kedar,
How do I pass yolov5-ti-pld-20240521.onnx and yolov5-ti-pld-20240521.prototxt to you?
Best regard,
George Lin

Hi George,

Due to limited bandwidth am unable to asses with this issue at the moment, am doing my best to come back on this.

Please connect with local FAE regarding prioritization of this issue.

Thanks 

Hi Pratik Kedar,
I found that the problem was at C7x_1.
[C7x_1 ]    412.410605 s:  VX_ZONE_INFO:[ownTargetKernelExecute:398] Done executing process callback for kernel [com.ti.tidl]
[C7x_1 ]    412.410640 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=62, pipe=1) executing on target 00000200 ... DONE !!!
[C7x_1 ]    412.410680 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:478] Parameter released (node=62, pipe=1, data_ref_q=168, queue=173, ref=95)
[C7x_1 ]    412.410708 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:515] Re-triggering (node=59)
[C7x_1 ]    412.410750 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:496] Parameter NOT released (node=62, pipe=1, data_ref_q=175, ref=97, users=1)
[C7x_1 ]    412.410795 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:616] Node (node=63, pipe=2) acquiring parameters on target 00000200
[C7x_1 ]    412.410830 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:211] Parameter ALREADY acquired (node=63, pipe=2, data_ref_q=169, ref=30)
[C7x_1 ]    412.410867 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:174] Parameter acquire failed ... BLOCKING (node=63, pipe=2, data_ref_q=176, queue=180)
[C7x_1 ]    412.410898 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireAllParameters:564] Parameter acquire ... ABORTING (node=63, pipe=2)
[C7x_1 ]    412.410925 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:717] Node (node=63, pipe=2) ... BLOCKED for resources on target 00000200
[C7x_1 ]    412.419688 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=64, pipe=3) ... BLOCKED for previous pipe instance node (node=63) to complete !!!

Can you help me and tell me how to solve this problem?
Below is the complete serial port log.
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   411.370411 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   411.373923 s:  VX_ZONE_INIT:Enabled
   411.373952 s:  VX_ZONE_ERROR:Enabled
   411.373959 s:  VX_ZONE_WARNING:Enabled
   411.376129 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
   411.376244 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
   411.376340 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
   411.376411 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
   411.376421 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
   411.379756 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Default param set!
Parsed user params!
Updated user params!
Scaler Init Done!
TIDL Init Done!
Pre Proc Update Done!
Pre Proc Init Done!
Draw detections Update Done!
Draw Detections Init Done!
Img Mosaic Init Done!
Display Init Done!
App Init Done!
App Create Graph Done!
[C7x_1 ]    411.825782 s:  VX_ZONE_INFO:Enabled
[C7x_1 ]    411.825801 s:  VX_ZONE_INFO:[tivxKernelTIDLCreate:721] DSP tivxKernelTIDLCreate -- num_params = 8
[C7x_1 ]    411.826162 s:  VX_ZONE_INFO:[tivxKernelTIDLCreate:852] Enabling preemption
[C7x_1 ]    411.826191 s:  VX_ZONE_INFO:[tivxKernelTIDLCreate:891] L1 = 16 KB, L2 = 448 KB, L3 = 7968 KB
[C7x_1 ]    411.826214 s:  VX_ZONE_INFO:[tivxAlgiVisionCreate:300] Calling ialg.algNumAlloc ...
[C7x_1 ]    411.826234 s:  VX_ZONE_INFO:[tivxAlgiVisionCreate:304] Allocating 15 memory records ...
[C7x_1 ]    411.826258 s:  VX_ZONE_INFO:[tivxAlgiVisionCreate:314] Calling ialg.algAlloc ...
[C7x_1 ]    411.849066 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 0 @ space = 17, size = 19712, align = 128 ...
[C7x_1 ]    411.849105 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 0 @ space = 17 and size = 19712, addr = 117026d00 ...
[C7x_1 ]    411.849135 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 1 @ space = 17, size = 656, align = 128 ...
[C7x_1 ]    411.849166 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 1 @ space = 17 and size = 656, addr = 11702bb00 ...
[C7x_1 ]    411.849195 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 2 @ space = 0, size = 16384, align = 128 ...
[C7x_1 ]    411.849225 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 2 @ space = 0 and size = 16384, addr = 64e00000 ...
[C7x_1 ]    411.849254 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 3 @ space = 1, size = 458752, align = 128 ...
[C7x_1 ]    411.849284 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 3 @ space = 1 and size = 458752, addr = 64800000 ...
[C7x_1 ]    411.849313 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 4 @ space = 2, size = 8159232, align = 128 ...
[C7x_1 ]    411.849342 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 4 @ space = 2 and size = 8159232, addr = 70020000 ...
[C7x_1 ]    411.849372 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 5 @ space = 17, size = 14352064, align = 128 ...
[C7x_1 ]    411.849546 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 5 @ space = 17 and size = 14352064, addr = 11702bf00 ...
[C7x_1 ]    411.849576 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 6 @ space = 17, size = 15232, align = 128 ...
[C7x_1 ]    411.849606 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 6 @ space = 17 and size = 15232, addr = 100000000 ...
[C7x_1 ]    411.849636 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 7 @ space = 17, size = 3147008, align = 128 ...
[C7x_1 ]    411.849694 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 7 @ space = 17 and size = 3147008, addr = 117ddbf00 ...
[C7x_1 ]    411.849723 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 8 @ space = 17, size = 128, align = 128 ...
[C7x_1 ]    411.849752 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 8 @ space = 17 and size = 128, addr = 100003c00 ...
[C7x_1 ]    411.849780 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 9 @ space = 17, size = 3200, align = 128 ...
[C7x_1 ]    411.849810 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 9 @ space = 17 and size = 3200, addr = 100004000 ...
[C7x_1 ]    411.849839 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 10 @ space = 17, size = 1385528, align = 128 ...
[C7x_1 ]    411.849882 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 10 @ space = 17 and size = 1385528, addr = 1180dc500 ...
[C7x_1 ]    411.849913 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 11 @ space = 17, size = 13995135, align = 128 ...
[C7x_1 ]    411.849942 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 11 @ space = 17 and size = 13995135, addr = 100005000 ...
[C7x_1 ]    411.849972 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 12 @ space = 17, size = 8159232, align = 128 ...
[C7x_1 ]    411.850002 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 12 @ space = 17 and size = 8159232, addr = 11822eb00 ...
App Verify Graph Done!
[C7x_1 ]    411.850090 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 13 @ space = 17, size = 14118956, align = 128 ...
[C7x_1 ]    411.850120 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 13 @ space = 17 and size = 14118956, addr = 1189f6c00 ...
[C7x_1 ]    411.850223 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:180] Allocating memory record 14 @ space = 17, size = 1, align = 128 ...
[C7x_1 ]    411.850276 s:  VX_ZONE_INFO:[tivxAlgiVisionAllocMem:189] Allocated memory record 14 @ space = 17 and size = 1, addr = 11976de00 ...
[C7x_1 ]    411.850306 s:  VX_ZONE_INFO:[tivxAlgiVisionCreate:325] Calling ialg.algInit ...
[C7x_1 ]    411.907120 s:  VX_ZONE_INFO:[tivxAlgiVisionCreate:375] Created AlgiVision handle.
[C7x_1 ]    411.907152 s:  VX_ZONE_INFO:[ownTargetKernelCreate:332] Done executing create callback for kernel [com.ti.tidl]
App Send MSC Command Done!
App Verify Graph Done!
Running frame 2307


 =========================
 TIDL Demo - Object Detection
 =========================

 p: Print performance statistics

 x: Exit

 Enter Choice: App Reading Input Done!
App Process Graph Done!
app_tidl_od: Frame ID 2307 of 2707 ... Done.
Running frame 2308
App Reading Input Done!
App Reading Input Done!
App Process Graph Done!
app_tidl_od: Frame ID 2308 of 2707 ... Done.
Running frame 2309
[C7x_1 ]    412.309164 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:616] Node (node=48, pipe=0) acquiring parameters on target 00000200
[C7x_1 ]    412.309206 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:211] Parameter ALREADY acquired (node=48, pipe=0, data_ref_q=167, ref=30)
[C7x_1 ]    412.309245 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:201] Parameter acquired (node=48, pipe=0, data_ref_q=174, queue=180, ref=27)
[C7x_1 ]    412.309277 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:667] Node (node=48, pipe=0) executing on target 00000200
[C7x_1 ]    412.309318 s:  VX_ZONE_INFO:[ownTargetKernelExecute:390] Executing process callback for kernel [com.ti.tidl]
[C7x_1 ]    412.309345 s:  VX_ZONE_INFO:[tivxKernelTIDLProcess:299] DSP tivxKernelTIDLProcess -- num_params = 8, kernel = b2eef6e0
App Reading Input Done!
App Process Graph Done!
app_tidl_od: Frame ID 2309 of 2707 ... Done.
Running frame 2310
App Reading Input Done!
App Process Graph Done!
app_tidl_od: Frame ID 2310 of 2707 ... Done.
Running frame 2311
App Reading Input Done!
App Process Graph Done!
app_tidl_od: Frame ID 2311 of 2707 ... Done.
Running frame 2312
[C7x_1 ]    412.344416 s:    0         1.00000 -335419050746174976000000000000000000000.00000 339672414016927948800000000000000000000.00000 6
[C7x_1 ]    412.344470 s:  In sufficient Memory to write Traces, requested size is : 3145728, allocated size : 128
[C7x_1 ]    412.344495 s: Trace write failed
[C7x_1 ]    412.357674 s: TIDL_updateLocConfHeadPtrs() cycle is 7517
[C7x_1 ]    412.357708 s: TIDL_allocInternalMemBuffers() cycle is 2312
[C7x_1 ]    412.359175 s: Maximum COunt M  is 0
[C7x_1 ]    412.359199 s: TIDL_validPointScoreCalc() cycle is 1464150
[C7x_1 ]    412.359222 s: For class = 0, countK = 0
[C7x_1 ]    412.359242 s: For class = 0, numDet = 0
[C7x_1 ]    412.359262 s: For class = 1, countK = 0
[C7x_1 ]    412.359281 s: For class = 1, numDet = 0
[C7x_1 ]    412.359299 s: For class = 2, countK = 0
[C7x_1 ]    412.359317 s: For class = 2, numDet = 0
[C7x_1 ]    412.359335 s: For class = 3, countK = 0
[C7x_1 ]    412.359353 s: For class = 3, numDet = 0
[C7x_1 ]    412.359370 s: For class = 4, countK = 0
[C7x_1 ]    412.359388 s: For class = 4, numDet = 0
[C7x_1 ]    412.359406 s: For class = 5, countK = 0
[C7x_1 ]    412.359424 s: For class = 5, numDet = 0
[C7x_1 ]    412.359442 s: For class = 6, countK = 0
[C7x_1 ]    412.359460 s: For class = 6, numDet = 0
[C7x_1 ]    412.359478 s: For class = 7, countK = 0
[C7x_1 ]    412.359495 s: For class = 7, numDet = 0
[C7x_1 ]    412.359513 s: For class = 8, countK = 0
[C7x_1 ]    412.359531 s: For class = 8, numDet = 0
[C7x_1 ]    412.359549 s: For class = 9, countK = 0
[C7x_1 ]    412.359567 s: For class = 9, numDet = 0
[C7x_1 ]    412.359585 s: For class = 10, countK = 0
[C7x_1 ]    412.359603 s: For class = 10, numDet = 0
[C7x_1 ]    412.359622 s: For class = 11, countK = 0
[C7x_1 ]    412.359640 s: For class = 11, numDet = 0
[C7x_1 ]    412.359658 s: For class = 12, countK = 0
[C7x_1 ]    412.359676 s: For class = 12, numDet = 0
[C7x_1 ]    412.359695 s: For class = 13, countK = 0
[C7x_1 ]    412.359713 s: For class = 13, numDet = 0
[C7x_1 ]    412.359731 s: For class = 14, countK = 0
[C7x_1 ]    412.359749 s: For class = 14, numDet = 0
[C7x_1 ]    412.359767 s: For class = 15, countK = 0
[C7x_1 ]    412.359786 s: For class = 15, numDet = 0
[C7x_1 ]    412.359804 s: For class = 16, countK = 0
[C7x_1 ]    412.359822 s: For class = 16, numDet = 0
[C7x_1 ]    412.359840 s: For class = 17, countK = 0
[C7x_1 ]    412.359858 s: For class = 17, numDet = 0
[C7x_1 ]    412.359876 s: For class = 18, countK = 0
[C7x_1 ]    412.359895 s: For class = 18, numDet = 0
[C7x_1 ]    412.359913 s: For class = 19, countK = 0
[C7x_1 ]    412.359931 s: For class = 19, numDet = 0
[C7x_1 ]    412.359952 s: TIDL_topKSelection() cycle is 362021
[C7x_1 ]    412.359976 s: TIDL_sparseLocDataFetch() cycle is 2657
[C7x_1 ]    412.359999 s: TIDL_boxParamsDecoding() cycle is 1323
[C7x_1 ]    412.360022 s: TIDL_applyNMSFast() cycle is 359486
[C7x_1 ]    412.360045 s: TIDL_objOuputPreperation() cycle is 2058
[C7x_1 ]    412.360307 s:  VX_ZONE_INFO:[ownTargetKernelExecute:398] Done executing process callback for kernel [com.ti.tidl]
App Reading Input Done!
App Process Graph Done!
app_tidl_od: Frame ID 2312 of 2707 ... Done.
[C7x_1 ]    412.360343 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=48, pipe=0) executing on target 00000200 ... DONE !!!
Running frame 2313
[C7x_1 ]    412.360383 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:478] Parameter released (node=48, pipe=0, data_ref_q=167, queue=173, ref=30)
[C7x_1 ]    412.360416 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:515] Re-triggering (node=58)
[C7x_1 ]    412.360457 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:496] Parameter NOT released (node=48, pipe=0, data_ref_q=174, ref=27, users=1)
[C7x_1 ]    412.360503 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:616] Node (node=62, pipe=1) acquiring parameters on target 00000200
[C7x_1 ]    412.360542 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:211] Parameter ALREADY acquired (node=62, pipe=1, data_ref_q=168, ref=95)
[C7x_1 ]    412.360580 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:201] Parameter acquired (node=62, pipe=1, data_ref_q=175, queue=180, ref=97)
[C7x_1 ]    412.360610 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:667] Node (node=62, pipe=1) executing on target 00000200
[C7x_1 ]    412.360648 s:  VX_ZONE_INFO:[ownTargetKernelExecute:390] Executing process callback for kernel [com.ti.tidl]
[C7x_1 ]    412.360674 s:  VX_ZONE_INFO:[tivxKernelTIDLProcess:299] DSP tivxKernelTIDLProcess -- num_params = 8, kernel = b2eef6e0
[C7x_1 ]    412.394996 s:    0         1.00000 -1163910987251626752000000000000000000.00000 339688315426057011200000000000000000000.00000 6
[C7x_1 ]    412.395049 s:  In sufficient Memory to write Traces, requested size is : 3145728, allocated size : 128
[C7x_1 ]    412.395074 s: Trace write failed
[C7x_1 ]    412.408059 s: TIDL_updateLocConfHeadPtrs() cycle is 7783
[C7x_1 ]    412.408093 s: TIDL_allocInternalMemBuffers() cycle is 2413
[C7x_1 ]    412.409558 s: Maximum COunt M  is 0
[C7x_1 ]    412.409583 s: TIDL_validPointScoreCalc() cycle is 1462822
[C7x_1 ]    412.409604 s: For class = 0, countK = 0
[C7x_1 ]    412.409625 s: For class = 0, numDet = 0
[C7x_1 ]    412.409645 s: For class = 1, countK = 0
[C7x_1 ]    412.409664 s: For class = 1, numDet = 0
[C7x_1 ]    412.409682 s: For class = 2, countK = 0
[C7x_1 ]    412.409700 s: For class = 2, numDet = 0
[C7x_1 ]    412.409719 s: For class = 3, countK = 0
[C7x_1 ]    412.409736 s: For class = 3, numDet = 0
[C7x_1 ]    412.409754 s: For class = 4, countK = 0
[C7x_1 ]    412.409772 s: For class = 4, numDet = 0
[C7x_1 ]    412.409790 s: For class = 5, countK = 0
[C7x_1 ]    412.409808 s: For class = 5, numDet = 0
[C7x_1 ]    412.409826 s: For class = 6, countK = 0
[C7x_1 ]    412.409843 s: For class = 6, numDet = 0
[C7x_1 ]    412.409861 s: For class = 7, countK = 0
[C7x_1 ]    412.409879 s: For class = 7, numDet = 0
[C7x_1 ]    412.409897 s: For class = 8, countK = 0
[C7x_1 ]    412.409915 s: For class = 8, numDet = 0
[C7x_1 ]    412.409933 s: For class = 9, countK = 0
[C7x_1 ]    412.409951 s: For class = 9, numDet = 0
[C7x_1 ]    412.409969 s: For class = 10, countK = 0
[C7x_1 ]    412.409987 s: For class = 10, numDet = 0
[C7x_1 ]    412.410006 s: For class = 11, countK = 0
[C7x_1 ]    412.410024 s: For class = 11, numDet = 0
[C7x_1 ]    412.410042 s: For class = 12, countK = 0
[C7x_1 ]    412.410061 s: For class = 12, numDet = 0
[C7x_1 ]    412.410079 s: For class = 13, countK = 0
[C7x_1 ]    412.410097 s: For class = 13, numDet = 0
[C7x_1 ]    412.410115 s: For class = 14, countK = 0
[C7x_1 ]    412.410133 s: For class = 14, numDet = 0
[C7x_1 ]    412.410152 s: For class = 15, countK = 0
[C7x_1 ]    412.410170 s: For class = 15, numDet = 0
[C7x_1 ]    412.410188 s: For class = 16, countK = 0
App Reading Input Done!
[C7x_1 ]    412.410206 s: For class = 16, numDet = 0
App Process Graph Done!
[C7x_1 ]    412.410224 s: For class = 17, countK = 0
app_tidl_od: Frame ID 2313 of 2707 ... Done.
Running frame 2314
[C7x_1 ]    412.410242 s: For class = 17, numDet = 0
[C7x_1 ]    412.410261 s: For class = 18, countK = 0
[C7x_1 ]    412.410279 s: For class = 18, numDet = 0
[C7x_1 ]    412.410297 s: For class = 19, countK = 0
[C7x_1 ]    412.410315 s: For class = 19, numDet = 0
[C7x_1 ]    412.410337 s: TIDL_topKSelection() cycle is 362502
[C7x_1 ]    412.410360 s: TIDL_sparseLocDataFetch() cycle is 2404
[C7x_1 ]    412.410383 s: TIDL_boxParamsDecoding() cycle is 1305
[C7x_1 ]    412.410406 s: TIDL_applyNMSFast() cycle is 360023
[C7x_1 ]    412.410429 s: TIDL_objOuputPreperation() cycle is 2382
[C7x_1 ]    412.410605 s:  VX_ZONE_INFO:[ownTargetKernelExecute:398] Done executing process callback for kernel [com.ti.tidl]
[C7x_1 ]    412.410640 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=62, pipe=1) executing on target 00000200 ... DONE !!!
[C7x_1 ]    412.410680 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:478] Parameter released (node=62, pipe=1, data_ref_q=168, queue=173, ref=95)
[C7x_1 ]    412.410708 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:515] Re-triggering (node=59)
[C7x_1 ]    412.410750 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:496] Parameter NOT released (node=62, pipe=1, data_ref_q=175, ref=97, users=1)
[C7x_1 ]    412.410795 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:616] Node (node=63, pipe=2) acquiring parameters on target 00000200
[C7x_1 ]    412.410830 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:211] Parameter ALREADY acquired (node=63, pipe=2, data_ref_q=169, ref=30)
[C7x_1 ]    412.410867 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:174] Parameter acquire failed ... BLOCKING (node=63, pipe=2, data_ref_q=176, queue=180)
[C7x_1 ]    412.410898 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireAllParameters:564] Parameter acquire ... ABORTING (node=63, pipe=2)
[C7x_1 ]    412.410925 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:717] Node (node=63, pipe=2) ... BLOCKED for resources on target 00000200
[C7x_1 ]    412.419688 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=64, pipe=3) ... BLOCKED for previous pipe instance node (node=63) to complete !!!

Hi George,

I want to summarize the current issues you are seeing and the steps you have taken so far - please correct me if any of the following is incorrect. 

1. You have a YOLOV5 model that is based of of edgeai-yolov5 examples, and you have confirmed that there is correct inference on edgeai-tidl-tools

   george lin said:

   And confirm edgeai-tidl-tools/output_images/py_out_yolov5-ti-pld-20240521_ADE_val_00000000.jpg to confirm that the desired parking space corner is obtained.
2. After changing the default configuration of the Deep Learning Object Detection demo application (linked here), to point to your model, you are not seeing any result from the screen

   george lin said:

   At this time, you will find that the screen is black, it seems that the EVB has crashed, no images can be seen, and there is no UART output.
   How to troubleshoot this problem?
3. You have tried running the default model that this demo application supports and are seeing the expected behavior.

   george lin said:

   If you restore the settings of tidl_config and tidl_network. That is, when set to tidl_io_peele_300_1.bin and tidl_net_peele_300.bin, the screen will display normally.

   george lin said:

   It was found that the code in drawBox() in /ti-processor-sdk-rtos-j721e-evm-09_02_00_05/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c was commented out.
   Not letting it do its original job doesn't have this problem.
   So the problem is caused by Draw Box.
   However, the parameters substituted by drawBox() are all the results of AI recognition.
   But where and how to troubleshoot these results requires the assistance of TI experts. grateful!

   Would you be able to further clarify your debugging steps at this point?

   george lin said:

   I found that the problem was at C7x_1.

   george lin said:

   [C7x_1 ]    412.419688 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=64, pipe=3) ... BLOCKED for previous pipe instance node (node=63) to complete !!!

   Could you clarify if this is the remote core log from when you run the demo application with your model? Does the log hang after this point in a way that is unrecoverable? 

   Also, do note that the demo application from Vision Apps has pre and post processing OpenVX code that would differ from running inference on edgeai-tidl-tools, your model would need to be compatible with the data flow diagram shown for that application. Have you modified this code in anyway? 

   Best,

   Asha

Hi Asha,

1. You have a YOLOV5 model that is based of of edgeai-yolov5 examples, and you have confirmed that there is correct inference on edgeai-tidl-tools
    george lin comment:
        Yes.
        I have tested copying edgeai-tidl-tools to EVB for execution.
        After executing python3 onnxrt_ep.py, /edgeai-tidl-tools/output_images/py_out_yolov5-ti-pld-20240521_ADE_val_000.jpg is generated.
        It can be seen that /edgeai-tidl-tools/model-artifacts/yolov5-ti-pld-20240521/subgraph_0_tidl_net.bin can be executed correctly.

2. After changing the default configuration of the Deep Learning Object Detection demo application (linked here), to point to your model, you are not seeing any result from the screen
    george lin comment:
        After enabling the following variables and #define, I can get the error message from the UART.
            tidlLogLevel = 1;
            tidlWriteLevel = 3;
            #define SYNCHRONIZATION_DEBUG
            #define TIDL_OD_L1_DEBUG
            #define TIDL_OD_L2_DEBUG
            #define TIDL_REF_BATCH_NORM_DEBUG
            #define TIDL_REF_BATCH_NORM_DEBUG
            #define TRANSPOSE_DEBUG 1
            #define DEBUG_PRINT 1

3. You have tried running the default model that this demo application supports and are seeing the expected behavior.
    george lin said:
        If you restore the settings of tidl_config and tidl_network. That is, when set to tidl_io_peele_300_1.bin and tidl_net_peele_300.bin, the screen will display normally.
            george lin comment:
                Yes.

    george lin said:
        It was found that the code in drawBox() in /ti-processor-sdk-rtos-j721e-evm-09_02_00_05/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c was commented out.
        Not letting it do its original job doesn't have this problem.
        So the problem is caused by Draw Box.
        However, the parameters substituted by drawBox() are all the results of AI recognition.
        But where and how to troubleshoot these results requires the assistance of TI experts. grateful!
            george lin comment:
                I may be wrong about this part, please refer to the final UART LOG.

4. Would you be able to further clarify your debugging steps at this point?
    george lin said:
        I found that the problem was at C7x_1.
    george lin said:
        [C7x_1 ]    412.419688 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=64, pipe=3) ... BLOCKED for previous pipe instance node (node=63) to complete !!!    
        
        george lin comment:
            a. Test Step:
                i. use the /vision_apps/apps/dl_demos/app_tidl_od to test
                ii. enable #define APP_DEBUG in /vision_apps/apps/dl_demos/app_tidl_od/app_common.h
                iii. Enable the following variables and #define in /c7x-mma-tidl/ti_dl
                    tidlLogLevel = 1;
                    tidlWriteLevel = 3;
                    #define SYNCHRONIZATION_DEBUG
                    #define TIDL_OD_L1_DEBUG
                    #define TIDL_OD_L2_DEBUG
                    #define TIDL_REF_BATCH_NORM_DEBUG
                    #define TIDL_REF_BATCH_NORM_DEBUG
                    #define TRANSPOSE_DEBUG 1
                    #define DEBUG_PRINT 1
                iv. make sdk & make linux_fs_install_sd to build and install sdcard.
                v. Copy subgraph_0_tidl_io_1.bin and subgraph_0_tidl_net.bin to sdcard and modify /opt/vision_apps/app_od.cfg as described below.
                    -tidl_config   /opt/vision_apps/test_data/psdkra/tidl_models/tidl_io_peele_300_1.bin
                    +tidl_config   /opt/model_zoo/yolov5-ti-pld-20240521/artifacts/subgraph_0_tidl_io_1.bin
                    -tidl_network  /opt/vision_apps/test_data/psdkra/tidl_models/tidl_net_peele_300.bin
                    +tidl_network  /opt/model_zoo/yolov5-ti-pld-20240521/artifacts/subgraph_0_tidl_net.bin

                    -start_frame 500
                    +start_frame 2307

                    -in_size   1024 512
                    +in_size   512 512

                    -dl_size   1024 512
                    +dl_size   512 512

                    -out_size  1024 512
                    +out_size  512 512

                    -num_classes 90
                    +num_classes 20
                vi. Follow the steps below to execute vx_app_tidl_od.out.
                    cd /opt/vision_apps
                    source ./vision_apps_init.sh
                    ./vx_app_tidl_od.out --cfg app_od.cfg
                vii. Confirmation screen and UART output.
            Result:
                The screen display is as follows.

                UART displays error message.
                    [C7x_1 ]     82.060935 s:  VX_ZONE_INFO:[ownTargetKernelExecute:398] Done executing process callback for kernel [com.ti.tidl]
                    [C7x_1 ]     82.060969 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=62, pipe=1) executing on target 00000200 ... DONE !!!
                    [C7x_1 ]     82.061009 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:478] Parameter released (node=62, pipe=1, data_ref_q=168, queue=173, ref=95)
                    [C7x_1 ]     82.061042 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:515] Re-triggering (node=59)
                    [C7x_1 ]     82.061083 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:496] Parameter NOT released (node=62, pipe=1, data_ref_q=175, ref=97, users=1)
                    [C7x_1 ]     82.061128 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:616] Node (node=63, pipe=2) acquiring parameters on target 00000200
                    [C7x_1 ]     82.061164 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:211] Parameter ALREADY acquired (node=63, pipe=2, data_ref_q=169, ref=30)
                    [C7x_1 ]     82.061201 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:174] Parameter acquire failed ... BLOCKING (node=63, pipe=2, data_ref_q=176, queue=180)
                    [C7x_1 ]     82.061232 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireAllParameters:564] Parameter acquire ... ABORTING (node=63, pipe=2)
                    [C7x_1 ]     82.061259 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:717] Node (node=63, pipe=2) ... BLOCKED for resources on target 00000200
                    [C7x_1 ]     82.070026 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=64, pipe=3) ... BLOCKED for previous pipe instance node (node=63) to complete !!!

            Expect:
                The screen displays normally.

5. Could you clarify if this is the remote core log from when you run the demo application with your model?
    george lin comment:
        Yes. The log is come from UART.
Does the log hang after this point in a way that is unrecoverable?
    george lin comment:
        Yes, the program cannot be stopped via key x. Ctrl+c is required to stop this program execution.
    
6. Also, do note that the demo application from Vision Apps has pre and post processing OpenVX code that would differ from running inference on edgeai-tidl-tools, your model would need to be compatible with the data flow diagram shown for that application. Have you modified this code in anyway?
    george lin comment:
        I have not modified any code in /vision_apps/apps/dl_demos/app_tidl_od.
        I only modified /vision_apps/apps/dl_demos/app_tidl_od/app_common.h. Just enable #define APP_DEBUG.
        I always test by modifying app_od.cfg.

Best regard,
George Lin

Dear TI Experts,

Any updates on this issue?
Best regard,

George Lin

Dear TI Experts,


I'm looking for a yolov5 model to customize.
But I can't find any existing yolov5 models that can run with app_tidl_od.
Can you tell me which existing yolov5 model will work with app_tidl_od?


Best regard,
George Lin

Dear TI Experts,
I have a question about "numDetObjects".
I can find "numDetObjects" in "/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c" and "/c7x-mma-tidl/ti_dl/algo/src/tidl_detectionOutput.c".
I print the value of it.
I found that the value is always the same when running in the official version.
[C6x_2 ]     39.689065 s: AAAA /home/georgelin/ti-processor-sdk-rtos-j721e-evm-09_02_00_05_work/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c:276 tivxKernelDrawBoxDetectionsProcess()
[C6x_2 ]     39.689122 s: AAAA (The number of Det) pHeader->numDetObjects:14.000000
[C7x_1 ]     39.686665 s: AAAA src/tidl_detectionOutput.c:394 TIDL_detectionOutputProcess()
[C7x_1 ]     39.686690 s: AAAA keepKCnt:14
[C7x_1 ]     39.686705 s: AAAA params->keepTopK:200
[C7x_1 ]     39.686727 s: AAAA outputData->numDetObjects:14.000000
The numDetObjects value in "/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c" and "src/tidl_detectionOutput.c" is 14.000000.

But after I modified tidl_config and tidl_network in /rootfs/opt/vision_apps/app_od.cfg for customization, numDetObjects would no longer sync.
[C6x_2 ]    236.878782 s: AAAA /home/georgelin/ti-processor-sdk-rtos-j721e-evm-09_02_00_05_work/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c:276 tivxKernelDrawBoxDetectionsProcess()
[C6x_2 ]    236.878830 s: AAAA (The number of Det) pHeader->numDetObjects:-1.000000
[C7x_1 ]    236.876966 s: AAAA src/tidl_detectionOutput.c:394 TIDL_detectionOutputProcess()
[C7x_1 ]    236.876992 s: AAAA keepKCnt:0
[C7x_1 ]    236.877008 s: AAAA params->keepTopK:200
[C7x_1 ]    236.877029 s: AAAA outputData->numDetObjects:0.000000
You can find that in "/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c", numDetObjects is -1.000000.
But in "src/tidl_detectionOutput.c", numDetObjects is 0.000000.
Why? This is such a strange question!
If I want to change the AI ​​Model, besides modifying app_od.cfg, what else should I modify?
Best regard,
George Lin

Hi George,

I do apologize for the delay in response as I have been on business travel. 

Thank you for confirming that the model itself works fine with edgeai-tidl-tools when you run on the target device. 

I'm reassigning to our vision apps/openvx expert to give you better clarity on how that could be integrated in the demo and debug the errors you are seeing. Please expect a response from them.

Best,

Asha

Dear Asha,
There has been no feedback on this issue for a long time.
Can you assign someone to handle this issue before Friday?
Best regard,
George Lin

Hi George,

Sorry for the delay in response here.

Can you share me the logs with tidlLogLevel = 2 and tidWriteLevel = 0 ?

I would like check if the hang is seen within the layer execution?

Additionally, I see that you have reduced the number of classes, from 90 to 20. I believe this is aligned as per your model right?

Regards,
Nikhil

Nikhil,

Sorry for feedback so late. I upload new logs to the following URL:
https://drive.google.com/drive/folders/14_8rOc6Q_e9T21_XKyj6K1yCOze-5wzE?usp=sharing

Please take a look.
If it is not complete or has any failures. Or if you have any other requirements, please feel free to let me know. Thanks.

Best regard,
George Lin

Hi George,

Thank you for sharing the logs.

I was expecting the below errors in the logs you shared, but it seems that the model (i.e. tidl node) has been executed successfully.

May I know what your current issue is?

Is it in the post processing node?

george lin said:

                    [C7x_1 ]     82.061042 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:515] Re-triggering (node=59)
                    [C7x_1 ]     82.061083 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:496] Parameter NOT released (node=62, pipe=1, data_ref_q=175, ref=97, users=1)
                    [C7x_1 ]     82.061128 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:616] Node (node=63, pipe=2) acquiring parameters on target 00000200
                    [C7x_1 ]     82.061164 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:211] Parameter ALREADY acquired (node=63, pipe=2, data_ref_q=169, ref=30)
                    [C7x_1 ]     82.061201 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:174] Parameter acquire failed ... BLOCKING (node=63, pipe=2, data_ref_q=176, queue=180)
                    [C7x_1 ]     82.061232 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireAllParameters:564] Parameter acquire ... ABORTING (node=63, pipe=2)
                    [C7x_1 ]     82.061259 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:717] Node (node=63, pipe=2) ... BLOCKED for resources on target 00000200
                    [C7x_1 ]     82.070026 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=64, pipe=3) ... BLOCKED for previous pipe instance node (node=63) to complete !!!

Regards,

Nikhil

Hi Nikhil,

I think the tidl node might run successfully. But the output should be wrong.
Because I printed the value of numDetObjects in tivxKernelDrawBoxDetectionsProcess() to be <0.
"[C6x_2 ]    236.878830 s: AAAA (The number of Det) pHeader->numDetObjects:-1.000000"
The value of numDetObjects should be >= 0.

Please look at line 261.

When pHeader->numDetObjects is -1.000000, numObjs will be assigned 4,294,967,295.
So the for loop on line 269 will run 4,294,967,295 times.
C6x_2 takes too much time to run.
Then C7x be BLOCKED.
                    [C7x_1 ]     82.060935 s:  VX_ZONE_INFO:[ownTargetKernelExecute:398] Done executing process callback for kernel [com.ti.tidl]
                    [C7x_1 ]     82.060969 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=62, pipe=1) executing on target 00000200 ... DONE !!!
                    [C7x_1 ]     82.061009 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:478] Parameter released (node=62, pipe=1, data_ref_q=168, queue=173, ref=95)
                    [C7x_1 ]     82.061042 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:515] Re-triggering (node=59)
                    [C7x_1 ]     82.061083 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:496] Parameter NOT released (node=62, pipe=1, data_ref_q=175, ref=97, users=1)
                    [C7x_1 ]     82.061128 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:616] Node (node=63, pipe=2) acquiring parameters on target 00000200
                    [C7x_1 ]     82.061164 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:211] Parameter ALREADY acquired (node=63, pipe=2, data_ref_q=169, ref=30)
                    [C7x_1 ]     82.061201 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:174] Parameter acquire failed ... BLOCKING (node=63, pipe=2, data_ref_q=176, queue=180)
                    [C7x_1 ]     82.061232 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireAllParameters:564] Parameter acquire ... ABORTING (node=63, pipe=2)
                    [C7x_1 ]     82.061259 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:717] Node (node=63, pipe=2) ... BLOCKED for resources on target 00000200
                    [C7x_1 ]     82.070026 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=64, pipe=3) ... BLOCKED for previous pipe instance node (node=63) to complete !!!

I changed some code in vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c to avoid this error.
My changed code is as follows:
@@ -239,7 +242,11 @@
         TIDL_ODLayerHeaderInfo *pHeader;
         TIDL_ODLayerObjInfo *pObjInfo;
         vx_float32 *pOut;
+#if 1
+        vx_int32 numObjs;
+#else
         vx_uint32 numObjs;
+#endif
         vx_float32 *output_buffer;
         vx_size output_sizes[4];


I need you to help me analyze why pHeader->numDetObjects < 0?

Best regard,

George Lin

Hi George,

Does your model expect a tensor of float datatype or is it a 8-bit / 16 bit interger datatype?

The reason for this question is because the tivxImgPreProcNode() used in the preprocessing in this application only supports, 8-bit / 16 bit interger datatype tensor output.

Regards,

Nikhil

Hi Nikhil,

Does your model expect a tensor of float datatype or is it a 8-bit / 16 bit interger datatype?
    <-- Our model is a tensor of float datatype.
I have three questions for you.
1. All TI vision app do not support float datatype?
2. How do I change a tensor to an 8-bit/16-bit integer datatype?
3. If the "python3 onnxrt_ep.py" in edgeai-tidl-tools can excute the AI model correctly. I think "vision app" should properly execute AI models on EVB boards. But why not?

Best regard,
George Lin

Hi George,

In the vision_apps framework, the pre-proc and post-proc node are custom specific to the models. 

In your case, you cannot use tivxImgPreProcNode() as this does not support float datatype.

The datatype of the tensor is extracted from the config (iobufdesc) in subgraph_0_tidl_io_1.bin.

You could use tivxDLPreProc node instead to support pre-processing of floating datatype tensors.

Regards,

Nikhil

Hi Nikhil,

I used tivxDLPreProcNode() to replace tivxImgPreProcNode() in app_create_graph_pre_proc().
You can see my change file - "diff_change.txt".
But I still can't get the correct pHeader->numDetObjects in tivxKernelDrawBoxDetectionsProcess().
pHeader->numDetObjects is not > 0.
Can you help me again?
I updated the new log file—"teraterm_vx_app_tidl_od_20240913-1826.log".
You can get them at drive.google.com/.../1dvOD8OoTIMOVx9qjP33Yjkmo830PlK3e

Best regard,
George Lin

Hi Nikhil,

Should I use tivxDLDrawBoxNode() instead of tivxDrawBoxDetectionsNode()?
I will give it a try and report back with the results later.

Best regard.
George Lin

Hi George,

Sorry for the delay in response. I was vacation last week.

The tivxDLDrawBoxNode() is not a complete implementation. I see that the target implementation is to be filled by the user.

george lin said:

pHeader->numDetObjects < 0

This information is from the output tensor of the TIDL node right?

Can you dump the tensor output and tensor input to the TIDL node and check the same with a golden reference (working case)?

Regards,

Nikhil

Nikhil Dasan said:

Can you dump the tensor output and tensor input to the TIDL node and check the same with a golden reference (working case)?

Dear Nikhil,

Can you teach me how to dump tensor output and tensor input?

Best regard,

George Lin

Hi George,

In the file vision_apps/modules/src/app_tidl_module.c you can see a function writeTIDLOutput(). This dumps the output tensor of the TIDL node into a file.

You can do the same for the input tensor as well.

Regards,

Nikhil

Dear Nikhil,
I have added the following code to dump the logs.
-------------------------------------------------------------------------------------------------
@@ -78,7 +78,7 @@
 #include "app_display_module.h"
 #include "app_test.h"
 
-/* #define WRITE_INTERMEDIATE_OUTPUTS */
+#define WRITE_INTERMEDIATE_OUTPUTS      // AAAA
 
 #ifndef x86_64
 #define APP_ENABLE_PIPELINE_FLOW
@@ -1132,6 +1132,20 @@
             if ((obj->en_out_img_write == 1) && (status == VX_SUCCESS))
             {
                 APP_PRINTF("App Writing Outputs Start...\n");
+
+// AAAA
+#ifdef WRITE_INTERMEDIATE_OUTPUTS
+                snprintf(output_file_name, APP_MAX_FILE_PATH, "%s/%010d_scaler_%dx%d.yuv", obj->output_file_path, frame_id, obj->scalerObj.output[0].width, obj->scalerObj.output[0].height);
+                writeScalerOutput(output_file_name, scalerObj->output[0].arr);
+
+                snprintf(output_file_name, APP_MAX_FILE_PATH, "%s/pre_proc_output_%010d", obj->output_file_path, frame_id);
+                writePreProcOutput(output_file_name, &obj->preProcObj);
+
+                snprintf(output_file_name, APP_MAX_FILE_PATH, "%s/tidl_output_%010d", obj->output_file_path, frame_id);
+                writeTIDLOutput(output_file_name, &obj->tidlObj);
+#endif
+// AAAA
+
                 snprintf(output_file_name, APP_MAX_FILE_PATH, "%s/mosaic_output_%010d_1920x1080.yuv", obj->output_file_path, (frame_id - APP_BUFFER_Q_DEPTH));
                 status = writeMosaicOutput(output_file_name, mosaic_output_image);
                 APP_PRINTF("App Writing Outputs Done!\n");
-------------------------------------------------------------------------------------------------
I have uploaded the dump log to my google driver (drive.google.com/.../1octum1yy55R3hr6EJx-KdFRceK4A_wy2. Please help with analysis. Of course, my side will also continue to analyze simultaneously.
After unpacking "vision_app_dump.tar.gz", there will be a folder of vision_app_dump. I will explain what information is contained in this folder.

1. There are two folders under "vision_app_dump". One is "tidl_io_peele_300_1". The other is "yolov5s6_640_ti_lite_37p4_56p0".
Because I want to clarify the problem, I use the AI ​​model provided by TI for testing.
    a. "tidl_io_peele_300_1" is the AI ​​originally used by app_tidl_od.
    I also put "tidl_io_peele_300_1" in the SD Card into "vision_app_dump/tidl_io_peele_300_1/ONNX_BIN".
    vision_app_dump/tidl_io_peele_300_1/ONNX_BIN/tidl_io_peele_300_1.bin
    vision_app_dump/tidl_io_peele_300_1/ONNX_BIN/tidl_net_peele_300.bin
    There are "app_tidl_od_out_1024x512" and "app_tidl_od_out_1280x720" folders under the "tidl_io_peele_300_1" directory.
    They all use the same AI model (tidl_io_peele_300_1) for testing, but the input image sizes are different.
    "app_tidl_od_out_1024x512" uses TI's original 1024x512 image file test.
    "app_tidl_od_out_1280x720" is tested using the 1280x720 image file I added.
    Although the "app_tidl_od_out_1280x720" recognition is not ideal, it can be implemented.
    
    b. "yolov5s6_640_ti_lite_37p4_56p0" is the dump after testing with TI yolov5s6_640_ti_lite_37p4_56p0.onnx.
    The steps for this test are as follows:
        i.  Download "yolov5s6_640_ti_lite_37p4_56p0.onnx", "yolov5s6_640_ti_lite_metaarch.prototxt" from "">github.com/.../edgeai-yolov5-gplv3"
        ii. Convert to "detections_tidl_io_1.bin" and "detections_tidl_net.bin" through "edgeai-tidl-tools" (tag:08_06_00_05).
        p.s.I cannot use edgeai-tidl-tools(tag:09_02_09_00) to convert to bin. There will be "Segmentation fault (core dumped)" error. All versions 09 have this problem. I also cannot use version 10 because my beta version cannot be booted using the version 10 vision app. In the end, we had to return version 08 for testing.
        iii.    Place the generated "detections_tidl_net.bin" and "detections_tidl_io_1.bin" in the SD Card, and modify the tidl_config and tidl_network in app_od.cfg.
        tidl_config /opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_io_1.bin
        tidl_network /opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_net.bin
        iv. call "./vx_app_tidl_od.out --cfg app_od.cfg" to test.
        
        The "detections_tidl_io_1.bin", "detections_tidl_net.bin", "yolov5s6_640_ti_lite_37p4_56p0.onnx" and "yolov5s6_640_ti_lite_metaarch.prototxt" mentioned above are all stored in "vision_app_dump/yolov5s6_640_ti_lite_37p4_56p 0/ONNX_BIN".

    d. But there are still problems with implementation. The problem is still that pHeader->numDetObjects in "/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c" will become -1.
2. Because this time we are using "yolov5s6_640_ti_lite_37p4_56p0.onnx" for testing, there should be no need to use "tivxDLPreProc()".
3. I found that the pre_proc_output...yuv content under yolov5s6_640_ti_lite_37p4_56p0 is incorrect. But I don't know how to fix this problem. Could you please remind me again. grateful!!

Best regard,
George Lin

Hi George, 

Thank you for sharing the files.

A bit of clarification on this

george lin said:

Because this time we are using "yolov5s6_640_ti_lite_37p4_56p0.onnx" for testing, there should be no need to use "tivxDLPreProc()".

May I know why DLPreProc is not needed here? Is it because the datatype of this model requirement is not float?

george lin said:

I found that the pre_proc_output...yuv content under yolov5s6_640_ti_lite_37p4_56p0 is incorrect. But I don't know how to fix this problem. Could you please remind me again. grateful!!

If the model expects a float input, then pre_proc node used (i.e. ImgPreProc node) does not support the same. Hence it was suggested to use DLPreProc, which supports float datatype.

Regards,

Nikhil

Dear Nikhil,

Nikhil Dasan said:

Can you share me the logs with tidlLogLevel = 2 and tidWriteLevel = 0 ?

In vision_app(08_06_00_12) I can't find tidlLogLevel and tidWriteLevel.

If I want to share the same log as in vision_App (09_02_00_05), which file in vision_App (08_06_00_12) should I modify? How to modify?
grateful!!

Best regard,

George Lin

Hi George, 

Can you search for traceLogLevel and traceWriteLevel. in vision_apps/modules/src/app_tidl_module.c ?

Regards,

Nikhil

Dear Nikhil,

I have used tivxDLPreProcNode() to replace tivxImgPreProcNode(), and I uploaded my modified code to google driver.
(drive.google.com/.../1eucmlK5DaBrdWdeigqjN30f7vdm2VGCQ
Path: /2024-10-18/sourcecode/vision_apps/apps/dl_demos/app_tidl_od/app_pre_proc_module.c
Since there is no reference code to refer to, I modified it myself by referring to tivxImgPreProcNode().
The codes related to it are all under the directory /2024-10-18/sourcecode/vision_apps/apps/dl_demos/app_tidl_od
But the unfortunate news is that it still hasn't been implemented successfully.
I have modified /2024-10-18/sourcecode/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c so that it prints the value of pHeader->numDetObjects.
Also attach uart log --> /2024-10-18/teraterm_vx_app_tidl_od_20241018-1728.log
From teraterm_vx_app_tidl_od_20241018-1728.log, you can see that when pHeader->numDetObjects:-1.000000, the app no ​​longer prints log!!
Therefore using tivxDLPreProcNode() is not enough for this program to execute properly.
I found that the content of /2024-10-18/yolov5s6_640_ti_lite_37p4_56p0/app_tidl_od_out/pre_proc_output_0000000501_640x640_ch0.yuv is incorrect.

I use "writePreProcOutput(output_file_name, &obj->preProcObj);" to dump it out.

Do you have a way to make the content of pre_proc_output_0000000501_640x640_ch0.yuv correct?

Best regard,
George Lin

Dear Nikhil,
I had vision_apps/modules/src/app_tidl_module.c.
----------------------------------------------------------------------
prms->isInbufsPaded = 1;
prms->quantRangeExpansionFactor = 1.0;
prms->quantRangeUpdateFactor = 0.0;
+#if 1 // AAAA
+ prms->traceLogLevel = 2;
+ prms->traceWriteLevel = 0;
+#else // AAAA
prms->traceLogLevel = 0;
prms->traceWriteLevel = 0;
+#endif // AAAA
+ printf("AAAA %s:%d %s() prms->traceLogLevel:%d\n",__FILE__,__LINE__,__FUNCTION__, prms->traceLogLevel);
+ printf("AAAA %s:%d %s() prms->traceWriteLevel:%d\n",__FILE__,__LINE__,__FUNCTION__, prms->traceWriteLevel);
}
else
{
----------------------------------------------------------------------
And get the failed run log and pass run log.
I uploaded two new UART logs to Google Drive.
You can get them from "">drive.google.com/.../1NMQgVoQ6SjnXsw_Ng1qSgad28S7S07mT
In this folder, there are two UART logs.
One is "teraterm_vx_app_tidl_od_20241021-1429_yolov5s6_640_ti_lite_37p4_56p0.log" and the other is "teraterm_vx_app_tidl_od_20241021-1433_tidl_io_peele_300_1.log".

"teraterm_vx_app_tidl_od_20241021-1429_yolov5s6_640_ti_lite_37p4_56p0.log" is the failed run log.
The failed run were using "/opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_io_1.bin" and "/opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_net.bin".
"detections_tidl_net.bin" and "detections_tidl_net.bin" are converted from "yolov5s6_640_ti_lite_37p4_56p0.onnx" through "edgeai-tidl-tools" (08_06_00_05).
----------------------------------------------------------------------
# location of conifg
-tidl_config /opt/vision_apps/test_data/psdkra/tidl_models/tidl_io_peele_300_1.bin
+tidl_config /opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_io_1.bin

# location of network
-tidl_network /opt/vision_apps/test_data/psdkra/tidl_models/tidl_net_peele_300.bin
+tidl_network /opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_net.bin
----------------------------------------------------------------------

"teraterm_vx_app_tidl_od_20241021-1433_tidl_io_peele_300_1.log" is the pass run log.
The pass run were using "/opt/vision_apps/test_data/psdkra/tidl_models/tidl_io_peele_300_1.bin" and "/opt/vision_apps/test_data/psdkra/tidl_models/tidl_net_peele_300.bin".
----------------------------------------------------------------------
# location of conifg
tidl_config /opt/vision_apps/test_data/psdkra/tidl_models/tidl_io_peele_300_1.bin

# location of network
tidl_network /opt/vision_apps/test_data/psdkra/tidl_models/tidl_net_peele_300.bin
----------------------------------------------------------------------

This test also uses tivxDLPreProcNode() instead of tivxImgPreProcNode().

Please help analyze what went wrong!

Best regard,
George Lin

Hi Nikhil,

The current situation is that I use yolov5s6_640_ti_lite_37p4_56p0.onnx and yolov5s6_640_ti_lite_metaarch.prototxt provided by TI, and use edgeai-tidl-tools (08_06_00_05) provided by TI to convert them into detections_tidl_io_1.bin and detections_tidl_net.bin.
Then I tested using ti-processor-sdk-rtos-j721e-evm-08_06_00_12/vision_apps/apps/dl_demos/app_tidl_od provided by TI. Replace the two settings tidl_config and tidl_network.
---------------------------------------------------------------------
 # location of conifg
-tidl_config /opt/vision_apps/test_data/psdkra/tidl_models/tidl_io_peele_300_1.bin
+tidl_config /opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_io_1.bin

 # location of network
-tidl_network /opt/vision_apps/test_data/psdkra/tidl_models/tidl_net_peele_300.bin
+tidl_network /opt/model_zoo/yolov5s6_640_ti_lite_37p4_56p0/artifacts/detections_tidl_net.bin
---------------------------------------------------------------------
But the execution always fails.
So I all use the AI ​​Model, conversion tools, and execution programs provided by TI.
It is unreasonable that this cannot be implemented correctly.
Can you please test within your company and use yolov5s6_640_ti_lite_37p4_56p0.onnx and yolov5s6_640_ti_lite_metaarch.prototxt to make vision_apps app_tidl_od execute correctly?

Best regard,
George Lin

Hi Nikhil,

Actually my question is very simple.
What I want to ask is whether the AI ​​Model of vision_apps app_tidl_od can be replaced by yolov5s6_640_ti_lite_37p4_56p0.onnx?

Best regard,
George Lin

Hi George, 

So,

1. It cannot be directly replaced in the vision_apps application as the pre-processing required for both the models (i.e. yolov5 and peele) are different and the current ImgPreProc does not support float data type tensor.

2. First change would be to use DLPreProc.

3. A difference between the edgeAI tidl way and the vision_apps way is that the pre-processing in EdgeAI TIDL is on A72, whereas it is on C66 in vision_apps.

4. Also, Correct me If I'm wrong, I believe you have given RGB input images to edgeAI tidl and YUV images to vision_apps as input right?

5. So there are 2 ways forward.

- One is to create your own preprocessing node on vision-apps which would convert the YUV to RGB image and also takes care of the mean and scale values as per the model

- Second is for me to recreate this issue at my end and then integrate the DL preproc node and then check why this error occurs. This would of-course take some time.

So typical suggestion from TI for any models, is to verify the model using edgeAI tidl tools (as you did before), which means the TIDL node is working fine. Then create custom nodes for pre-processing and post-processing to integrate the same into vision_apps. 

Please let me know your thoughts on this.

Regards,

Nikhil

Nikhil Dasan said:

4. Also, Correct me If I'm wrong, I believe you have given RGB input images to edgeAI tidl and YUV images to vision_apps as input right?

Hi Nikhil,

In "edgeAI tidl", I use h264 files as its "inputs".
Maybe "edgeAI tidl" extracts RGB images from h264 files.

Best regard,
George Lin

Nikhil Dasan said:

- One is to create your own preprocessing node on vision-apps which would convert the YUV to RGB image and also takes care of the mean and scale values as per the model

Hi Nikhil,

My understanding is that if my PreProcess reads BMP images, is it ok?
If it's not right, please tell me.
I think I can try this method first.
Best regard,
George Lin

Hi George,

The aim of your pre-processing node would be to cater you input source to a tensor of the datatype required by your model. The input source can be anything unless you have tools to convert the same to the required format

For example, the demo pre-proc converts a yuv input image (vx_image) to rgb output tensor (vx_tensor) (the logic implemented in the node)

So you can have a custom node, which reads a bmp image to the OpenVX object vx_image and then convert it to a vx_tensor.

Regards,

Nikhil

Hi Nikhil,

I found a deadlock in TIDL_process() (located in tidl_j721e_08_06_00_10/tidl/algo/src/tidl_alg.c).
2233        if (intAlgHandle->isPreEmptionEnable)
2234        {
2235          if (TIDL_isThisLayerOptimalForPreemption(intAlgHandle->preEmptHandle, i))
2236          {
2237            /* Wait for layer padding DMA before restoring the interrupts */
2238            TIDL_syncPadTransfer(intAlgHandle, trMem);
2239            status = TIDL_updateContextMemPtrsForPreemption(&intAlgHandle->preEmptContextInfo, intAlgHandle->privContext,
2240            intAlgHandle->preEmptHandle, intAlgHandle->perfSimOutput, intAlgHandle->sysScratchPtr.l2BasePtr,
2241            intAlgHandle->sysScratchPtr.l3BasePtr, i, intAlgHandle->createParams->net->numLayers);
2242  
2243            TIDL_unlockForPreemption(oldIntState, intAlgHandle->createParams, &lockState);
2244          }
2245        }
TIDL_isThisLayerOptimalForPreemption() will always be false, so TIDL_unlockForPreemption() will not be executed.
So dead lock will occur!!
Why does TIDL_isThisLayerOptimalForPreemption() always be false?
Because TIDL_init() has not been completely executed.
Because TIDL_init() did not execute to TIDL_initializeHandleForPreemption() on line 1686, execution was forced to stop because TIDL_process() was executed.
1665      if (algHandle->isPreEmptionEnable)
1666      {
1667        /* Allocate memory for preemption handle */
1668        algHandle->preEmptHandle = (uint8_t *)TIDL_getMemoryChunkFromSysmem(algHandle->sysMems,
1669          (uint32_t)TIDL_getHandleSizeForPreemption(algHandle->createParams->net->numLayers),
1670          128U,
1671          TIDL_SYSMEM_L3_SCRATCH, (uint32_t)IALG_PERSIST);
1672  
1673        if (algHandle->preEmptHandle == NULL)
1674        {
1675          status = IALG_EFAIL;
1676          tidl_printf(0, "Not able to allocate preEmption Handle\n");
1677        }
1678  
1679        /* Initialize preemption handle */
1680        if(status == IALG_EOK)
1681        {
1682          algHandle->preemptPerfHW = 1;
1683          /* Avoided initializing preEmptHandle in L3, instead initializing it in the contextMemPtr (DDR)
1684             and copy it to preEmptHandle in L3 in the activate function
1685          */
1686          TIDL_initializeHandleForPreemption(algHandle->preEmptContextInfo.contextMemPtr, algHandle->perfSimOutput,
1687          algHandle->createParams->net->numLayers, algHandle->createParams->maxPreEmptDelay,
1688          algHandle->sysScratchPtr.l2BasePtr, algHandle->ddrBytesPerCPUCycle,
1689          algHandle->perfSimOutput->simConfig.freqMHz);
1690          /* To copy this contextMemPtr to preEmptHandle in the activate function */
1691          algHandle->preEmptContextInfo.contextBufInfo[0].src = (uint8_t *) algHandle->preEmptHandle;
1692          algHandle->preEmptContextInfo.contextBufInfo[0].dst = (uint8_t *) algHandle->preEmptContextInfo.contextMemPtr;
1693          algHandle->preEmptContextInfo.contextBufInfo[0].size = TIDL_getHandleSizeForPreemption(algHandle->createParams->net->numLayers);
1694          algHandle->preEmptContextInfo.earlyPreEmpt = 1;
1695          algHandle->preEmptContextInfo.numContextBufs = 1;
1696        }
1697        tidl_printf(1, "TIDL_initializeHandleForPreemption is completed \n");
1698      }
1699    }

I would like to ask when TIDL_process() will be called and executed??
How to make TIDL_process() be called after TIDL_init() has finished executing?

Best regard,
George Lin

Hi Nikhil,

I think you misunderstood me.
I'm currently analyzing ti-processor-sdk-rtos-j721e-evm-08_06_00_12/tidl_j721e_08_06_00_10/ti_dl/algo/src .
I think this is still within tidl scope.
My question is why TIDL_process() (located in tidl_j721e_08_06_00_10/tidl/algo/src/tidl_alg.c) has a deadlock.

Line:2243 cannot always be executed.
2233        if (intAlgHandle->isPreEmptionEnable)
2234        {
2235          if (TIDL_isThisLayerOptimalForPreemption(intAlgHandle->preEmptHandle, i))
2236          {
2237            /* Wait for layer padding DMA before restoring the interrupts */
2238            TIDL_syncPadTransfer(intAlgHandle, trMem);
2239            status = TIDL_updateContextMemPtrsForPreemption(&intAlgHandle->preEmptContextInfo, intAlgHandle->privContext,
2240            intAlgHandle->preEmptHandle, intAlgHandle->perfSimOutput, intAlgHandle->sysScratchPtr.l2BasePtr,
2241            intAlgHandle->sysScratchPtr.l3BasePtr, i, intAlgHandle->createParams->net->numLayers);
2242  
2243            TIDL_unlockForPreemption(oldIntState, intAlgHandle->createParams, &lockState);
2244          }
2245        }
How to resolve the deadlock?

Thanks,

Best regard,
George Lin

Hi Nikhil,

My job is very simple, which is to make the vision app use yolov5's ai model.
But I have been working for several months and still can't succeed.
Could you please tell me which yolov5 ai model can be used by vision app?

Best regard,
George Lin

Hi George;

Thank you for your question. Since this thread is becoming too long, and contains various info. In order to address your question effectively, we recommend you to open a new thread. So we can have your questions answered with the right experts quickly.

If you agree, please close this thread. And open a new one, you can copy and paste the related info here.

Best regards

Wen Li 

Hi Wen Li,

Okay, I've created the new thread.
(e2e.ti.com/.../j721excpxevm-vision_apps-uses-the-yolov5-ai-model-for-inference)

Please help assign an owner to the new thread.
I don't want to complain. But this problem has been unresolved for too long.
I just want a simple example now. Let me prove that vision_apps can use yolov5 AI model for inference.

Best regard,
George Lin