• State
  • Locked Locked
  • Replies 27 replies
  • Subscribers 101 subscribers
  • Views 1106 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VH-Q1: Issue with the artifacts generated by TIDLModelImport in J784S4

srikar varaganti
Intellectual 280 points
Part Number: TDA4VH-Q1

Hello,

I have recently generated network and config .bin files for our OD application using the ONNX and Protxt files for the models in TDA high (J784s4). Please note that these ONNX and Protxt files when used to generate artifacts in TDA mid or previous versions, they was no issue with the .bin files. I followed the similar procedure in TDA high to obtain the artifacts here, but I am facing an issue during the run graph stage while the app is running.

The following is the step where the model is getting stuck and is not able to proceed:

As compared to properly working model: we used default Peele model artifacts and ran our OD application and found that there were a few other log messages as below:

What have I done so far:

1. Followed the procedure mentioned here: TI Deep Learning Product User Guide: Getting Started with TIDL-RT

2. Ran the TIDLModelImport as mentioned and used the generated artifacts. 

Kindly let me know in what direction I need to be working on in order solve this issue. Thanks

  • 0
    Intellectual 280 points

    Below is hte log message I am receiving with newly generated artifacts:

  • 0 in reply to srikar varaganti
    Intellectual 280 points

    Fullscreen model_v1_2iterations.txt Download 
    Iteration Number: 0 of total 2 iterations.
Running frame 0
Pipeup
         Reading input image file from path: /opt/vision_apps/test_data/od_data/0000000000.bmp at enque count: 0 


 =========================
 TIDL Demo - Object Detection
 =========================

 p: Print performance statistics

 x: Exit

 Enter Choice:   time taken for reading .bmp file : in 102 msec!
[C7x_2 ]  78509.975372 s:  VX_ZONE_INFO:[ownTargetKernelInstanceAlloc:107] attempting to call kernel org.khronos.openvx.color_convert on this CPU
[C7x_2 ]  78509.975418 s:  VX_ZONE_INFO:[ownTargetKernelCreate:316] Executing create callback for kernel []
[C7x_2 ]  78509.975446 s:  VX_ZONE_INFO:[ownTargetKernelCreate:322] Done executing create callback for kernel []
[C7x_2 ]  78509.975786 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=161, pipe=0) acquiring parameters on target 00000000
[C7x_2 ]  78509.975817 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=161, pipe=0) executing on target 00000000
[C7x_2 ]  78509.975848 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel []
         time taken for converting rgb to yuv file : in 25 msec!
Reading Input doneenqueing parameter initializedPrameter enqueuedPipeup finished 
App Process Graph Done!
app_tidl_od: Frame ID 0 of 30 Total Time : 128... Done.
Running frame 10
Pipeup
         Reading input image file from path: /opt/vision_apps/test_data/od_data/0000000010.bmp at enque count: 1 
[MCU2_0]  78510.000448 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=40, pipe=0) acquiring parameters on target 00000203
[MCU2_0]  78510.000537 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=40, pipe=0, data_ref_q=109, queue=119, ref=6)
[MCU2_0]  78510.000620 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=40, pipe=0, data_ref_q=121, queue=126, ref=12)
[MCU2_0]  78510.000698 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=40, pipe=0, data_ref_q=127, queue=132, ref=15)
[MCU2_0]  78510.000765 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=40, pipe=0) executing on target 00000203
[MCU2_0]  78510.000837 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[C7x_2 ]  78509.999969 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel []
[C7x_2 ]  78510.000008 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=161, pipe=0) executing on target 00000000 ... DONE !!!
[C7x_2 ]  78510.000188 s:  VX_ZONE_INFO:[ownTargetKernelDelete:348] Executing delete callback for kernel []
[C7x_2 ]  78510.000214 s:  VX_ZONE_INFO:[ownTargetKernelDelete:354] Done executing delete callback for kernel []
[MCU2_0]  78510.006502 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[MCU2_0]  78510.006583 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=40, pipe=0) executing on target 00000203 ... DONE !!!
         time taken for reading .bmp file : in 97 msec!
[MCU2_0]  78510.006675 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:463] Parameter released (node=40, pipe=0, data_ref_q=109, queue=120, ref=6)
[MCU2_0]  78510.006743 s:  VX_ZONE_INFO:[ownTargetObjDescSendRefConsumed:116] Sending reference consumed event (data_ref_q=109) !!!
[MCU2_0]  78510.006842 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=40, pipe=0, data_ref_q=121, ref=12, users=1)
[MCU2_0]  78510.006978 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=40, pipe=0, data_ref_q=127, ref=15, users=1)
[C7x_1 ]  78510.008050 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=42, pipe=0) acquiring parameters on target 00000100
[C7x_1 ]  78510.008095 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:205] Parameter ALREADY acquired (node=42, pipe=0, data_ref_q=133, ref=30)
[C7x_1 ]  78510.008131 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=42, pipe=0, data_ref_q=139, queue=144, ref=27)
[C7x_1 ]  78510.008161 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=42, pipe=0) executing on target 00000100
[C7x_1 ]  78510.008200 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel [com.ti.tidl]
[C7x_2 ]  78510.007081 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=41, pipe=0) acquiring parameters on target 00000000
[C7x_2 ]  78510.007118 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:205] Parameter ALREADY acquired (node=41, pipe=0, data_ref_q=121, ref=12)
[C7x_2 ]  78510.007155 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=41, pipe=0, data_ref_q=133, queue=138, ref=30)
[C7x_2 ]  78510.007185 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=41, pipe=0) executing on target 00000000
[C7x_2 ]  78510.007219 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel [com.ti.img_proc.img.preprocess]
[C7x_2 ]  78510.007857 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel [com.ti.img_proc.img.preprocess]
[C7x_2 ]  78510.007893 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=41, pipe=0) executing on target 00000000 ... DONE !!!
[C7x_2 ]  78510.007932 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:463] Parameter released (node=41, pipe=0, data_ref_q=121, queue=126, ref=12)
[C7x_2 ]  78510.007968 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=41, pipe=0, data_ref_q=133, ref=30, users=1)
[C7x_2 ]  78510.346630 s:  VX_ZONE_INFO:[ownTargetKernelInstanceAlloc:107] attempting to call kernel org.khronos.openvx.color_convert on this CPU
[C7x_2 ]  78510.346676 s:  VX_ZONE_INFO:[ownTargetKernelCreate:316] Executing create callback for kernel []
[C7x_2 ]  78510.346704 s:  VX_ZONE_INFO:[ownTargetKernelCreate:322] Done executing create callback for kernel []
[C7x_2 ]  78510.347003 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=165, pipe=0) acquiring parameters on target 00000000
[C7x_2 ]  78510.347033 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=165, pipe=0) executing on target 00000000
[C7x_2 ]  78510.347063 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel []
         time taken for converting rgb to yuv file : in 25 msec!
Reading Input doneenqueing parameter initializedPrameter enqueuedPipeup finished 
Dequeuing the inputdequeue done  Reading input image file from path: /opt/vision_apps/test_data/od_data/0000000010.bmp at enque count: 0 
[MCU2_0]  78510.371729 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=46, pipe=1) acquiring parameters on target 00000203
[MCU2_0]  78510.371816 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=46, pipe=1, data_ref_q=111, queue=119, ref=8)
[MCU2_0]  78510.371900 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=46, pipe=1, data_ref_q=122, queue=126, ref=67)
         time taken for reading .bmp file : in 23 msec!
[MCU2_0]  78510.372024 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=46, pipe=1, data_ref_q=128, queue=132, ref=69)
[MCU2_0]  78510.372095 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=46, pipe=1) executing on target 00000203
[MCU2_0]  78510.372168 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[MCU2_0]  78510.377708 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[MCU2_0]  78510.377783 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=46, pipe=1) executing on target 00000203 ... DONE !!!
[MCU2_0]  78510.377871 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:463] Parameter released (node=46, pipe=1, data_ref_q=111, queue=120, ref=8)
[MCU2_0]  78510.377984 s:  VX_ZONE_INFO:[ownTargetObjDescSendRefConsumed:116] Sending reference consumed event (data_ref_q=111) !!!
[MCU2_0]  78510.378082 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=46, pipe=1, data_ref_q=122, ref=67, users=1)
[MCU2_0]  78510.378167 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=46, pipe=1, data_ref_q=128, ref=69, users=1)
[C7x_2 ]  78510.371311 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel []
[C7x_2 ]  78510.371345 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=165, pipe=0) executing on target 00000000 ... DONE !!!
[C7x_2 ]  78510.371501 s:  VX_ZONE_INFO:[ownTargetKernelDelete:348] Executing delete callback for kernel []
[C7x_2 ]  78510.371527 s:  VX_ZONE_INFO:[ownTargetKernelDelete:354] Done executing delete callback for kernel []
[C7x_2 ]  78510.378271 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=50, pipe=1) acquiring parameters on target 00000000
[C7x_2 ]  78510.378307 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:205] Parameter ALREADY acquired (node=50, pipe=1, data_ref_q=122, ref=67)
[C7x_2 ]  78510.378343 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=50, pipe=1, data_ref_q=134, queue=138, ref=79)
[C7x_2 ]  78510.378372 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=50, pipe=1) executing on target 00000000
[C7x_2 ]  78510.378407 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel [com.ti.img_proc.img.preprocess]
[C7x_2 ]  78510.379093 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel [com.ti.img_proc.img.preprocess]
[C7x_2 ]  78510.379130 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=50, pipe=1) executing on target 00000000 ... DONE !!!
[C7x_2 ]  78510.379171 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:463] Parameter released (node=50, pipe=1, data_ref_q=122, queue=126, ref=67)
[C7x_2 ]  78510.379207 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=50, pipe=1, data_ref_q=134, ref=79, users=1)
[C7x_2 ]  78510.707515 s:  VX_ZONE_INFO:[ownTargetKernelInstanceAlloc:107] attempting to call kernel org.khronos.openvx.color_convert on this CPU
[C7x_2 ]  78510.707560 s:  VX_ZONE_INFO:[ownTargetKernelCreate:316] Executing create callback for kernel []
[C7x_2 ]  78510.707588 s:  VX_ZONE_INFO:[ownTargetKernelCreate:322] Done executing create callback for kernel []
[C7x_2 ]  78510.707864 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=169, pipe=0) acquiring parameters on target 00000000
[C7x_2 ]  78510.707894 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=169, pipe=0) executing on target 00000000
         time taken for converting rgb to yuv file : in 25 msec!
         Reading input image file from path at enque count: 0 
[C7x_2 ]  78510.707924 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel []
File name: /opt/vision_apps/test_data/od_data/0000000010.bmp 
        App Reading Input Done 
inside the if statement 
[C7x_2 ]  78510.732013 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel []
Out of if statement 
App Process Graph Done!
[C7x_2 ]  78510.732049 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=169, pipe=0) executing on target 00000000 ... DONE !!!
app_tidl_od: Frame ID 10 of 30 Total Time : 1045... Done.
Running frame 20
[C7x_2 ]  78510.732205 s:  VX_ZONE_INFO:[ownTargetKernelDelete:348] Executing delete callback for kernel []
Dequeuing the inputdequeue done  Reading input image file from path: /opt/vision_apps/test_data/od_data/0000000020.bmp at enque count: 1 
[C7x_2 ]  78510.732232 s:  VX_ZONE_INFO:[ownTargetKernelDelete:354] Done executing delete callback for kernel []
[MCU2_0]  78511.045870 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=47, pipe=2) acquiring parameters on target 00000203
[MCU2_0]  78511.045995 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=47, pipe=2, data_ref_q=113, queue=119, ref=6)
[MCU2_0]  78511.046082 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=47, pipe=2, data_ref_q=123, queue=126, ref=12)
[MCU2_0]  78511.046165 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=47, pipe=2, data_ref_q=129, queue=132, ref=71)
[MCU2_0]  78511.046235 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=47, pipe=2) executing on target 00000203
[MCU2_0]  78511.046309 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[MCU2_0]  78511.051841 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[MCU2_0]  78511.051914 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=47, pipe=2) executing on target 00000203 ... DONE !!!
[MCU2_0]  78511.052045 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:463] Parameter released (node=47, pipe=2, data_ref_q=113, queue=120, ref=6)
[MCU2_0]  78511.052109 s:  VX_ZONE_INFO:[ownTargetObjDescSendRefConsumed:116] Sending reference consumed event (data_ref_q=113) !!!
[MCU2_0]  78511.052203 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=47, pipe=2, data_ref_q=123, ref=12, users=1)
[MCU2_0]  78511.052281 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=47, pipe=2, data_ref_q=129, ref=71, users=1)
[C7x_2 ]  78511.052375 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=51, pipe=2) acquiring parameters on target 00000000
[C7x_2 ]  78511.052411 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:205] Parameter ALREADY acquired (node=51, pipe=2, data_ref_q=123, ref=12)
[C7x_2 ]  78511.052446 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:169] Parameter acquire failed ... BLOCKING (node=51, pipe=2, data_ref_q=135, queue=138)
[C7x_2 ]  78511.052477 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireAllParameters:549] Parameter acquire ... ABORTING (node=51, pipe=2)
[C7x_2 ]  78511.052505 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:717] Node (node=51, pipe=2) ... BLOCKED for resources on target 00000000
         time taken for reading .bmp file : in 101 msec!
[C7x_2 ]  78511.147732 s:  VX_ZONE_INFO:[ownTargetKernelInstanceAlloc:107] attempting to call kernel org.khronos.openvx.color_convert on this CPU
[C7x_2 ]  78511.147777 s:  VX_ZONE_INFO:[ownTargetKernelCreate:316] Executing create callback for kernel []
[C7x_2 ]  78511.147804 s:  VX_ZONE_INFO:[ownTargetKernelCreate:322] Done executing create callback for kernel []
[C7x_2 ]  78511.148131 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=173, pipe=0) acquiring parameters on target 00000000
[C7x_2 ]  78511.148162 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=173, pipe=0) executing on target 00000000
[C7x_2 ]  78511.148192 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel []
         time taken for converting rgb to yuv file : in 25 msec!
         Reading input image file from path at enque count: 1 
File name: /opt/vision_apps/test_data/od_data/0000000020.bmp 
        App Reading Input Done 
inside the if statement 
Out of if statement 
App Process Graph Done!
app_tidl_od: Frame ID 20 of 30 Total Time : 127... Done.
app_tidl_od: Iteration 0 of 2 ... Done.
Iteration Number: 1 of total 2 iterations.
Running frame 0
Dequeuing the inputdequeue done  Reading input image file from path: /opt/vision_apps/test_data/od_data/0000000000.bmp at enque count: 0 
[MCU2_0]  78511.172915 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=48, pipe=3) acquiring parameters on target 00000203
[MCU2_0]  78511.173032 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=48, pipe=3, data_ref_q=115, queue=119, ref=8)
[MCU2_0]  78511.173113 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=48, pipe=3, data_ref_q=124, queue=126, ref=67)
         time taken for reading .bmp file : in 23 msec!
[MCU2_0]  78511.173191 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=48, pipe=3, data_ref_q=130, queue=132, ref=73)
[MCU2_0]  78511.173259 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=48, pipe=3) executing on target 00000203
[MCU2_0]  78511.173330 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[MCU2_0]  78511.178852 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel [com.ti.hwa.vpac_msc_multi_scale]
[MCU2_0]  78511.178970 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=48, pipe=3) executing on target 00000203 ... DONE !!!
[MCU2_0]  78511.179060 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:463] Parameter released (node=48, pipe=3, data_ref_q=115, queue=120, ref=8)
[MCU2_0]  78511.179124 s:  VX_ZONE_INFO:[ownTargetObjDescSendRefConsumed:116] Sending reference consumed event (data_ref_q=115) !!!
[MCU2_0]  78511.179217 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=48, pipe=3, data_ref_q=124, ref=67, users=1)
[MCU2_0]  78511.179297 s:  VX_ZONE_INFO:[ownTargetNodeDescReleaseParameter:481] Parameter NOT released (node=48, pipe=3, data_ref_q=130, ref=73, users=1)
[C7x_2 ]  78511.172484 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel []
[C7x_2 ]  78511.172519 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=173, pipe=0) executing on target 00000000 ... DONE !!!
[C7x_2 ]  78511.172681 s:  VX_ZONE_INFO:[ownTargetKernelDelete:348] Executing delete callback for kernel []
[C7x_2 ]  78511.172707 s:  VX_ZONE_INFO:[ownTargetKernelDelete:354] Done executing delete callback for kernel []
[C7x_2 ]  78511.179397 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:726] Node (node=52, pipe=3) ... BLOCKED for previous pipe instance node (node=51) to complete !!!
[C7x_2 ]  78511.506998 s:  VX_ZONE_INFO:[ownTargetKernelInstanceAlloc:107] attempting to call kernel org.khronos.openvx.color_convert on this CPU
[C7x_2 ]  78511.507043 s:  VX_ZONE_INFO:[ownTargetKernelCreate:316] Executing create callback for kernel []
[C7x_2 ]  78511.507070 s:  VX_ZONE_INFO:[ownTargetKernelCreate:322] Done executing create callback for kernel []
[C7x_2 ]  78511.507368 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=177, pipe=0) acquiring parameters on target 00000000
[C7x_2 ]  78511.507398 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=177, pipe=0) executing on target 00000000
[C7x_2 ]  78511.507428 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel []
         time taken for converting rgb to yuv file : in 25 msec!
         Reading input image file from path at enque count: 0 
File name: /opt/vision_apps/test_data/od_data/0000000000.bmp 
        App Reading Input Done 
inside the if statement 
Out of if statement 
App Process Graph Done!
app_tidl_od: Frame ID 0 of 30 Total Time : 359... Done.
Running frame 10
Dequeuing the inputdequeue done  Reading input image file from path: /opt/vision_apps/test_data/od_data/0000000010.bmp at enque count: 1 
[MCU2_0]  78511.531966 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=49, pipe=4) acquiring parameters on target 00000203
[MCU2_0]  78511.532053 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:195] Parameter acquired (node=49, pipe=4, data_ref_q=117, queue=119, ref=6)
[MCU2_0]  78511.532132 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireParameter:169] Parameter acquire failed ... BLOCKING (node=49, pipe=4, data_ref_q=125, queue=126)
[MCU2_0]  78511.532197 s:  VX_ZONE_INFO:[ownTargetNodeDescAcquireAllParameters:549] Parameter acquire ... ABORTING (node=49, pipe=4)
[MCU2_0]  78511.532260 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:717] Node (node=49, pipe=4) ... BLOCKED for resources on target 00000203
[C7x_2 ]  78511.531507 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel []
         time taken for reading .bmp file : in 22 msec!
[C7x_2 ]  78511.531543 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=177, pipe=0) executing on target 00000000 ... DONE !!!
[C7x_2 ]  78511.531704 s:  VX_ZONE_INFO:[ownTargetKernelDelete:348] Executing delete callback for kernel []
[C7x_2 ]  78511.531729 s:  VX_ZONE_INFO:[ownTargetKernelDelete:354] Done executing delete callback for kernel []
[C7x_2 ]  78511.867870 s:  VX_ZONE_INFO:[ownTargetKernelInstanceAlloc:107] attempting to call kernel org.khronos.openvx.color_convert on this CPU
[C7x_2 ]  78511.867914 s:  VX_ZONE_INFO:[ownTargetKernelCreate:316] Executing create callback for kernel []
[C7x_2 ]  78511.867942 s:  VX_ZONE_INFO:[ownTargetKernelCreate:322] Done executing create callback for kernel []
[C7x_2 ]  78511.868229 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:601] Node (node=181, pipe=0) acquiring parameters on target 00000000
[C7x_2 ]  78511.868259 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:660] Node (node=181, pipe=0) executing on target 00000000
[C7x_2 ]  78511.868290 s:  VX_ZONE_INFO:[ownTargetKernelExecute:380] Executing process callback for kernel []
         time taken for converting rgb to yuv file : in 25 msec!
         Reading input image file from path at enque count: 1 
File name: /opt/vision_apps/test_data/od_data/0000000010.bmp 
        App Reading Input Done 
inside the if statement 
Out of if statement 
App Process Graph Done!
app_tidl_od: Frame ID 10 of 30 Total Time : 361... Done.
Running frame 20
[C7x_2 ]  78511.892616 s:  VX_ZONE_INFO:[ownTargetKernelExecute:388] Done executing process callback for kernel []
[C7x_2 ]  78511.892651 s:  VX_ZONE_INFO:[ownTargetNodeDescNodeExecute:683] Node (node=181, pipe=0) executing on target 00000000 ... DONE !!!
[C7x_2 ]  78511.892808 s:  VX_ZONE_INFO:[ownTargetKernelDelete:348] Executing delete callback for kernel []
[C7x_2 ]  78511.892834 s:  VX_ZONE_INFO:[ownTargetKernelDelete:354] Done executing delete callback for kernel []
    peele_2iterations.txt

    I ran the app with two model artifacts (peele which are default artifacts provided in the sdk, model_v1 which is our customized model). Please note that model_v1 artifacts when generated in tda mid sdk(j721e) is not giving this issue.

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Hi,

    Can you help me with which version of J784S4 SDK you are using?

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    it is sdk 08_06_01_03 in linux release mode.

  • 0 in reply to srikar varaganti
    TI__Mastermind 22145 points

    Hi Srikar,

    Can you please share the model/config and steps to re-producde this issues.

    BTW, is this model working fine in PC Hot emulation mode?

  • 0 in reply to kumar.desappan
    Intellectual 280 points

    Hello,

    Below is the procedure i followed to generate the artifacts:

    Import and Inference using TIDL-RT (1).docx

  • 0 in reply to srikar varaganti
    Intellectual 280 points

    1. below is the config file that is passed as the argument to the .out file:

    Fullscreen 2870.tidl_import_model_v1.txt Download 
    modelType          = 2 
numParamBits       = 8
numFeatureBits     = 8
quantizationStyle  = 3
#quantizationStyle  = 2

inputNetFile       = "../../test/testvecs/models/public/onnx/model_v1/model.onnx"
outputNetFile      = "../../test/testvecs/config/tidl_models/onnx/model_v1/tidl_net_model_v1.bin"
outputParamsFile   = "../../test/testvecs/config/tidl_models/onnx/model_v1/tidl_io_model_v1_"
inDataNorm  = 1
inMean = 123.675 116.28 103.53
inScale = 0.017429 0.017507 0.017125
inDataFormat = 0
inWidth  = 320
inHeight = 320 
inNumChannels = 3
numFrames = 1
inData  =   "../../test/testvecs/config/detection_list.txt"
perfSimConfig = ../../test/testvecs/config/import/device_config.cfg
inElementType = 0
#outDataNamesList = "convolution_output,convolution_output1,convolution_output2"
metaArchType = 4
metaLayersNamesList =  "../../test/testvecs/models/public/onnx/model_v1/model.prototxt"

postProcType = 2

    2. I generated the artifacts in PC emulation mode and checked the by passing the .bin file into sample OD app provided with the SDK and observed that the model is running however the output images are not as expected and is attached below. Please note that we tried generating artifacts from two different models and observed similar distracted output in both the cases while ran in PC emulation mode. whereas, these artifacts when used in Lunix release mode, the app is not executing.

    3. I tried to upload ONNX and Prototxt files being used but I am not able to upload here in e2e.

  • 0 in reply to srikar varaganti
    TI__Mastermind 36585 points

    Dear TI Team, I setup and external TI Drive for Srikar to upload the files to. This has taken place so please proceed as required.

    Steve D 

    https://tidrive.ext.ti.com/u/_Yn1uQGJ9ZHvifW_/bb17d220-1604-4db2-b1a4-d5a847076702?l

     rj7.YuX5

  • 0 in reply to ducky35061
    TI__Genius 9065 points

    Hi Srikar,

    Apologies for the delay in response. I tried running the model you have shared above by generating artifacts with attached config file and running inference using TIDL host emulation executable on SDK 9.0 release and I get the correct output there.

    Can you please confirm this observation on your end by running inference using the TIDL inference executable (PC_dsp_test_dl_algo.out) in host emulation mode?

    Below are the infer config parameters I used:

    inFileFormat = 2
    numFrames = 1
    netBinFile = "testvecs/config/tidl_models/onnx/model_v1/tidl_net_model_v1.bin"
    ioConfigFile = "testvecs/config/tidl_models/onnx/model_v1/tidl_io_model_v1_1.bin"
    inData = testvecs/config/detection_list.txt
    outData = testvecs/output/tidl_model_sre.bin
    postProcType = 2

    That would help rule out setup issues if any and also help me narrow down issue to demo code rather than core TIDL inference.

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    Hey Anand, I tried generating .bin files using the tidl importer in the new sdk 9.00.00.02 version in linux release mode. But i've encountered an error while running the .out tidl_model_import.out file with the same config file.

    I am attaching the error log files here for your reference. Also, I've requested Daviel for PC emulation installer run file for this version of SDK and will update you based on the result obtained there. Meanwhile, can you look into the issue with generation of the artifacts itself in the new sdk. 

    Fullscreen importer_sdk_9.0_error_message.txt Download 
    sre@sre-VirtualBox:~/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/utils/tidlModelImport$ ./out/tidl_model_import.out /home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/onnx/model_v1/tidl_import_model_v1.txt 
TIDL Meta PipeLine (Proto) File  : /home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/models/public/onnx/model_v1/model.prototxt  
yolo_v3
yolo_v3
ONNX Model (Proto) File  : /home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/models/public/onnx/model_v1/model.onnx  
TIDL Network File      : /home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_net_model_v1.bin  
TIDL IO Info File      : /home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_io_model_v1_  
Current ONNX OpSet Version   : 11  
Empty prototxt path, running calibration

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_import_model_v1.txt.qunat_stats_config.txt 
Illegal instruction (core dumped)

 
 
 *****************   Calibration iteration number 0 started ************************ 
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_import_model_v1.txt.qunat_stats_config.txt 
Illegal instruction (core dumped)

 
 
 *****************   Calibration iteration number 0 completed ************************ 
 
 
 

------------------ Network Compiler Traces -----------------------------
NC running for device: 1
Running with OTF buffer optimizations
successful Memory allocation
INFORMATION: [TIDL_ResizeLayer] Resize_149 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_168 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
TIDL ALLOWLISTING LAYER CHECK: TIDL_E_QUANT_STATS_NOT_AVAILABLE] tidl_quant_stats_tool.out fails to collect dynamic range. Please look into quant stats log. This model will get fault on target.
****************************************************
**          2 WARNINGS          1 ERRORS          **
****************************************************

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Hi Srikar,

    We are observing this issue recently if the PC used for build and inference have different OS characteristics. Can you please build the libraries on your machine by following below instructions and retry running the inference?

    • Download and untar PSDK RTOS
    • Download targetfs and bootfs tar files to the above folder
    • export BUILD_EMULATION_MODE=yes
    • export PROFILE=release
    • sdk_builder/scripts/setup_psdk_rtos.sh --install_tidl_deps
    • make tidl_pc_tools

    This should rebuild the TIDL tools required for PC inference. If you do this build, I expect the inference to work fine.

    Can you give a try and let me know?

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    Hello, 

    Are the targetfs and bootfs part of PC emulation installer.run file?

    I am asking this because I cannot find these tar files within linux .bin 

    Thank you.

  • 0 in reply to srikar varaganti
    TI__Expert 3265 points

    Srikar, 

    The targetfs and bootfs for 9.0 can be found here: https://www.ti.com/tool/download/PROCESSOR-SDK-LINUX-J784S4 

    where the targetfs is tisdk-adas-image-j784s4-evm.tar.xz and Bootfs is boot-adas-j784s4-evm.tar.gz  

  • 0 in reply to Daviel Almonte
    Intellectual 280 points

    We tried to finish run the setup and make tidl tools command as suggested by taking the respective tar files from the download link shown in sdk but unfortunately, we are not able to complete the steps successfully. Do you think this might be because we do not have PC Emulation tools installed in the sdk currently. I have requested for the PC emulation installer.run file and I am awaiting for TI's approval on it. Meanwhile, I am attaching here the logs for these two runs. 

    Please advise.

     

    Fullscreen log_setup_tidl_dependencies.txt Download 
    sre@sre-VirtualBox:~/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02$ ./sdk_builder/scripts/setup_psdk_rtos.sh --install_tidl_deps
[sudo] password for sre: 
Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu jammy InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu jammy-updates InRelease
Get:4 http://us.archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB]
Fetched 109 kB in 1s (128 kB/s)    
Reading package lists... Done
[dof] Creating/Updating system link to libDOF.so ...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'libgraphviz-dev' instead of 'graphviz-dev'
bison is already the newest version (2:3.8.2+dfsg-1build1).
build-essential is already the newest version (12.9ubuntu3).
flex is already the newest version (2.6.4-8build2).
libpng-dev is already the newest version (1.6.37-3build5).
python3-pyelftools is already the newest version (0.27-1).
graphviz is already the newest version (2.42.2-6).
libgraphviz-dev is already the newest version (2.42.2-6).
mono-runtime is already the newest version (6.8.0.105+dfsg-3.2).
ninja-build is already the newest version (1.10.1-1).
pkgconf is already the newest version (1.8.0-1).
swig is already the newest version (4.0.2-1ubuntu1).
cmake is already the newest version (3.22.1-1ubuntu1.22.04.1).
curl is already the newest version (7.81.0-1ubuntu1.13).
git is already the newest version (1:2.34.1-1ubuntu1.10).
libgles2-mesa-dev is already the newest version (23.0.4-0ubuntu1~22.04.1).
libprotobuf-dev is already the newest version (3.12.4-1ubuntu7.22.04.1).
libprotoc-dev is already the newest version (3.12.4-1ubuntu7.22.04.1).
python3-distutils is already the newest version (3.10.8-1~22.04).
python3-setuptools is already the newest version (59.6.0-1.2ubuntu0.22.04.1).
u-boot-tools is already the newest version (2022.01+dfsg-2ubuntu2.4).
unzip is already the newest version (6.0-26ubuntu3.1).
libc6:i386 is already the newest version (2.35-0ubuntu3.1).
libtinfo5 is already the newest version (6.3-2ubuntu0.1).
protobuf-compiler is already the newest version (3.12.4-1ubuntu7.22.04.1).
python3-pip is already the newest version (22.0.2+dfsg-1ubuntu0.3).
0 upgraded, 0 newly installed, 0 to remove and 24 not upgraded.
[psdk linux tisdk-adas-image-j784s4-evm.tar.xz] Checking ...
[psdk linux tisdk-adas-image-j784s4-evm.tar.xz] Done 
[psdk linux boot-adas-j784s4-evm.tar.gz] Checking ... 
[psdk linux boot-adas-j784s4-evm.tar.gz] Done 
export BIOS_VERSION=
export XDC_VERSION=
export GCC_ARCH64_VERSION=9.2-2019.12
export CGT_C6X_VERSION=8.3.7
export CGT_C7X_VERSION=3.1.0.LTS
export CGT_ARM_VERSION=
export CGT_ARMLLVM_VERSION=2.1.3.LTS
export NDK_VERSION=
export NS_VERSION=
export SYSCONFIG_VERSION=
export SYSCONFIG_BUILD=
[ti-cgt-armllvm_2.1.3.LTS] Checking ...
[ti-cgt-armllvm_2.1.3.LTS] Done
[ti-cgt-c7000_3.1.0.LTS] Checking ...
[ti-cgt-c7000_3.1.0.LTS] Done
[gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf] Checking ...
[gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf] Done
[arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu] Checking ...
[arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu] Done
[gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf] Checking ...
[gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf] Done
[protobuf-3.11.3] Checking ...
[protobuf-3.11.3] Done
[protobuf-3.11.3] Libs Checking ...
[Bulding protobuf-3.11.3 libs for TIDL PC emualtion mode]
[protobuf-3.11.3 build Done]
[protobuf-3.11.3] Libs Done
[opencv-4.1.0] Checking ...
--2023-08-28 15:24:23--  https://github.com/opencv/opencv/archive/4.1.0.zip
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/opencv/opencv/zip/refs/tags/4.1.0 [following]
--2023-08-28 15:24:23--  https://codeload.github.com/opencv/opencv/zip/refs/tags/4.1.0
Resolving codeload.github.com (codeload.github.com)... 140.82.112.9
Connecting to codeload.github.com (codeload.github.com)|140.82.112.9|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-08-28 15:24:24 ERROR 403: Forbidden.

unzip:  cannot find or open 4.1.0.zip, 4.1.0.zip.zip or 4.1.0.zip.ZIP.
rm: cannot remove '4.1.0.zip': No such file or directory
[Bulding opencv-4.1.0 libs for TIDL PC emualtion mode]
./sdk_builder/scripts/setup_psdk_rtos.sh: line 359: cd: opencv-4.1.0/cmake: No such file or directory
/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/protobuf-3.11.3
[opencv-4.1.0 build Done]
[opencv-4.1.0] Done
[flatbuffers-1.12.0] Checking ...
--2023-08-28 15:24:24--  https://github.com/google/flatbuffers/archive/v1.12.0.zip
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/google/flatbuffers/zip/refs/tags/v1.12.0 [following]
--2023-08-28 15:24:24--  https://codeload.github.com/google/flatbuffers/zip/refs/tags/v1.12.0
Resolving codeload.github.com (codeload.github.com)... 140.82.112.9
Connecting to codeload.github.com (codeload.github.com)|140.82.112.9|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-08-28 15:24:24 ERROR 403: Forbidden.

unzip:  cannot find or open v1.12.0.zip, v1.12.0.zip.zip or v1.12.0.zip.ZIP.
rm: cannot remove 'v1.12.0.zip': No such file or directory
[Bulding flatbuffers-1.12.0 libs for TIDL PC emualtion mode]
./sdk_builder/scripts/setup_psdk_rtos.sh: line 375: cd: flatbuffers-1.12.0: No such file or directory
/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02
[flatbuffers-1.12.0 build Done]
[flatbuffers-1.12.0] Done
[tvm] Checking ...
Cloning into 'tvm'...
remote: Enumerating objects: 5918, done.
remote: Counting objects: 100% (5918/5918), done.
remote: Compressing objects: 100% (5295/5295), done.
remote: Total 5918 (delta 1357), reused 1558 (delta 393), pack-reused 0
Receiving objects: 100% (5918/5918), 12.85 MiB | 10.10 MiB/s, done.
Resolving deltas: 100% (1357/1357), done.
Note: switching to '18a6fb59259201edb05dbbc40412cc0c396004b2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Submodule '3rdparty/OpenCL-Headers' (https://github.com/KhronosGroup/OpenCL-Headers.git) registered for path '3rdparty/OpenCL-Headers'
Submodule '3rdparty/cnpy' (https://github.com/rogersce/cnpy.git) registered for path '3rdparty/cnpy'
Submodule '3rdparty/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path '3rdparty/cutlass'
Submodule 'dlpack' (https://github.com/dmlc/dlpack.git) registered for path '3rdparty/dlpack'
Submodule 'dmlc-core' (https://github.com/dmlc/dmlc-core.git) registered for path '3rdparty/dmlc-core'
Submodule '3rdparty/libbacktrace' (https://github.com/tlc-pack/libbacktrace.git) registered for path '3rdparty/libbacktrace'
Submodule '3rdparty/rang' (https://github.com/agauniyal/rang.git) registered for path '3rdparty/rang'
Cloning into '/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/tvm/3rdparty/OpenCL-Headers'...
Cloning into '/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/tvm/3rdparty/cnpy'...
Cloning into '/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/tvm/3rdparty/cutlass'...
Cloning into '/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/tvm/3rdparty/dlpack'...
Cloning into '/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/tvm/3rdparty/dmlc-core'...
Cloning into '/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/tvm/3rdparty/libbacktrace'...
Cloning into '/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/tvm/3rdparty/rang'...
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 41, done.
remote: Counting objects: 100% (41/41), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 21 (delta 15), reused 3 (delta 1), pack-reused 0
Unpacking objects: 100% (21/21), 22.24 KiB | 785.00 KiB/s, done.
From https://github.com/KhronosGroup/OpenCL-Headers
 * branch            b590a6bfe034ea3a418b7b523e3490956bcb367a -> FETCH_HEAD
Submodule path '3rdparty/OpenCL-Headers': checked out 'b590a6bfe034ea3a418b7b523e3490956bcb367a'
Submodule path '3rdparty/cnpy': checked out '4e8810b1a8637695171ed346ce68f6984e585ef4'
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 1344, done.
remote: Counting objects: 100% (1344/1344), done.
remote: Compressing objects: 100% (565/565), done.
remote: Total 818 (delta 635), reused 333 (delta 235), pack-reused 0
Receiving objects: 100% (818/818), 743.92 KiB | 7.29 MiB/s, done.
Resolving deltas: 100% (635/635), completed with 368 local objects.
From https://github.com/NVIDIA/cutlass
 * branch            d8359c804b7e3915a0f0668c19213f63ae88aac6 -> FETCH_HEAD
Submodule path '3rdparty/cutlass': checked out 'd8359c804b7e3915a0f0668c19213f63ae88aac6'
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 12 (delta 9), reused 4 (delta 1), pack-reused 0
Unpacking objects: 100% (12/12), 1.69 KiB | 82.00 KiB/s, done.
From https://github.com/dmlc/dlpack
 * branch            e2bdd3bee8cb6501558042633fa59144cc8b7f5f -> FETCH_HEAD
Submodule path '3rdparty/dlpack': checked out 'e2bdd3bee8cb6501558042633fa59144cc8b7f5f'
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 91, done.
remote: Counting objects: 100% (91/91), done.
remote: Compressing objects: 100% (43/43), done.
remote: Total 46 (delta 40), reused 5 (delta 0), pack-reused 0
Unpacking objects: 100% (46/46), 9.40 KiB | 332.00 KiB/s, done.
From https://github.com/dmlc/dmlc-core
 * branch            09511cf9fe5ff103900a5eafb50870dc84cc17c8 -> FETCH_HEAD
Submodule path '3rdparty/dmlc-core': checked out '09511cf9fe5ff103900a5eafb50870dc84cc17c8'
Submodule path '3rdparty/libbacktrace': checked out '08f7c7e69f8ea61a0c4151359bc8023be8e9217b'
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 16, done.
remote: Counting objects: 100% (16/16), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 9 (delta 5), reused 2 (delta 0), pack-reused 0
Unpacking objects: 100% (9/9), 1.86 KiB | 272.00 KiB/s, done.
From https://github.com/agauniyal/rang
 * branch            cabe04d6d6b05356fa8f9741704924788f0dd762 -> FETCH_HEAD
Submodule path '3rdparty/rang': checked out 'cabe04d6d6b05356fa8f9741704924788f0dd762'
/home/sre/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02
[tvm] Done
[DL conformance Models] Checking ...
[DL conformance Models] Done
[core secdev] Checking ...
[core secdev] Checking ... Done
[ATF] Checking ...
Cloning into 'trusted-firmware-a'...
remote: Enumerating objects: 52565, done.
remote: Counting objects: 100% (52565/52565), done.
remote: Compressing objects: 100% (15713/15713), done.
remote: Total 123406 (delta 49285), reused 36855 (delta 36852), pack-reused 70841
Receiving objects: 100% (123406/123406), 25.21 MiB | 15.38 MiB/s, done.
Resolving deltas: 100% (84869/84869), done.
Note: switching to '2fcd408bb3a6756767a43c073c597cef06e7f2d5'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 2fcd408bb feat(ti): do not handle EAs in EL3
v2.8-226-g2fcd408bb
[ATF] Checking ... Done
[OPTEE] Checking ...
Cloning into 'optee_os'...
remote: Enumerating objects: 68791, done.
remote: Counting objects: 100% (1212/1212), done.
remote: Compressing objects: 100% (647/647), done.
remote: Total 68791 (delta 743), reused 801 (delta 564), pack-reused 67579
Receiving objects: 100% (68791/68791), 28.16 MiB | 17.83 MiB/s, done.
Resolving deltas: 100% (48794/48794), done.
Note: switching to '8e74d47616a20eaa23ca692f4bbbf917a236ed94'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 8e74d4761 Update CHANGELOG for 3.20.0
3.20.0
[OPTEE] Checking ... Done
[opkg-utils] Checking ...
--2023-08-28 15:24:51--  https://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/snapshot/opkg-utils-master.tar.gz
Resolving git.yoctoproject.org (git.yoctoproject.org)... 199.204.45.23
Connecting to git.yoctoproject.org (git.yoctoproject.org)|199.204.45.23|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://git.yoctoproject.org/opkg-utils/snapshot/opkg-utils-master.tar.gz [following]
--2023-08-28 15:24:52--  https://git.yoctoproject.org/opkg-utils/snapshot/opkg-utils-master.tar.gz
Reusing existing connection to git.yoctoproject.org:443.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘opkg-utils-master.tar.gz’

opkg-utils-master.tar.gz                                [ <=>                                                                                                             ]  36.68K  --.-KB/s    in 0.03s   

2023-08-28 15:24:52 (1.21 MB/s) - ‘opkg-utils-master.tar.gz’ saved [37560]

[opkg-utils] Done
[glm] Checking ...
--2023-08-28 15:24:52--  https://github.com/g-truc/glm/releases/download/0.9.8.0/glm-0.9.8.0.zip
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/5695275/f8a9f524-781f-11e6-8d42-fb49cc9c429c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230828%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230828T192452Z&X-Amz-Expires=300&X-Amz-Signature=1af6f255cd4af05ac8ee85a37db1e50a6b69767a7ce7096718986d4486f5e006&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5695275&response-content-disposition=attachment%3B%20filename%3Dglm-0.9.8.0.zip&response-content-type=application%2Foctet-stream [following]
--2023-08-28 15:24:52--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/5695275/f8a9f524-781f-11e6-8d42-fb49cc9c429c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230828%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230828T192452Z&X-Amz-Expires=300&X-Amz-Signature=1af6f255cd4af05ac8ee85a37db1e50a6b69767a7ce7096718986d4486f5e006&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5695275&response-content-disposition=attachment%3B%20filename%3Dglm-0.9.8.0.zip&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.108.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4349363 (4.1M) [application/octet-stream]
Saving to: ‘glm-0.9.8.0.zip’

glm-0.9.8.0.zip                                     100%[================================================================================================================>]   4.15M  --.-KB/s    in 0.1s    

2023-08-28 15:24:53 (28.6 MB/s) - ‘glm-0.9.8.0.zip’ saved [4349363/4349363]

[glm] Done
[pip] Checking ...
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /home/sre/.local/lib/python3.10/site-packages (23.2.1)
[pip] Checking ... Done
[pip] Installing dependant python packages ...
Requirement already satisfied: pycryptodomex in /home/sre/.local/lib/python3.10/site-packages (3.18.0)
Requirement already satisfied: meson in /home/sre/.local/lib/python3.10/site-packages (1.2.1)
Requirement already satisfied: jsonschema in /home/sre/.local/lib/python3.10/site-packages (4.19.0)
Requirement already satisfied: attrs>=22.2.0 in /home/sre/.local/lib/python3.10/site-packages (from jsonschema) (23.1.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/sre/.local/lib/python3.10/site-packages (from jsonschema) (2023.7.1)
Requirement already satisfied: referencing>=0.28.4 in /home/sre/.local/lib/python3.10/site-packages (from jsonschema) (0.30.2)
Requirement already satisfied: rpds-py>=0.7.1 in /home/sre/.local/lib/python3.10/site-packages (from jsonschema) (0.9.2)
[pip] Installing dependant python packages ... Done
Packages installed successfully

    logs_make_tidl_tools.txt

  • 0 in reply to srikar varaganti
    Intellectual 280 points

    Also,

    Anand and Daviel.

    As we need to run the OD application in Target mode, we are looking more towards resolving this issue in target mode. Do you guys mind trying to generate these artifacts in target mode (Linux release mode) and let me know if you were able to run the tool.

    thank you.

  • 0 in reply to srikar varaganti
    Intellectual 280 points

    After gaining access to installer file for Linux. I completed successful setup using "--instal_tidl_deps" option. further when i tried "make tidl_pc_tools" I am still getting the errors as mentioned in the error logs attached:

    new_logs_make_tidl_tools.txt

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Hi Srikar,

    Can you do below updates and check if "illegal instruction" error gets resolved :

    In sdk_builder/concerto/compilers/gcc_linux.mak

    ifeq ($(TARGET_FAMILY),ARM)

    $(_MODULE)_COPT += -mapcs -mno-sched-prolog -mno-thumb-interwork

    ifeq ($(TARGET_OS),LINUX)

    $(_MODULE)_COPT += -mabi=aapcs-linux

    endif

    endif

     

    ifeq ($(TARGET_CPU), $(filter $(TARGET_CPU), X86 x86_64))

    $(_MODULE)_COPT += -march=x86-64 -pthread

    else ifeq ($(HOST_CPU),$(TARGET_CPU))

    $(_MODULE)_COPT += -march=native -pthread

    else ifeq ($(TARGET_CPU),M3)

    Note: The order of ifeq conditions above is reversed compared to existing code (x86-64 followed by host_cpu)

    Also, in c7x-mma-tidl repo (I know you have already tried below, putting here for completeness)

    makerules/rules.mk

      ifeq ($(TARGET_BUILD), debug)

        COMPILER_FLAGS += -std=c++14  -DHOST_EMULATION -w -D_HOST_BUILD -DGCC_BUILD

        GCC_DEBUG_CFLAGS :=-ggdb -ggdb3 -gdwarf-2

      else

        ifeq ($(DEVELOPER_BUILD), 0)

          COMPILER_FLAGS += -std=c++14 -O3 -DHOST_EMULATION -D_HOST_BUILD -DGCC_BUILD -march=x86-64

        else

          COMPILER_FLAGS += -std=c++14 -O3 -DHOST_EMULATION -D_HOST_BUILD -DGCC_BUILD -g -march=x86-64

        endif

      endif

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    Hi Anand,

    I was able to compile the tidl_pc_tools step with the changes you mentioned above. However, the import tool is still giving the same error. I have attached the logs file for your reference.

    Fullscreen importer_logs.txt Download 
    stoneridge@stoneridge-VirtualBox:~/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/utils/tidlModelImport$ ./out/tidl_model_import.out /home/stoneridge/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/onnx/model_v1/tidl_import_model_v1.txt 
TIDL Meta PipeLine (Proto) File  : /home/stoneridge/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/models/public/onnx/model_v1/model.prototxt  
yolo_v3
yolo_v3
ONNX Model (Proto) File  : /home/stoneridge/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/models/public/onnx/model_v1/model.onnx  
TIDL Network File      : /home/stoneridge/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_net_model_v1.bin  
TIDL IO Info File      : /home/stoneridge/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_io_model_v1_  
Current ONNX OpSet Version   : 11  
Empty prototxt path, running calibration

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/stoneridge/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_import_model_v1.txt.qunat_stats_config.txt 
Illegal instruction (core dumped)

 
 
 *****************   Calibration iteration number 0 started ************************ 
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/stoneridge/ti-processor-sdk-rtos-j784s4-evm-09_00_00_02/c7x-mma-tidl/ti_dl/test/testvecs/output/onnx/model_v1/tidl_import_model_v1.txt.qunat_stats_config.txt 
Illegal instruction (core dumped)

 
 
 *****************   Calibration iteration number 0 completed ************************ 
 
 
 

------------------ Network Compiler Traces -----------------------------
NC running for device: 1
Running with OTF buffer optimizations
successful Memory allocation
INFORMATION: [TIDL_ResizeLayer] Resize_149 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_168 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
TIDL ALLOWLISTING LAYER CHECK: TIDL_E_QUANT_STATS_NOT_AVAILABLE] tidl_quant_stats_tool.out fails to collect dynamic range. Please look into quant stats log. This model will get fault on target.
****************************************************
**          2 WARNINGS          1 ERRORS          **
****************************************************

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Hi Srikar,

    We regenerated the tidl_tools required for compilation to prevent the illegal instruction issue you are observing. They are uploaded at this link now : https://software-dl.ti.com/jacinto7/esd/tidl-tools/09_00_00_02/TIDL_TOOLS/AM69A/tidl_tools.tar.gz

    You can download and replace the existing tidl_tools folder with above in your c7x-mma-tidl repo.

    Can you please try re-compiling with this change? We have validated these tools on different machines at our end to ensure the error does not occur.

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    Hi Anand,

    Thank you for the response.

    As mentioned, I performed following steps and still could see the same error from the tool: I do not specifically find edgai-tidl-tools folder in sdk, but I found this the contents of the tar.gz file to be similar to the contents at $(PSDKR_PATH)/c7x-mma-tidl/tidl_tools. So, I am assuming by edgeai-tidl-tools you meant c7x-mma-tidl/tidl_tools. 

    1. Extracted the new tidl_tools.tar.gz file at $(PSDKR_PATH)/c7x-mma-tidl/ (The tar file had tidl_tools folder which replaced the previous tidl_tools folder in this path)

    2. copied the new tidl_model_import.out file from new tidl_tools folder to $(PSDKR_PATH)/c7x-mma-tidl/ti_dl/utils/tidlModelImport/out/ 

    3. I ran this import tool with the .txt file (file containing paths for onnx ,prototxt, output folders) previously attached and found the error was still persistent. 

    4. I replaced the current device_config.cfg with the file from new tar.gz folder and ran the importer tool: still found the same error.

    5. I also tried the: "setup_psdk_rtos.sh --install_tidl_deps" step and then "make tidl-pc-tools" and ran the import tool as well and found the error was still persistent. 

    Let me know your thoughts on this. 

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Hi Srikar,

    Can you please try replacing all the contents of this tidl_tools folder in the correponding soft link locations in the released TIDL package similar to what you did for tidl_model_import.out?

    Just on sidelines FYI -- edgeai-tidl-tools is another open source interface based entry point for TIDL (https://github.com/TexasInstruments/edgeai-tidl-tools) which uses tidl_tools as they are in the form I shared. What you are running now using import and infer config files is TIDL custom interface way to run the same thing but it requires the tidl_tools in the relative paths as packaged in release, so you will need to update the binaries in the actual soft linked paths.

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    Hi Anand,

    Could you please elaborate on what you meant by soft-link locations here. We tried reviewing the folders and found that no soft links were used within the respective folders. Also, by extracting the tar.gz file I have replaced all the files in tidl_tools folder with the updated files. And I have updated the files in  $(PSDKR_PATH)/c7x-mma-tidl/ti_dl/utils/tidlModelImport/out/ as well with the updated files and found still couldn't resolve the issue.

    Let me know if by updating corresponding soft links you meant to change the files in the respective folders as I did or is it a different procedure you are referring to? We also checked with Daviel regarding this, and he was also confused regarding this. Please advise.

    Thank you,

    Srikar. V

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Hi Srikar,

    The release package tidl_tools contain following:

    ls -al tidl_tools:

    The execution picks up the tools from the actual location pointed to by the soft link and not directly from tidl_tools folder. So, as an example, you will need to replace ti_cnnperfsim.out in "../ti_dl/utils/perfsim/ti_cnnperfsim.out" path rather than just in tidl_tools/ti_cnnperfsim.out, which is similar to what you did for tidl_model_import.out (update in ti_dl/utils/tidlModelImport/out/).

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    Hello Anand,

    I tried updating the soft-link locations as you mentioned in the image in your previous comment. I am now able to generate artifacts for the custom model without any errors. However, when used these artifacts with the default OD application provided, I could not see any detections in the output generated. While the default peele artifacts were able to generate the detection results. Please note that the customized model was able to generate the detections with TDA mid. I am also not seeing any errors during the run stage; however, the results are just blank images without detections drawn on them. 

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Ok Srikar, good to know artifacts could be generated and you are able to compile the model now. From desciption, this seems to be a functional issue in the model which could either be related to a bug or some quantization loss. Normal way to go about this debug would be following sequential steps:

    1) Compile with numParamBits = 32 and run inference in host emulation mode to check if detection boxes appear (rules out basic functional issues)

    2) If yes, try with numParamBits = 16 - check emulation mode, if it works fine, check on H/W target (rules out basic quantization issues)

    3) If 2 works fine, check with numParamBits = 8 in host emulation mode, and then check on HW  (if 2 works and 3 does not, that means there is quantization loss and advanced calibration options need to be used with more frames and more iterations)

    Can you try out above in sequence and let me know till what point detections work fine?

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Intellectual 280 points

    Hi Anand,

    We have tried running the OD app with the newly generated artifacts for our custom model in Linux release mode and was able to successfully run the application inference. 

    One thing I am curious about is to understand what was the major change in the new tidl_tools (I tried generating the artifacts using the tool in the new sdk: 09_00_01_01 and this time the artifacts are generated without any errors) compared to the previous source code. I tried comparing the .cpp and .c files to understand if there was any change but found no differences in the code. Could you please brief up on what the new tidl_tools changed from the previous version.

  • 0 in reply to srikar varaganti
    TI__Genius 9065 points

    Hi Srikar,

    It's mainly about the build flags, not about code change. The tools for previous release version were built with a high end machine at our end using march = native flag for build, which built the tools for the high end machine's native architecture, making it incompatible with any older architectures. This version rebuilt the tools with march = x86-64 flag which would work for generic x86 arch and not any specific arch.

    Regards,

    Anand