• State TI Thinks Resolved
  • Replies 13 replies
  • Answers 2 answers
  • Subscribers 103 subscribers
  • Views 346 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

TDA4VL-Q1: [E-mirror] Is TIDL in PC emulation mode different form target mode?

alun yan
Intellectual 845 points
Part Number: TDA4VL-Q1

Tool/software:

Hello TI:

           We have a .onnx model and it is HRNet. We use tidl_model_import.out convert to two bin file. And follow is our import txt file.

tidl_import_mymodel_detection.txt 
modelType          = 2
numParamBits       = 8
numFeatureBits     = 8
#quantizationStyle  = 3
quantizationStyle  = 2
inputNetFile       = "../../test/testvecs/sh_model/keypoint/model/mymodel.onnx"
outputNetFile      = "../../test/testvecs/sh_model/keypoint/out/tidl_net_mymodel.bin"
outputParamsFile   = "../../test/testvecs/sh_model/keypoint/out/tidl_io_mymodel_"
inData  =   "../../test/testvecs/sh_model/keypoint/in/detection_list.txt"

perfSimConfig = ../../test/testvecs/config/import/device_config.cfg
#perfSimConfig = ../../test/testvecs/sh_model/keypoint/in/device_config.cfg

inDataNorm  = 1
inMean = 123.675 116.28 103.53
inScale = 0.017125 0.017507 0.017429

inWidth  = 256
inHeight = 256
inNumChannels = 3

inDataFormat = 1
inResizeType = 1
numFrames = 1
inElementType = 0

     Then we get two bin file. And we write a app, follow is our graph.

     And we find that the result we get from pc emulation is correct. But the result was wrong when the we run in target.

     We use same bin file in both emulation mode and target mode. Our model is HRNet which is used for face detection.

     So we want to know whether or not emulation mode is different from target mode when we use tidl?

regards,

  • 0
    TI__Genius 10170 points

    Hello,

    I apologize for the delay in getting to your question. 

    Can you clarify what version of TIDL and SDK you are using? 

    Before integrating your model as a TIDL node, did you test the inference within the TIDL import/inference interface with PC_dsp_test_dl_algo.out (host emulation) and TI_DEVICE_a72_test_dl_algo_host_rt.out (target) and verify that the results were correct? 

    If not, could you follow the steps for debugging target vs host emulation mismatch in Steps to Debug Error Scenarios for target(EVM) execution that is documented here? Please report the results that you get back on this thread. 

    Best,

    Asha

  • 0 in reply to Asha Bhandarkar
    Intellectual 845 points

    Hello, 

    Asha Bhandarkar said:
    Can you clarify what version of TIDL and SDK you are using? 

            Our sdk version is 8.5.

    Asha Bhandarkar said:
    Before integrating your model as a TIDL node, did you test the inference within the TIDL import/inference interface with PC_dsp_test_dl_algo.out (host emulation) and TI_DEVICE_a72_test_dl_algo_host_rt.out (target) and verify that the results were correct? 

           We found that the result of host emulation is correct but the result of target is wrong. 

    Asha Bhandarkar said:
    If not, could you follow the steps for debugging target vs host emulation mismatch in Steps to Debug Error Scenarios for target(EVM) execution that is documented here? Please report the results that you get back on this thread. 

           We use this method to debug error. And we found that the two results is difference begin at the data id of 0048.

          

          How to local data id? 

    tidl_net_mymodel.bin.layer_info.txt 
    0 0 input_original 
1 1 1842 
2 2 1845 
3 3 1848 
4 5 1851 
5 6 3075 
6 4 3078 
7 7 1857 
8 8 1860 
9 9 1863 
10 10 3087 
11 11 1867 
12 12 1870 
13 13 1873 
14 14 3096 
15 15 1877 
16 16 1880 
17 17 1883 
18 18 3105 
19 19 1887 
20 20 1893 
21 22 1924 
22 24 3141 
23 26 1928 
24 28 1931 
25 30 3147 
26 32 1935 
27 34 1938 
28 36 3153 
29 38 1942 
30 40 1945 
31 42 3159 
32 44 1949 
33 46 3162 
34 48 1956 
35 21 1890 
36 23 1896 
37 25 3117 
38 27 1900 
39 29 1903 
40 31 3123 
41 33 1907 
42 35 1910 
43 37 3129 
44 39 1914 
45 41 1917 
46 43 3135 
47 45 1921 
48 47 3165 
49 49 1962 
50 51 1996 
51 54 3198 
52 57 2000 
53 60 2003 
54 63 3204 
55 66 2007 
56 69 2010 
57 72 3210 
58 75 2014 
59 78 2017 
60 81 3216 
61 84 2021 
62 87 3243 
63 92 2056 
64 88 3261 
65 53 1965 
66 55 2024 
67 59 3222 
68 62 2028 
69 64 2031 
70 68 3228 
71 71 2035 
72 73 2038 
73 77 3234 
74 80 2042 
75 82 2045 
76 86 3240 
77 89 2049 
78 94 3252 
79 98 2076 
80 96 3246 
81 99 2064 
82 50 1958 
83 52 1968 
84 56 3174 
85 58 1972 
86 61 1975 
87 65 3180 
88 67 1979 
89 70 1982 
90 74 3186 
91 76 1986 
92 79 1989 
93 83 3192 
94 85 1993 
95 90 3249 
96 93 2069 
97 101 2078 
98 104 2119 
99 107 3291 
100 110 2123 
101 113 2126 
102 116 3297 
103 119 2130 
104 122 2133 
105 125 3303 
106 128 2137 
107 131 2140 
108 134 3309 
109 137 2144 
110 140 3336 
111 145 2179 
112 141 3354 
113 91 2081 
114 97 3258 
115 100 2086 
116 103 2088 
117 106 2147 
118 109 3315 
119 112 2151 
120 115 2154 
121 118 3321 
122 121 2158 
123 124 2161 
124 127 3327 
125 130 2165 
126 133 2168 
127 136 3333 
128 139 2172 
129 143 3345 
130 148 2199 
131 146 3339 
132 150 2187 
133 95 2057 
134 102 2066 
135 105 2091 
136 108 3267 
137 111 2095 
138 114 2098 
139 117 3273 
140 120 2102 
141 123 2105 
142 126 3279 
143 129 2109 
144 132 2112 
145 135 3285 
146 138 2116 
147 142 3342 
148 147 2192 
149 151 2201 
150 155 2242 
151 158 3384 
152 161 2246 
153 164 2249 
154 167 3390 
155 170 2253 
156 173 2256 
157 176 3396 
158 179 2260 
159 182 2263 
160 185 3402 
161 188 2267 
162 191 3429 
163 196 2302 
164 192 3447 
165 144 2204 
166 152 3351 
167 154 2209 
168 157 2211 
169 159 2270 
170 163 3408 
171 166 2274 
172 168 2277 
173 172 3414 
174 175 2281 
175 177 2284 
176 181 3420 
177 184 2288 
178 186 2291 
179 190 3426 
180 193 2295 
181 198 3438 
182 202 2322 
183 200 3432 
184 203 2310 
185 149 2180 
186 153 2189 
187 156 2214 
188 160 3360 
189 162 2218 
190 165 2221 
191 169 3366 
192 171 2225 
193 174 2228 
194 178 3372 
195 180 2232 
196 183 2235 
197 187 3378 
198 189 2239 
199 194 3435 
200 197 2315 
201 205 2324 
202 208 2365 
203 211 3477 
204 214 2369 
205 217 2372 
206 220 3483 
207 223 2376 
208 226 2379 
209 229 3489 
210 232 2383 
211 235 2386 
212 238 3495 
213 241 2390 
214 246 3540 
215 245 3522 
216 249 2425 
217 195 2327 
218 201 3444 
219 204 2332 
220 207 2334 
221 210 2393 
222 213 3501 
223 216 2397 
224 219 2400 
225 222 3507 
226 225 2404 
227 228 2407 
228 231 3513 
229 234 2411 
230 237 2414 
231 240 3519 
232 243 2418 
233 250 3525 
234 253 2433 
235 251 3531 
236 254 2445 
237 199 2303 
238 206 2312 
239 209 2337 
240 212 3453 
241 215 2341 
242 218 2344 
243 221 3459 
244 224 2348 
245 227 2351 
246 230 3465 
247 233 2355 
248 236 2358 
249 239 3471 
250 242 2362 
251 244 3528 
252 247 2438 
253 257 2447 
254 260 2491 
255 264 3573 
256 267 2495 
257 271 2498 
258 276 3579 
259 279 2502 
260 283 2505 
261 288 3585 
262 291 2509 
263 295 2512 
264 300 3591 
265 303 2516 
266 306 3642 
267 313 2579 
268 311 3666 
269 312 2646 
270 323 3684 
271 248 2450 
272 255 3537 
273 258 2455 
274 261 2457 
275 262 2519 
276 268 3597 
277 272 2523 
278 274 2526 
279 280 3603 
280 284 2530 
281 286 2533 
282 292 3609 
283 296 2537 
284 298 2540 
285 304 3615 
286 310 2544 
287 319 3645 
288 325 2587 
289 320 3654 
290 326 2607 
291 321 3687 
292 265 2460 
293 269 2547 
294 273 3621 
295 277 2551 
296 281 2554 
297 285 3627 
298 289 2558 
299 293 2561 
300 297 3633 
301 301 2565 
302 305 2568 
303 315 3639 
304 322 2572 
305 330 3669 
306 335 2633 
307 331 3648 
308 392 2595_TIDL_0 
309 394 2595 
310 333 3657 
311 336 2615 
312 252 2426 
313 256 2435 
314 259 2463 
315 263 3549 
316 266 2467 
317 270 2470 
318 275 3555 
319 278 2474 
320 282 2477 
321 287 3561 
322 290 2481 
323 294 2484 
324 299 3567 
325 302 2488 
326 307 3651 
327 314 2600 
328 332 2608 
329 339 2617 
330 342 2685 
331 345 3717 
332 348 2689 
333 351 2692 
334 354 3723 
335 357 2696 
336 360 2699 
337 363 3729 
338 366 2703 
339 369 2706 
340 372 3735 
341 375 2710 
342 379 2840 
343 390 3828 
344 386 3810 
345 387 3786 
346 391 2773 
347 308 2620 
348 317 3663 
349 324 2625 
350 327 2626 
351 338 2635 
352 341 2713 
353 344 3741 
354 347 2717 
355 350 2720 
356 353 3747 
357 356 2724 
358 359 2727 
359 362 3753 
360 365 2731 
361 368 2734 
362 371 3759 
363 374 2738 
364 377 3789 
365 381 2781 
366 378 3798 
367 384 2801 
368 380 3831 
369 309 2638 
370 318 2641 
371 328 3678 
372 334 2649 
373 337 2652 
374 340 2654 
375 343 2741 
376 346 3765 
377 349 2745 
378 352 2748 
379 355 3771 
380 358 2752 
381 361 2755 
382 364 3777 
383 367 2759 
384 370 2762 
385 373 3783 
386 376 2766 
387 382 3792 
388 393 2789_TIDL_0 
389 412 2789 
390 383 3813 
391 388 2827 
392 385 3801 
393 389 2809 
394 316 2580 
395 329 2588 
396 395 2597 
397 396 2657 
398 397 3693 
399 398 2661 
400 399 2664 
401 400 3699 
402 401 2668 
403 402 2671 
404 403 3705 
405 404 2675 
406 405 2678 
407 406 3711 
408 407 2682 
409 408 3795 
410 411 2794 
411 413 2802 
412 416 2811 
413 421 2879 
414 426 3861 
415 430 2883 
416 434 2886 
417 438 3867 
418 442 2890 
419 446 2893 
420 450 3873 
421 454 2897 
422 458 2900 
423 462 3879 
424 466 2904 
425 468 3954 
426 471 3034 
427 480 3972 
428 474 3930 
429 476 2967 
430 409 2814 
431 414 3807 
432 418 2819 
433 420 2820 
434 424 2829 
435 429 2907 
436 433 3885 
437 437 2911 
438 441 2914 
439 445 3891 
440 449 2918 
441 453 2921 
442 457 3897 
443 461 2925 
444 465 2928 
445 470 3903 
446 475 2932 
447 477 3975 
448 479 3942 
449 487 2995 
450 485 3933 
451 489 2975 
452 410 2832 
453 415 2835 
454 422 3822 
455 425 2843 
456 427 2846 
457 431 2848 
458 435 2935 
459 439 3909 
460 443 2939 
461 447 2942 
462 451 3915 
463 455 2946 
464 459 2949 
465 463 3921 
466 467 2953 
467 472 2956 
468 481 3927 
469 488 2960 
470 491 3945 
471 497 3003 
472 492 3957 
473 498 3021 
474 496 3936 
475 509 2983_TIDL_0 
476 511 2983 
477 417 2774 
478 419 2782 
479 423 2791 
480 428 2851 
481 432 3837 
482 436 2855 
483 440 2858 
484 444 3843 
485 448 2862 
486 452 2865 
487 456 3849 
488 460 2869 
489 464 2872 
490 469 3855 
491 473 2876 
492 478 3939 
493 486 2988 
494 490 2996 
495 500 3005 
496 505 3047 
497 482 3008 
498 493 3951 
499 499 3013 
500 501 3014 
501 503 3023 
502 507 3052 
503 483 3026 
504 494 3029 
505 502 3966 
506 504 3037 
507 506 3040 
508 508 3042 
509 510 3057_TIDL_0 
510 513 3057 
511 484 2968 
512 495 2976 
513 512 2985 
514 514 3058 
515 515 3061 
516 516 output 
517 0 input_original

    Best,

    Alun

  • 0 in reply to alun yan
    TI__Genius 10170 points

    Hi Alun,

    If you are having difficulty determining the layer from the layer_info.txt file, you can try the second method that is given in the documentation:

    Second way is to read the output of model visualization tool (*.svg file, gets generated in the same folder as TIDL model output location), here this information can be read from each layer box inside the square brackets [layerIdx, dataIdx].

    Best,

    Asha

  • 0 in reply to Asha Bhandarkar
    Intellectual 845 points

    Hi, 

        OK. Thank you very much. I get it.

    Best,

    Alun

  • 0 in reply to alun yan
    TI__Genius 10170 points

    Hi Alun,

    I apologize for not responding earlier to this. This looks like it pinpoints to the Resize layer being the issue. Are you able to test this model on the latest version of the toolset (this is now 10.0 SDK) as there would have been changes since 8.5 SDK. If you can also share the model that would also be beneficial.

    Best,

    Asha

  • 0 in reply to Asha Bhandarkar
    Intellectual 845 points

    Hi, 

       This is our model.

    mymodel.zip

    Best,

    Alun

  • 0 in reply to Asha Bhandarkar
    Intellectual 845 points

    Hi, 

        Sorry, We have no plans to  update  to 10.0  SDK.

  • 0 in reply to alun yan
    TI__Guru**** 436215 points

    Unlock this thread. 

  • 0 in reply to Brijesh Jadav
    Intellectual 845 points

    Hello, 

         Anything update?

    Best，

    Alun

  • 0 in reply to alun yan
    TI__Guru**** 436215 points

    Hi Alun,

    i would request to help. 

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    TI__Expert 6700 points

    Hi Alun,

    Asha has moved to another role, and I will do my best to help.   Is this still an issue?

    Chris

  • 0 in reply to Chris Tsongas
    Intellectual 845 points

    it is still an issue.

  • 0 in reply to alun yan
    TI__Expert 6700 points

    Hi Alun,

    The model appears to compile/run in testing with 10.0 but gives some warnings that may be causing the incorrect behaivior you are seeing.  Or at least inconsisthent behavior between emulation and device execution.  Perhaps this check was not in the 8.2 versiion and is what is causing the delta.

    WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored

    One point to make is I would encourage you to go to TIDL 10.1 as the 8.2 version 4 releases back and is no longer supported for updates.  You may be able to get by on 8.2 but updates will be difficult and you may not see the better diagnositcs available in newer TIDL releases. 

    Compile:

    mymodel_compile.txt 
    #!wget https://akm-img-a-in.tosshub.com/indiatoday/images/story/201804/jet.jpeg
modeldir = 'mymodel/'
modelname = 'mymodel.onnx'

# Set TIDL_TOOLS you want to use
# TIDL tools should be pre-installed from the command line
# after cloning edgeai-tidl-tools and running source ./setup.sh 
# after setting SOC environment variable
TIDL_TOOLS='/home/a0194920local/10_0/edgeai-tidl-tools/'
#!wget https://git.ti.com/cgit/jacinto-ai/jacinto-ai-modelzoo/plain/models/vision/classification/imagenet1k/torchvision/resnet18_opset9.onnx

!mkdir -p {modeldir} 
!cp jet.jpeg {modeldir}
#!wget https://akm-img-a-in.tosshub.com/indiatoday/images/story/201804/car.jpeg
#!mv car.jpeg {model_dir}
Create config and in_data_list.txt files.
These files contain the simulator configuration and a set of input images with their groundtruth.

!mkdir -p  {modeldir}
with open(modeldir+"/config", "w") as myfile:
    myfile.write("perfSimConfig = "+ TIDL_TOOLS + "tidl_tools/device_config.cfg")
#!echo "perfSimConfig = "+ TIDL_TOOLS + "tidl_tools/device_config.cfg" >>  {modeldir}/config
!echo "/home/a0194920local/colab-notebooks/{modeldir}/jet.jpeg 895" >  {modeldir}/in_data_list.txt
#!echo "/home/a0194920local/colab-notebooks/{modeldir}/car.jpeg 846" >>  {modeldir}/in_data_list.txt

#with open(modeldir+"in_data_list.txt", "a") as myfile:
#    myfile.write(TIDL_TOOLS + " {modeldir}/jet.jpeg 895")
Model import/compilation to generate artifacts for model inference on device






print("Compiling: ",modelname)
!/home/a0194920local/10_0/edgeai-tidl-tools/tidl_tools/tidl_model_import.out {modeldir}/config --modelType 2 \
--inputNetFile {modelname}  --outputNetFile {modeldir}/tidl_net.bin \
--outputParamsFile {modeldir}/tidl_io_buff  --inDataNorm 1 \
--inMean 123.675 116.28 103.53  --inScale 0.017125 0.017507 0.017429 \
--inData {modeldir}/in_data_list.txt --inFileFormat 2 \
--tidlStatsTool /home/a0194920local/10_0/edgeai-tidl-tools/tidl_tools//PC_dsp_test_dl_algo.out \
--perfSimTool /home/a0194920local/10_0/edgeai-tidl-tools/tidl_tools/ti_cnnperfsim.out \
--graphVizTool /home/a0194920local/10_0/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser.out \
--inHeight 224 --inWidth 224 --inNumChannels 3 --numFrames 1

Compiling:  mymodel.onnx
========================= [Model Compilation Started] =========================

Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning

============================== [Version Summary] ==============================

-------------------------------------------------------------------------------
|          TIDL Tools Version          |              10_00_08_00             |
-------------------------------------------------------------------------------
|         C7x Firmware Version         |              10_00_02_00             |
-------------------------------------------------------------------------------

ONNX model (Proto) file      : mymodel.onnx  
TIDL network file            : mymodel//tidl_net.bin  
TIDL IO info file            : mymodel//tidl_io_buff  
Current ONNX OpSet version   : 11  
============================ [Optimization started] ============================

----------------------------- Optimization Summary -----------------------------
--------------------------------------------------------------------------------
|         Layer         | Nodes before optimization | Nodes after optimization |
--------------------------------------------------------------------------------
| TIDL_EltWiseLayer     |                       170 |                      170 |
| TIDL_ConcatLayer      |                         1 |                        1 |
| TIDL_ReLULayer        |                       269 |                        0 |
| TIDL_ResizeLayer      |                        34 |                       38 |
| TIDL_ConvolutionLayer |                       307 |                      307 |
--------------------------------------------------------------------------------

=========================== [Optimization completed] ===========================


-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %

------------------ Fixed-point Calibration Iteration [1 / 1]: ------------------
[=============================================================================] 100 %

==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================


------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

========================= [Memory Planning Completed] =========================

[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
[TIDL Import]  WARNING: TIDL does not use coordinate_transformation_mode attribute in Resize layer, it will be ignored
======================== Subgraph Compiled Successfully ========================

    Host Run:

    mymodel_emu_run.txt

    Device Run:

    mymodel.txt