• State Resolved
  • Locked Locked
  • Replies 10 replies
  • Subscribers 100 subscribers
  • Views 1395 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: The inference of edgeai-yolov5 does not work well

Binqiao You
Prodigy 40 points
Part Number: TDA4VM

github.com/.../edgeai-yolov5

I trained the model of edgeai-yolov5 and got the prototxt file and the onnx file with the new export.py. After that, I complied the onnx model file to a int8 bin file and a float32 bin file. I run these two bin file with the PC_dsp_test_dl_algo.out file in ti-processor-sdk-rtos-j721e-evm-08_01_00_11/tidl_j7_08_01_00_05/ti_dl/test, and found bin file of float32 can't get the right result. 

The following picture is the import config. In the file, I set numParamBits = 8 and run tidl_model_import.out to get the bin file of int8

The result of float32 inference

The result of int8 inference

It can be seen from the picture above that the inference result of the float32 model has no rectangle found. Is there some thing wrong with my config file? Why the bin file of float32 can't get the right inference result?

Here is the import log of float32:

Because I will use the model of int8 in TDA4, is the above import config file right for the int8 bin file? 

  • 0
    TI__Intellectual 2406 points

    Hi,

    We have identified a bug in float32 mode recently[1833] . It will fixed in sdk8.2 release. In the meantime, you can try in 16 bits for close to float accuracy. 

    Regarding the config , set numParamBits = 8 as well. For better accuracy in 8 bit, you can use advance calibration by setting calibrationOption=7 and more number of frames(~50) for calibration.

    Fullscreen tidl_import_yolov5.txt Download 
    modelType          = 2
numParamBits       = 8
numFeatureBits     = 8
quantizationStyle  = 2
#quantizationStyle  = 3
inputNetFile       = ./yolov5s6_640_ti_lite_37p4_56p0.onnx
outputNetFile      = ./networks/tidl-q8/yolov5s6_640_fl16/config/import/tidl_net.bin
outputParamsFile   = ./networks/tidl-q8/yolov5s6_640_fl16/config/import/tidl_io_
inDataNorm  = 1
inMean = 0 0 0
inScale = 0.00392156862745098 0.00392156862745098 0.00392156862745098
inDataFormat = 1
inWidth  = 640
inHeight = 640 
inNumChannels = 3
numFrames = 50
inData  =   "../../test/testvecs/config/detection_list.txt"
perfSimConfig = ../../test/testvecs/config/import/device_config.cfg
inElementType = 0
metaArchType = 4
metaLayersNamesList = ./yolov5s6_640_ti_lite_metaarch.prototxt
postProcType = 2
calibrationOption = 7

  • 0 in reply to Debapriya Maji
    Prodigy 40 points

    Hi,

    I have compare your config and mine, and found numFrames is different. When I set numFrames = 50 as yours, the accuracy in int8 mode will be improved.

    So what is the function of this parameter? when I set numFrames = 10 and numFrames = 50, the inference result is different. Which value of numFrames is required to be set that can works for good accuracy in int8 mode.

  • 0 in reply to Binqiao You
    TI__Intellectual 2406 points

    Hi,

     You can set numFrames=50 and calibrationOption = 7 to improve the accuracy. Generally speaking, more the number of frames, better will be the  quantization. This will increase the time taken to import the model significantly. However, the accuracy will be better as well.

    Another important aspect of these YOLOV5 models is to use mixed precision where we use only the first and last convolution layers in 16 bits and rest of the layers in 8 bits. In our experiments with COCO, we were able to bridge the gap from float to close to 1% with these settings.

    You can do it by setting outputFeature16bitNamesList to appropriate values. If you can share the onnx model, I can help with setting this parameter.

    Regards, Debapriya

  • 0 in reply to Debapriya Maji
    Prodigy 40 points

    I have tried upload the onnx model, but failed. I saw outputFeature16bitNamesList can be got from <outputNetFile>.layer_info.txt. Can I share you the file and you help me setting this parameter.

  • 0 in reply to Binqiao You
    TI__Intellectual 2406 points

    Hi ,

    Please share the file. 

    Regards, Debapriya

  • 0 in reply to Debapriya Maji
    Prodigy 40 points

    Here is the file, thank you very much.

    Fullscreen tidl_net_yolo5.bin.layer_info.txt Download 
    0 0 images_original 
1 1 130 
2 2 132 
3 3 134 
4 4 143 
5 5 136 
6 6 138 
7 7 140 
8 8 141 
9 9 144 
10 10 146 
11 11 148 
12 12 150 
13 14 152 
14 15 154 
15 16 155 
16 17 157 
17 18 159 
18 19 160 
19 20 162 
20 21 164 
21 22 165 
22 13 167 
23 23 168 
24 24 170 
25 25 172 
26 26 174 
27 28 176 
28 29 178 
29 30 179 
30 31 181 
31 32 183 
32 33 184 
33 34 186 
34 35 188 
35 36 189 
36 27 191 
37 37 192 
38 38 194 
39 39 196 
40 40 198 
41 41 199 
42 44 200 
43 42 201 
44 45 202 
45 47 203 
46 49 204 
47 43 205 
48 46 206 
49 48 207 
50 50 208 
51 51 209 
52 52 210 
53 53 211 
54 54 213 
55 55 215 
56 57 217 
57 58 219 
58 56 221 
59 59 222 
60 60 224 
61 61 226 
62 62 231 
63 63 232 
64 64 234 
65 66 236 
66 67 238 
67 65 240 
68 68 241 
69 69 243 
70 70 245 
71 71 250 
72 72 251 
73 73 253 
74 75 255 
75 76 257 
76 74 259 
77 77 260 
78 78 262 
79 79 264 
80 81 265 
81 82 267 
82 84 269 
83 85 271 
84 83 273 
85 86 274 
86 87 276 
87 89 353 
88 80 291 
89 88 278 
90 90 279 
91 91 281 
92 93 283 
93 94 285 
94 92 287 
95 95 288 
96 96 290 
97 97 415 
98 98 tidl_yol_od_output_layer 
99 0 images_original

  • 0 in reply to Binqiao You
    TI__Intellectual 2406 points

    HI ,

     Can you please share the .svg file as well. It's difficult to tell which are the last conv layers in the detection heads that need to be set to 16 bits.

    Regards, Debapriya 

  • 0 in reply to Debapriya Maji
    Prodigy 40 points

    best.onnx.txt

    Hi,

    Here is the modeI file. I have tried and found this can only upload txt file. So I modify the suffix. After you download the mode file, you should modify the suffix to onnx. Please help me setting this parameter.

    Regards, You

  • +1 in reply to Binqiao You
    TI__Intellectual 2406 points

    Hi,

    Thanks for sharing the model. For this model, you can set

    outputFeature16bitNamesList = "130, 291, 353,  415 "

    You can also refer to this tutorial for further details about various parameters.

    https://training.ti.com/process-efficient-object-detection-using-yolov5-and-tda4x-processors

    Regards, Debapriya