TDA4VM: model quantization accuracy loss

chao yang

Part Number: TDA4VM

448*448 model quantization accuracy loss

1.Use quantization files

modelType          = 2
numParamBits = 8
numFeatureBits     = 8
quantizationStyle = 2
inputNetFile       = "/root/model/zhurh_yolo/model/2_stage/20220114_two_stage/export_ssd_2stage448.onnx"
outputNetFile      = "/root/model/zhurh_yolo/model/2_stage/20220114_two_stage/out/tidl_net_448_Reg.bin"
outputParamsFile   = "/root/model/zhurh_yolo/model/2_stage/20220114_two_stage/out/tidl_io_448_Reg_"
inDataNorm  = 1
inMean = 128 128 128
inScale = 0.0078125 0.0078125 0.0078125
inDataFormat = 0
inWidth  = 448
inHeight = 448
inNumChannels = 3
numFrames = 100
inData = /root/model-8.0/model/Front_Detect_model/dra_imgs/dra_image_list_dra_448.txt
postProcType = 1
#perfSimConfig = ../../test/testvecs/config/import/perfsim_base.cfg

2.Use environment

model : v0.63

SDK: 8.0

3.problem phenomenon

PC:

Emulator:

Describe：

Before quantization, the model locates the front and rear target boxes with accurate accuracy, but after quantization, the model has large deviations in the front and rear target boxes.

over 4 years ago

0 Anshu Jain over 4 years ago

TI__Guru 56820 points

Hi,

Please refer the following troubleshooting steps for debugging quantization loss:

https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_01_00_11/exports/docs/tidl_j7_08_01_00_05/ti_dl/docs/user_guide_html/md_tidl_fsg_steps_to_debug_mismatch.html

Regards,

Anshu

0 chao yang over 4 years ago in reply to Anshu Jain

Prodigy 170 points

model_results.zip

My problem is the inconsistency between the host emulation and EVM results.

The experimental investigation found the following conclusions：

The PC results are similar to the PC emulator (Host emulation after quantization) results , but the results of the EVM execution and the PC emulation are abnormal after quantization.

I use the binary comparison tool to compare the data of each layer, and I find that the difference between the EVM execution result and the quantized PC simulator result is different from the 60th layer.

the 60th layer used AvgPool（stride=2，kernel=3，padding=0）

The data sent contains the following(model_results)：

1）SDK8.0 quantized model file ：tidl_io_448_Reg_1.bin , tidl_net_448_Reg.bin

2）Test the original image ：4x4.bmp , img.y

3）abnormal data : hardware, pc_emulator

0 Anand Pathak over 4 years ago in reply to chao yang

TI__Genius 9065 points

Hi,

Can you please share the model and the config files you are using as well so I can reproduce the issue on my end and investigate further?

Regards,

Anand

0 chao yang over 4 years ago in reply to Anand Pathak

Prodigy 170 points

ok，

model : export_ssd_2stage_448_Reg.onnx

the config files : import_ssd_class.txt

see model.zip

8764.model.zip

0 chao yang over 4 years ago in reply to chao yang

Prodigy 170 points

TI :

Can you pls help to fix the warning at your model?

Can you pls share the root/model-8.0/model/Front_Detect_model/dra_imgs/dra_image_list_dra_448.txt file and input image. It’s for quantization.

Current ONNX OpSet Version : 9

Warning : Merging Pad layer with Average Pooling layer. This is expected to work but this flow is functionally not validated with ONNX model format.

INFO : Couldn't open inData file: /root/model-8.0/model/Front_Detect_model/dra_imgs/dra_image_list_dra_448.txt , Skipping Range Collection for Quantization

===》

answer：

ok，see change_model_and_imgs.zip

change_model_and_imgs.zip

0 Anand Pathak over 4 years ago in reply to chao yang

TI__Genius 9065 points

Hi,

I am able to reproduce the issue and am investigating it further. Will get back soon with further update.

Jira reference for TI internal tracking:

https://jira.itg.ti.com/browse/TIDL-1877

Regards,

Anand

0 chao yang over 4 years ago in reply to Anand Pathak

Prodigy 170 points

HI， Anand Pathak

1, When will you be able to solve this problem?Do you have a specific time point?

2，What do you suggest to modify this model？The model suffers from a drop in accuracy when modifying the avgpool layer.

0 Anand Pathak over 4 years ago in reply to chao yang

TI__Genius 9065 points

Hi,

I have identified the issue in average pool layer and it will be resolved as part of the upcoming SDK 8.2 release which is scheduled for around 25th March.

Regards,

Anand

0 chao yang over 4 years ago in reply to Anand Pathak

Prodigy 170 points

Thank you!

0 Anand Pathak over 4 years ago in reply to chao yang

TI__Genius 9065 points

Can I go ahead and close this thread?

0 Anand Pathak over 4 years ago in reply to Anand Pathak

TI__Genius 9065 points

Closing this thread. Feel free to reopen if you have any further questions.

Regards,

Anand

Processors

Processors forum

TDA4VM: model quantization accuracy loss

448*448 model quantization accuracy loss

1.Use quantization files

2.Use environment

3.problem phenomenon

model_results.zip

My problem is the inconsistency between the host emulation and EVM results.

The experimental investigation found the following conclusions：

The data sent contains the following(model_results)：