PROCESSOR-SDK-TDAX: The int8 quantization scheme has serious loss of accuracy

bruce cui

The resnet50 network uses int8 quantization, and the model accuracy drops very seriously. Then use int16 quantization scheme, and the accuracy loss is acceptable. Is there a problem with int8 quantization?

The model used can be downloaded from here：github.com/.../pytorch-image-models

Below are my configuration parameters:

modelType = 2
numParamBits = 8
numFeatureBits = 8
quantizationStyle = 2
inputNetFile      = "./resnet50.onnx"
outputNetFile      = ".resnet50.bin"
outputParamsFile   = "./resnet50_io_"
inDataNorm = 1
inMean = 123.675 116.28 103.53
inScale =  0.017125 0.017507 0.017429
inWidth  = 224
inHeight = 224
inNumChannels = 3
inData = ./calibration.txt
postProcType = 0
#debugTraceLevel = 3
#writeTraceLevel = 3
calibrationOption = 0
flowCtrl = 0

over 2 years ago

0 Xu(SH) Liu over 2 years ago

TI__Expert 3995 points

As we aligned, kindly please figure out from which layer at first place the output feature map is about to loss too much of accuracy.

When this layer has been identified, we can look into the model and try to suggest how to insert some ancillary Batchnormal/scale layer to manage the dynamic range of numbers.

0 bruce cui over 1 year ago in reply to Xu(SH) Liu

Prodigy 70 points

Thank you for your reply， I agree with what you said. Is there any specific process that I can refer to?

0 Xu(SH) Liu over 1 year ago in reply to bruce cui

TI__Expert 3995 points

check with your algo engineer please to figure out which name of the first layer starts to loss too much of accuracy.

0 bruce cui over 1 year ago in reply to Xu(SH) Liu

Prodigy 70 points

Well, my understanding is that to perform 8-bits quantization on each layer of the network, and then check the layer with greatest accuracy loss. Does the TI quantization toolkit support layer-by-layer fixed points ?

0 Varun Tripathi over 1 year ago

TI__Genius 10415 points

Bruce,
We have a resnet50 model as part of our model zoo : https://github.com/TexasInstruments/edgeai-modelzoo/tree/main/models/vision/classification
You can take a look at the accuracy & try the off the shelf model for the same (We see negligible drop for 8-bit inference with resnet based architectures)

Thanks,
Varun

Processors

Processors forum

PROCESSOR-SDK-TDAX: The int8 quantization scheme has serious loss of accuracy