SK-TDA4VM: Which calibration methods does tidl importer use?

Mingzhen Zhu

Part Number: SK-TDA4VM

Tool/software:

During PTQ quantization, there are max, entropy and percentile methods to do calibration. I wonder which one does tidl importer use?

I refer to https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/07_03_00_07/exports/docs/tidl_j7_02_00_00_07/ti_dl/docs/user_guide_html/md_tidl_model_import.html and https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_01_00_13/exports/docs/tidl_j7_08_01_00_05/ti_dl/docs/user_guide_html/md_tidl_fsg_quantization.html but both don't help.

8 months ago

0 Wen Li 8 months ago

TI__Expert 7071 points

Hi MingZhen; Thanks for the question, our development team will answer your questions.

Best regards

Wen Li

0 Adam Hua 8 months ago

TI__Expert 4900 points

Hi Mingzhen,

Please refer to latest doc for quantization: https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_quantization.md

There are many classification method for quantization and the term varies in different framework.

Regards,

Adam

0 Mingzhen Zhu 8 months ago in reply to Adam Hua

Prodigy 130 points

If infer to NVIDIA https://arxiv.org/pdf/2004.09602, section 3.4 Calibration, TI's min and max calibration method doesn't match any of those three, do I miss something?

0 Adam Hua 8 months ago in reply to Mingzhen Zhu

TI__Expert 4900 points

Please refer to the doc I mentioned before. TI provides these several calibration methods, among which the simple calibration may correspond to the "max" method defined by Nvidia. It needs careful cross compare to determine whether TI's methos can be rated with Nvidia. You can check the doc and source code if you are interested.

B.1. Simple Calibration

This calibration can be enabled by setting accuracy_level = 0 during model compilation
- Supports Power of 2 and Non Power of 2 scales for parameters, this can be controlled using advanced_options:quantization_scale_type parameter
- Supports only power of 2 scales for feature maps
- Scale selected based on min and max values in the given layer
- Range for each feature maps are calibrated offline with few sample inputs
- Calibrated range (Min and Mix) Values are used for Quantizing feature maps in target during inference (real time)
- Observed accuracy drop less than 1% w.r.t floating point for many networks with 8-bits
  - For example models such as Resnets, SqueezeNet, VGG, etc ( especially models which don't use Depthwise convolution layers)

B.2. Advanced Calibration

TIDL product provides some advance options for calibration as listed below:

B.2.1. Advanced Bias calibration

This feature can be enabled by user by setting accuracy_level = 1 which is one of the optional parameter for model compilation. Typically no other parameter is required to be set because default parameters works for most of the cases. It is observed that using 50 or more number of images gives considerable accuracy boost.
This feature applies a clipping to the weights and update the bias to compensate the DC errors introduced because of quantization.
User can also experiment with following parameters related to this option if required:
- advanced_options:calibration_iterations: Number of iteration to be used for bias calibration.
- advanced_options:calibration_frames: Number of input frames to be used for bias calibration.

B.2.2. Histogram based activation range collection

To enable this feature user needs to set accuracy_level = 9 and advanced_options:activation_clipping = 1/0 to enable/disable this feature.
This feature uses the histogram of feature map activation ranges to remove outliers which can affect the overall range. This may help in reducing the accuracy loss due to quantization in some of the networks.