TIDL QAT Model Compilation

The details of QAT is here: https://github.com/TexasInstruments/edgeai-torchvision#guidelines-for-model-training--quantization

This page gives high level overview of all the tools: https://github.com/TexasInstruments/edgeai

Hi Manu,

Thank you for the quick reply.

I have gone through the link mentioned above, but I want to know which ONNX file to use while importing, the one with clips?

If so, how does it convert fp32 weights to int8 as we are not giving any information related to scale and zero_point values in the ONNX files?

Thanks

The QAT model is the one with clips. The clip range is sufficient for TIDL to derive the quantization parameters. 

When importing using TIDL onnxruntime/tflite_runtime/tvm, accuracy_level should be set to 1 and advanced_options:quantization_scale_type should be set to 1

github.com/.../config_settings.py

'tensor_bits': 8,
'accuracy_level': 1,
# 0 (non-power of 2, default), 1 (power of 2, might be helpful sometimes, needed for qat models)
'advanced_options:quantization_scale_type': 1,

Hi Manu,

I am not using open-source runtimes, but trying to port it on the platform.

While compiling, I am using the below settings in the config file for the QAT model with clip nodes-

quantizationStyle = 3
calibrationOption = 64 

but I am facing floating point exception while compilation process. Can you please help with this?

Thanks