QAT quantization scheme

Cecilia

Prodigy 10 points

Is there a possibility to apply int16, fp16 instead of int8 at specific layers in QAT?

over 2 years ago

0 Pratik Kedar over 2 years ago

TI__Mastermind 24041 points

Hi,

Could you please share which flow you are using ? OSRT Or TIDL-RT ?

0 Cecilia over 2 years ago

Prodigy 10 points

Hi, I use this QAT tool:

edgeai-modeloptimization/torchmodelopt/edgeai_torchmodelopt/xmodelopt/quantization/v1/docs/qat.md at r9.1 · TexasInstruments/edgeai-modeloptimization · GitHub

0 Pratik Kedar over 2 years ago in reply to Cecilia

TI__Mastermind 24041 points

Hi Cecilia,

From the QAT, we only support 8 bit quantization, the mixed precision quantization support it not available as part of QAT offering.

Though i would recommend you to please check out PQT (Post Quantization Tool) documentation as part of edgeai-tidl-tools repos where you can opt for mixed precision quantization option. Something like all layers are in 8 bit and few(selected ones) in 16 bit or Vice Versa.

Hope this helps you.

Please find the link to PQT here : https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_quantization.md#a-post-training-quantization-ptq

Thank you.

Processors

Processors forum

QAT quantization scheme