TDA4VM: TDA4VM-SK per-layer quantization

Dagan Levy

Prodigy 40 points

Part Number: TDA4VM

Hi
How can we define per-layer quantisation (fixed point positions, etc) per layer, for a given model?

Thanks,
- Yuval

over 2 years ago

0 kumar.desappan over 2 years ago

TI__Mastermind 22145 points

For TDA4VM we currently support QAT models from below repo to support quantization information from model instead using calibration provided by edgai-tidl-tools

https://github.com/TexasInstruments/edgeai-torchvision/blob/master/docs/pixel2pixel/Quantization.md

The ONNX model exported from above will have CLIP operator after every layer to inform the per layer range to model compilations tools in edgai-tidl-tools, this avoids the needs for calibration images/ iterations

Along with above, AM62a, 68A, and 69A devices also supports TFlite quantized models which can have per layer quantization information, and the same can be used without need for calibration images/ iterations

0 Dagan Levy over 2 years ago in reply to kumar.desappan

Prodigy 40 points

Hi, I have a couple of questions regarding this:
1. Can we add these CLIP operator ourselves, and not use the TI QAT tools?
2. Assuming we would add a clip operator after every layer in the model, would the clip operator cost any additional runtime on the board?
3. Is the clip operator effective after every layer type? or would only specific layer types be affected by it?
4. Assuming we add these clip operators, how should we change the optional parameters in the conversion process?

Thanks,
- Yuval

0 kumar.desappan over 2 years ago in reply to Dagan Levy

TI__Mastermind 22145 points

Hi Yual,

Find my answers

1. yes.

2. No additional runtime on board

3. Every layer where dat range can change like conv2d, add, mull, avg pool etc

4.Default options shall be ok. If you observe any bias saturated related issues, please set advanced_options:bias_clipping to 1

0 Dagan Levy over 2 years ago in reply to kumar.desappan

Prodigy 40 points

thanks, setting this as resolved for now

Processors

Processors forum

TDA4VM: TDA4VM-SK per-layer quantization