This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: TDA4VM-SK per-layer quantization

Part Number: TDA4VM

Hi
How can we define per-layer quantisation (fixed point positions, etc) per layer, for a given model?

Thanks,
 - Yuval

  • For TDA4VM  we currently support QAT models from below repo to support quantization information from model instead using calibration provided by edgai-tidl-tools 

    https://github.com/TexasInstruments/edgeai-torchvision/blob/master/docs/pixel2pixel/Quantization.md

    The ONNX model exported from above will have CLIP operator after every layer to inform the per layer range to  model compilations tools in edgai-tidl-tools, this avoids the needs for calibration images/ iterations

    Along with above, AM62a, 68A, and 69A devices also supports TFlite quantized models which can have per layer quantization information, and the same can be used without need for calibration images/ iterations

     

  • Hi, I have a couple of questions regarding this:
    1. Can we add these CLIP operator ourselves, and not use the TI QAT tools?
    2. Assuming we would add a clip operator after every layer in the model, would the clip operator cost any additional runtime on the board?
    3. Is the clip operator effective after every layer type? or would only specific layer types be affected by it?
    4. Assuming we add these clip operators, how should we change the optional parameters in the conversion process?

    Thanks,
     - Yuval

  • Hi Yual,

    Find my answers 

    1. yes.

    2. No additional runtime on board

    3. Every layer where dat range can change like conv2d, add, mull, avg pool etc

    4.Default options shall be ok. If you observe any bias saturated related issues, please set advanced_options:bias_clipping to 1

  • thanks, setting this as resolved for now