• State
  • Locked Locked
  • Replies 1 reply
  • Subscribers 99 subscribers
  • Views 430 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

J721EXCPXEVM: 8-bit quantization error

user6521395
Prodigy 180 points

Part Number: J721EXCPXEVM

Hi:

     My model is a key point detection model (using the sigmoid activation function at the end). I used the QAT tool provided by Ti to quantize my model to 16 bits. The accuracy of the 16bit quantized model did           not decrease, which is great.
      In order to make the model inference faster on TDA4, QAT is used to quantify the model into 8-bit, but the accuracy of the 8-bit model is reduced by about 10% compared to the float model, which is                     unacceptable.
     I have taken the suggestions in "ti_dl/docs/user_guide_html/md_tidl_fsg_quantization.html".
      How can the accuracy of the 8-bit model not decrease?

      The sdk version is 7.2.

Best regards,

Gong

  • 0
    TI__Guru 56820 points

    Hi Gong,

        Can you confirm that you are using following parameters in your import config file as mentioned in the following documentation:

    https://git.ti.com/cgit/jacinto-ai/pytorch-jacinto-ai-devkit/about/docs/Quantization.md

    If you are using TIDL to infer a model trained using QAT (or Calibrated model using the PTQ Calibration that is simulated here) tools provided in this repository, please set the following in the import config file of TIDL for best accuracy:
    quantizationStyle = 3 to use power of 2 quantization.
    foldPreBnConv2D = 0 to avoid a slight accuracy degradation due to incorrect folding of BatchNormalization that comes before Convolution (input mean/scale is implemented in TIDL as a PreBN - so this affects most networks).
    calibrationOption = 0 to avoid further Calibration in TIDL.

    Regards,

    Anshu