• State Resolved
  • Locked Locked
  • Replies 3 replies
  • Subscribers 98 subscribers
  • Views 751 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: import quantized model from tflite to TIDL

YouJun Shih
Prodigy 40 points
Part Number: TDA4VM

Hi TI members:

I use tensorflow to do quantization aware training and convert it to tflite model with int8 format.  And I want to import the tflite model to TIDL execution.

Version info:

    TIDL: tidl_j7_02_00_00_07

I have two questions:

1. Can I use 16 bit to quantize bias in TIDL_importQuantLayerParams() and use 16 bit to do TIDL_conv2dBiasSplit() ?

    tflite use int32 to store bias of convolution layer.

    In phase of TIDL importer, TIDL_importQuantLayerParams() use 8 bit to quantize bias (but use int16 to store it?) and it cause some precision loss.

    In phase of TIDL inference, TIDL_conv2DInit() use int8 bias to do TIDL_conv2dBiasSplit(). But ref implementation of TIDL_refConv2dProcess() use int16 biasPtr to do TIDL_conv2dBiasSplit().

    Does TIDL_conv2dMMAProcess() also use int16 bias to do TIDL_conv2dBiasSplit()?   (I can not find implementation code of TIDL_conv2dMMAProcess)

    Can I use 16 bit to quantize bias in TIDL_importQuantLayerParams() and use 16 bit to do TIDL_conv2dBiasSplit() when run on EVM?

2. How can I enable per-channel weights quantization?

    tflite use per-channel weights quantization by default. TIDL support per-channel weights quantization only the case: (numGroups == numInChannels == numOutChannels).

    I try to modify some codes both of TIDL importer and TIDL inference modules, and get good results when run PC_dsp_test_dl_algo.out.

    I use the generated net.bin and params.bin to run the model on EVM, the results is bad. I think that it's because of the conv kernel do not support per-channel weights quantization.

    How can I enable per-channel weights quantization of conv kernel?

Thanks.

YouJun Shih

  • 0
    TI__Guru 56820 points

    Hi YouJin Shih,

    YouJun Shih said:

        Does TIDL_conv2dMMAProcess() also use int16 bias to do TIDL_conv2dBiasSplit()?   (I can not find implementation code of TIDL_conv2dMMAProcess)

    YouJun Shih said:
    Can I use 16 bit to quantize bias in TIDL_importQuantLayerParams() and use 16 bit to do TIDL_conv2dBiasSplit() when run on EVM?

         For 8-bit quantization TIDL uses 16-bit bias. Bias Split function will split the 16--bit bias into two 8-bit quantity.

    YouJun Shih said:
    How can I enable per-channel weights quantization?

    TIDL for TDA4VM supports per-channel quantization only weights for depth wise convolution layers and it has to be used along with power of 2 quantization ( quantizationStyle=3).

    Have you tried calibrationOption=7 to see if your accuracy improves , you can find details of this option in section A.2.2. Advanced Bias calibration in following documentation : 

    https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_02_00_05/exports/docs/tidl_j721e_08_02_00_11/ti_dl/docs/user_guide_html/md_tidl_fsg_quantization.html#did_tidl_quantization_1


    Regards,

    Anshu

    : 

  • 0 in reply to Anshu Jain
    Prodigy 40 points

    Hi Anshu, thanks for your reply.

    I known that TIDL for TDA4VM supports per-channel quantization only weights for depth wise convolution layers.

    My questions are:

    1. Does TIDL support per-channel weights quantization of convolution only meet the case: (numGroups == numInChannels == numOutChannels)?

    For general depthwise convolution, the numInChannels and numOutChannels may be different.

    2. Does convolution kernel support per-channel weights quantization?

    If the answer is yes, how can I enable per-channel weights quantization of conv kernel? Can you provide any suggestions?

    If the answer is no, does this feature will be supported in future versions?

  • +1 in reply to YouJun Shih
    TI__Guru 56820 points
    YouJun Shih said:

    1. Does TIDL support per-channel weights quantization of convolution only meet the case: (numGroups == numInChannels == numOutChannels)?

    For general depthwise convolution, the numInChannels and numOutChannels may be different.

    It supports only for numGroups == numInChannels == numOutChannels

    YouJun Shih said:
    If the answer is no, does this feature will be supported in future versions?

        Future devices which will contain MMAv2 will support per channel quantization but for TDA4VM this will not be supported.

    Regards,

    Anshu