• State Resolved
  • Locked Locked
  • Replies 2 replies
  • Subscribers 98 subscribers
  • Views 716 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA2EXEVE : validate the performace between TIDL and caffe-jacinto

Ahan Tseng
Prodigy 190 points

Hi,


I have a semantic segmentation model to validate the performance between the caffe-jacinto and TIDL.
And the test databse consists of the day, night, city and suburb scenes.
I tried different settings on TIDL, including the numParamBits, createParams.quantHistoryParam1, createParams.quantHistoryParam2 and createParams.quantMargin.
Then, I got a setting can be nearest to the caffe-jacinto results. (TIDL/Caffe-Jacinto accuracy : 80.8%/85.7%)
numParamBits = 10
createParams.quantHistoryParam1 = 10
createParams.quantHistoryParam2 = 10
createParams.quantMargin = 20

My question is
(1) how to set the parameters on TIDL to make TIDL and caffe-jacinto results be the same? (any cue)
(2) when I use another model to validate the results between TIDL and caffe-jacinto on the same settings, I can't get good result.
But, using another settings can get a better result.
numParamBits = 8
createParams.quantHistoryParam1 = 0
createParams.quantHistoryParam2 = 0
createParams.quantMargin = 0
(TIDL_10bits/TIDL_8bits/caffe-jacinto accuracy : 80.2%/82.2%/86.1%)
How could I solve the question to get a better results?

Best Regards,
Ahan Tseng

  • 0
    TI__Mastermind 22145 points

    As a first step, measure the accuracy using  Quantisation stats collection tool.

    Choose right number of bits for getting accuracy closer to Caffe Jacinto. Start with 8 bits and try increasing up to 12 bits. We expect to get better accuracy at Higher numbers of bits. The history and margin parameters are not applicable to this step.

    Now during inference set both history parameters with 10 (zero is not a reasonable parameter) and Set margin to 10. If the accuracy is not closer to the one observed during first step, start increasing margin and try measuring accuracy again.

    If the accuracy measured in the first step itself is much lesser than the Cafe Jacinto then please share the model. We will analyse the model & root cause the reason for higher quantisation loss and we get back to you.

    Thanks and Regards

    Kumar D