• State
  • Locked Locked
  • Replies 1 reply
  • Subscribers 100 subscribers
  • Views 278 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: Edge AI Cloud - Not able to get accuracy benchmark for quantized models

Kruti Yadav
Prodigy 20 points
Part Number: TDA4VM


Hello,

I'm trying to run different quantized models on TI Edge AI Cloud. I'm able to get Performance benchmarking results (FPS, Latency, DDR BW, GMAC). But not able to get the accuracy.

For Inception v1 Quant model, It's correctly predicting the output.

For Inception v3 Quant model, It's predicting wrong output.

For Mobilenet v2 Quant & other models also, It's misclassifying.

Runtime: TFlite
Models: Available quant models from Edge AI's model-zoo & from Tensorflow's official hosted model's page

Can anyone help me to get the accuracy evaluation for quant models?

  • 0
    TI__Mastermind 40580 points

    Hi, to understand better. You tried TFL-CL0-0038-InceptionNetV1 (from dropdown box) and it worked OK. This model corresponds to: inception_v1_224_quant.tflite

    But, when you tried InceptionNetV3 quant and MobileNetV2 quant from TF model's page classification is incorrect? Is my understanding correct?.

    If so, how are you compiling those models? are you using custom-model-tfl.ipynb as a baseline notebook?... again if so, then you could increase number of calib_images (currently we are using only 4 in our notebook, maybe to 20 or 50 if possible) and also you could increase "accuracy_level" to 1 and "advanced_options:calibration_iterations" to 10 or more.

    thank you,

    Paula