Hi team,
Here's an issue from the customer may need your help:
When deploying a pretrained model yolov5M6_640_ti_lite_44p1_62p9.onnx to a TDA4 EVM (8bits quantization), accuracy tested based on coco dataset val2017, conf 0.001, NMS 0.65, AP[0.5:0.95] result on EVM is 0.284. The accuracy of 0.441 is significantly different from the floating point model in the TI release page. TIDL tried 16bits of quantization, and the result on the EVM was 0.311, and the accuracy loss was still significant.
1) Has TI been tested for accuracy on the TDA4 EVM based on this pretrained yolov5m model?
2) Could you help check the configuration of the import if there are any parameters that need to be improved? Or any recommended configuration for this model?
TI_device_a72_test_dl_algo_host_RT.out is used for handle val2017, and the import configuration file is as follows:
modelType = 2
numParamBits = 8
numFeatureBits = 8
#quantizationStyle = 3
quantizationStyle = 2
inputNetFile = "../../test/testvecs/models/public/onnx/yolov5m6_640_ti_lite_44p1_62p9.onnx"
outputNetFile = "../../test/testvecs/config/tidl_models/onnx/tidl_net_yolov5m6_640_ti_lite_44p1_62p9.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/onnx/tidl_io_yolov5m6_640_ti_lite_44p1_62p9_"
inDataNorm = 1
inMean = 0 0 0
inScale = 0.003921568627 0.003921568627 0.003921568627
inDataFormat = 1
inWidth = 640
inHeight = 640
inNumChannels = 3
numFrames = 1
inData = "../../test/testvecs/config/detection_list.txt"
perfSimConfig = ../../test/testvecs/config/import/device_config.cfg
inElementType = 0
metaArchType = 4
metaLayersNamesList = "../../test/testvecs/config/import/public/onnx/yolov5m6_640_ti_lite_metaarch.prototxt"
postProcType = 2
foldPreBnConv2D = 0
#calibrationOption = 7
For prototxt file, only the confidence_threshold: 0.001 has been modified regarding the release configuration. And the infer configuration file is as follows:
inFileFormat = 2
numFrames = 5000
netBinFile = "testvecs/config/tidl_models/onnx/tidl_net_yolov5m6_640_ti_lite_44p1_62p9.bin"
ioConfigFile = "testvecs/config/tidl_models/onnx/tidl_io_yolov5m6_640_ti_lite_44p1_62p9_1.bin"
inData = testvecs/config/name_list_acc.txt
outData = testvecs/output/tidl_yolov5m6_640_ti_lite_44p1_62p9_od.bin
inResizeMode = 0
debugTraceLevel = 0
writeTraceLevel = 0
postProcType = 2
Could you help check this case? Thanks.
Best Regards,
Cherry
