TDA4VM: how improve fps of yolov5?

They are using SDKv8.1 now. this measurement FPS would be improved and matched to official FPS when they update SDK to latest one?

Thanks and Best Regards,

SI.

Hi SI,

(1) Yes, latest SDK is expected to give 46 fps, so it is possible there could be some fixes in later release versions which improved the fps. 

(2) they tried to run yolov5m model of ultralytics on TDA4 SK EVM with TI's optimized Focus and spp layer, but reached only 1.82 fps as below --> 

--> I suspect some of the layers in this converted model are not supported on TIDL. Can you check the runtimes_visualization.svg file generated as part of artifacts folder? That could give a clue on unsupported layers.

One question I have though : Any specific reason to convert the ultralytics models when we have a model fully supported on TIDL provided as part of model zoo, in addition to separate repo with instructions on how to train and export it?

Regards,

Anand

Can you gather more information on the setup they are using to get the performance results? Are they using edgeai-tidl-tools, edgeai-benchmark or their own scripts? How many images are being run when above performance is measured?

Regards,

Anand

Hi Anand,

They measured performance results using timer. Could you please let me know how you measured it?

Thanks and Best Regards,

SI.

Ok timer may give lower fps. We use a function to get the actual processing time for inference. You can refer to this function : https://github.com/TexasInstruments/edgeai-tidl-tools/blob/3dc359873f0b80c1e1a0b4eec5d1d02e35d4e532/examples/osrt_python/ort/onnxrt_ep.py#L58

Can you check what performance is obtained using this?  We would be looking at the offload time which is calculated here: https://github.com/TexasInstruments/edgeai-tidl-tools/blob/3dc359873f0b80c1e1a0b4eec5d1d02e35d4e532/examples/osrt_python/ort/onnxrt_ep.py#L226

Regards,

Anand

Hi Anand,

Thanks for your response. I'll let you know when I get the new data.

I also suggested customer to check it again with pre-built image included in SDKv8.4 and update PC tools accordingly.

Have you checked it on pre-built image of SDKv8.4?

Thanks and Best Regards,

SI.

Yes SI, to be precise, I did this experiment with edgeai-benchmark repo(github.com/.../edgeai-benchmark), which can be used to benchmark performance better (edgeai-benchmark uses edgeai-tidl-tools in the backend but has some additional scripts to help ease benchmarking). If customer is looking for benchmarking more models, may be a good idea to use this repo. However, I expect the performance to be similar if you use the above mentioned function.

Regards,

Anand