TDA4VM: maxpool kernel and difference between TI-RT and OSRT

Hi SI,

There seems to be some gap in understanding of the OSRT offering vs TIDL-RT offering. Let me clear that first.

TIDL-RT is TIDL custom runtime which is optimized for TDA4. It is optimized for performance on TI platforms, however it can support only the layers whose implementation is supported in TIDL. OSRT builds on top of TIDL-RT to provide ease of use to customers in terms of using open source interfaces like tflite runtime, ONNX runtime, TVM/Neo-AI runtime and also provides better coverage for networks since it supports heterogeneous execution (TIDL supported layers on TI accelerator and TIDL unsupported layers on ARM).

You can refer to this documentation for further understanding: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_01_00_13/exports/docs/tidl_j7_08_01_00_05/ti_dl/docs/user_guide_html/index.html

OSRT related examples can be found in this github repo, it also has further documentation regarding OSRTs : https://github.com/TexasInstruments/edgeai-tidl-tools

Coming to your questions:

1. The errors are getting reported since the kernel size and stride combinations are not supported by TIDL-RT. Refer following for supported layer configurations in TIDL-RT : https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_01_00_13/exports/docs/tidl_j7_08_01_00_05/ti_dl/docs/user_guide_html/md_tidl_layers_info.html

If the model is run using OSRT, the network should work fine with these layers getting offloaded to ARM. However, it will come at some performance cost. For Yolov5, we support TI variant of the model which replaces these max pool layers with TIDL-RT supported configurations as mentioned in above doc. Further explanation for the same can be found here : https://github.com/TexasInstruments/edgeai-yolov5#yolov5-ti-lite-model-definition . This enables entire model to be delegated to TI accelerator giving better performance, and now the model can be run successfully using TIDL-RT as well.

2. As explained above, TIDL-RT is optimized for TDA4, so if all layers in network are supported by TIDL, the TIDL-RT vs OSRT performance will be similar since all layers are executing on TI accelerator. However, if some layers are not supported, the model will not run on TIDL-RT, but will run on OSRT with unsupported layers running on ARM.

Let me know if you have any further questions.

Regards,

Anand