• State Resolved
  • Locked Locked
  • Replies 7 replies
  • Subscribers 98 subscribers
  • Views 674 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM-Q1: Using edgeai-benchmark imported artifacts on the EVM board

James Graves1
Prodigy 170 points
Part Number: TDA4VM-Q1
Other Parts Discussed in Thread: SK-TDA4VM

Hello All,

According to the edgeai-benchmark GitHub page, imported artifacts of a model can be used to perform inference on the J721E EVM. However, we have not been able to run inference on any models (such as efficientdet_lite0-3) on the J721E EVM board using artifacts from edgeai-benchmark. We can do so on the SK-TDA4VM board, but not on the J721E EVM board. We are able to get working artifacts on the J721E EVM by importing the models using the TIDL tools included with the PROCESSOR RTOS SDK v8.2.

Are edgeai-benchmark's imported artifacts supposed to work on the J721E EVM board without modification? If not, what modifications are required? We are hoping to use a single model compilation tool for both the J721E EVM and the SK-TDA4VM.

Thank you in advance.

  • 0
    TI__Guru 50423 points

    Hello James,

    There are some differences in the runtime being used on PSDK RTOS for J721E EVM and Edge AI SDK for SK-TDA4VM (ie PSDK RTOS makes use of TIDL-RT and Edge AI SDK uses OSRT such as ONNX, Tensorflow Lite, TVM/Neo). 

    Could I get back to you next week on whether the models in the edgeai-benchmark can be used with PSDK RTOS? 

    Regards,

    Takuma

  • 0 in reply to Takuma Fujiwara
    Prodigy 170 points

    Yes, we would be glad to hear an update on how to use the models with the PSDK RTOS release.

    Thank you.

  • 0 in reply to Takuma Fujiwara
    Prodigy 170 points

    From what we have seen so far, it seems like the artifacts generated by the edgeai-benchmark are not compatible with the code which runs them in the PSDK RTOS DL applications. (We have primarily been looking at object detection so far.)

    Also, we have noticed that with the PSDK RTOS, we had to manually specify a bunch of settings, whereas with the edgeai-benchmark, it had a bunch of presets for these instead.  Which were apparently set for running on the SK-TDA4VM EdgeAI board?  We do see how some variables can be changed in the pipeline settings though.

    So if there is a roadmap for supporting both use cases (EVM and Edge-AI) we would like more information.

    Thanks,

    James

  • 0 in reply to James Graves1
    TI__Guru 50423 points

    Hi James,

    Apologies for the delay. I have gotten response from the engineer that has been working on creating these tools. 

    Models created with edgeai-benchmark can be used with PSDK RTOS. However, the runtime used in PSDK RTOS, TIDL-RT, can only use fully supported models (it cannot offload to ARM). If the model is compiled using edgeai-tidl-tools or edgeai-benchmark and all the layers are offloaded to hardware accelerators and DSP cores, then the artifacts can be used by TIDL-RT as well.

    Regards,

    Takuma 

  • 0 in reply to Takuma Fujiwara
    Prodigy 170 points

    Thank you for the response.

    We have found ways to disable offloading to ARM for the TVM and ONNX runtimes but could not find a way to disable it for the TFLite runtime.

    Could you please specify what changes are needed to disable offloading to ARM for the TFLite runtime?

  • +1 in reply to James Graves1
    TI__Guru 50423 points

    Hi James,

    There are options for disabling offload (to DSP/MMA)=ARM only mode=all the model runs on ARM instead of partly offloading to DSP/MMA. I do not think there was an option to disable offloading to ARM for TVM or ONNX.

    What I got from the team is that there are no model importing tools that were intentionally designed to work for both Edge AI SDK and PSDK RTOS. However, it is possible for models created with edgeai-tidl-tools/edgeai-benchmark from Edge AI SDK to also run with TIDL-RT used in PSDK RTOS, as long as the model only use layers and operations supported by TIDL-RT: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_00_00_12/exports/docs/tidl_j7_08_00_00_10/ti_dl/docs/user_guide_html/md_tidl_layers_info.html 

    So, some possible choices are:

    • Use TIDL Model Importer for PSDK RTOS, and edgeai-tidl-tools for Edge AI SDK - This is the recommended method since these two SDK are meant for different use cases (automotive vs industrial), have different pipeline flows (OpenVX vs GStreamer), and focus on different underlying OS (RTOS vs Linux), so all the tools included with each SDK are not designed with the intent to mix and match
    • When picking models, ensure all layers/operations are supported by TIDL-RT by referencing link above, and compile using edgeai-tidl-tools/edgeai-benchmark - Not recommended, since this differs from the intended application development flow and support from TI will be limited, but possible.

    Regards,

    Takuma

  • 0 in reply to Takuma Fujiwara
    Prodigy 170 points

    Hi Takuma,

    Thank you for the reply.

    We've discussed this internally, and we'll keep using the model importer for the PSDK RTOS, and the edgeai-tidl-tools for the Edge AI SDK. We will eventually consolidate all development on a single platform.

    Best Regards,

    James