AM62A3: Switching CNN at runtime

Hi Geoffrey,

Great questions.

Geoffrey Ficara said:

Is it possible to switch from 1 CNN to another one in run-time.

- Yes, multiple CNNs can be loaded at a time. These can be setup using typical runtimes like tensorflow-lite or onnxruntime. We have a gstreamer example that uses multple models on the same input: https://github.com/TexasInstruments/edgeai-gst-apps/blob/main/configs/single_input_multi_infer.yaml

Geoffrey Ficara said:

Is it possible to implement complex non-linear logic, ie switching from CNN number 1 to 2, then 3, then 4, then 1, then 4, then 2, then 1 …  ?

- I believe this will be fine. We are able to have multiple models setup and a time and called in an ad hoc fashion. There should not be any restrictions on the order in which these are run.

Geoffrey Ficara said:

Do we have a profiler tool for run time operation ?

- Good question. We have profiling tools to get more information at compile time, runtime, and on host emulation. For host emulation, devs can run a bit-accurate model of the accelerator on a PC. I'm posting a few sections of a troubleshooting guide below that applies to accuracy and performance. The gist is that you set a debug_level option when setting up the model for run time:

- On Target: https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_osr_debug.md#troubleshooting-for-performance-issues

- On Host: https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_osr_debug.md#steps-to-debug-functional-mismatch-in-host-emulation

Geoffrey Ficara said:

Can you run YoloX ?

- Absolutely. We have several variants of YOLOX available that have already been optimized for full offload to the accelerator. There are several variants to help tradeoff accuracy and performance. Variants nano, tiny, s (small), and m (medium) all run on AM62A and range from 120 to ~7 fps, respectively.

- The model analyzer tool (https://dev.ti.com/edgeaisession/) shows benchmarks for the models that we already have supported -- custom models are possible too.

Best,
Reese