• State
  • Locked Locked
  • Replies 5 replies
  • Subscribers 98 subscribers
  • Views 491 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: compiling custom ONNX model and run inference on a TI Edge AI processor for evaluation

Mohammad Gudarzi
Prodigy 20 points
Part Number: TDA4VM


Hi,

I am trying to compile an ONNX custom model on the TI Edge AI Cloud. I can run it on the CPU and get the outputs or I can put all layers to the deny_list of the compile_options and compile it and run the whole model on the CPU.
But, while I want to compile the model run the whole or part of it on the accelerator the kernel dies during the calibration and no issue is logged in the custon-model-onnx_out.log.

Given that it has been mentioned that if an ONNX model could be run on CPU, by excluding some layers not supported by the accelerator, the rest could be run on the accelerator, how this problem can be resolved?

Thanks,

  • 0
    TI__Genius 9065 points

    Hi,

    Can you set debug_level to 1 as part of compile options and share the log?

    Regards,

    Anand

  • 0 in reply to Anand Pathak
    Prodigy 20 points

    Hi,
    I have attached the log with the debug_level = 1,
    Thanks

    custon-model-onnx_out.log

  • 0 in reply to Mohammad Gudarzi
    TI__Genius 9065 points

    Hi,

    Can you set the deny_list as follows and check:

    "deny_list":"Gather, Resize, Reshape, Concat, Sigmoid, Softmax, Transpose"
     
    There are some warning related to visualization (visualization in svg may not be correct), but they won't impact the actual compilation of the model. 
    Regards,
    Anand
  • 0 in reply to Anand Pathak
    Prodigy 20 points

    Hi,
    Thanks for the response.
    This config resolves the problem of the compilation and running.
    But, I still have two concerns:
    1. Inference of this compiled model mostly happens on cpu which makes it slow. Is there anyway to make it mostly run on the accelerator like the default example which is a classifier?
    2. Even with current model architecture, if I increase the input size (right now it s 128x128, increasing to 256x256) the kernel would die during the compilation.

    I am wondering if there is anyway to resolve these issues?

    Thanks

  • 0 in reply to Mohammad Gudarzi
    TI__Genius 9065 points

    Hi,

    1. Some hints : 

    - I see that most of the subgraphs are created due to Gather-Conv-Mul combination, is there a way to modify network to have TIDL supported layers here instead of these?

    - Resize layer - if you can export the model to have scales in the resize layer instead of sizes (e.g can export using scales = output_size/input_size), the resize layer would be supported on dsp and will result in fewer subgraphs to ARM

    2. Can you please share the log here again with debug_level = 1?

    Regards,

    Anand