SK-AM62A-LP: Custom ONNX model cannot work on AM62A

Hello Takayuki,

It is our pleasure to help. 

I would like some more information before we start looking at the model itself. Did the compiled model work on host PC? How are you trying to run it on the EVM? Are you using edgeai-gst-apps? 

Best regards,

Qutaiba

Hi Takayuki,

To save time, this is a suggested solution if you are using edgeai-gst-apps. Take a look at this thread: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1329198/faq-how-do-i-get-a-valid-param-yaml-file-for-my-tidl-deep-learning-model-the-demos-in-edgeai-gst-apps-throw-errors-on-my-model

Would you please let me know if this solved your problem? Otherwise, please provide more context about the error you are getting when running the model on the EVM. Did the model work on the host pc? What code are you using to run on EVM? What errors are you getting?

Best regards,

Qutaiba

Hi, Qutaiba-san,

Thank you for your reply.

I read the thread you suggested. There are no errors in the part of parsing the yaml file.

I provide more context about the error.

To run the model:

1. write the test code

import numpy as np
from edgeai_dl_inferer import ModelConfig

TIDL_ENABLE = True

model_path = '/home/root/cl-ort-mymodel'
model_obj = ModelConfig(model_path, TIDL_ENABLE, 1)
print("create runtime")
model_obj.create_runtime()
print("create done")

def infer(images):
    print("infer start")
    ort_outs = model_obj.run_time(images)
    print("infer end")
    result = np.array(ort_outs)
    return result

img = np.random.randint(255, size=(1,3,224,224),dtype=np.uint8)
img = img.astype(np.float32)
result = infer(img)

2. run the code

create runtime
libtidl_onnxrt_EP loaded 0x18c42280
Final number of subgraphs created are : 1, - Offloaded Nodes - 133, Total Nodes - 133
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  7848.312205 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  7848.312347 s:  VX_ZONE_INIT:Enabled
  7848.312378 s:  VX_ZONE_ERROR:Enabled
  7848.312388 s:  VX_ZONE_WARNING:Enabled
  7848.313496 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
  7848.313907 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
  7848.314204 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
  7848.314614 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
  7848.314667 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
  7848.315058 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
create done
infer start

Logs are output to the console, but after "infer start", no logs are output and inference does not complete.

If TIDL_ENABLE is False, the inference can be completed, so I don't think there is a problem with the ONNX model itself.

Best regards,

Takayuki

Hi Takayuki-san,

Let's try to get more details from the runtime and tidl so that we can figure the issue. Would you please execute the following:

• Runt this command so that tidl-tool collect more log details: export TIDL_RT_DEBUG=1
• After running the inference, the output log should be in /opt/vx_app_arm_remote_log.out.
• Also edit the following file: /usr/lib/python3.10/site-packages/edgeai_dl_inferer.py. In the class onnxrt, in the dictionary named "runtime_options", add this extra option "debug_level": 2. 

This should show more logs about the inference. Would you please share the logs?

Best regards,

Qutaiba

Hi, Qutaiba-san,

Thank you for your suggestion.

I have attached the vx_app_arm_remote_log.out file.

5706.vx_app_arm_remote_log.zip

I also tried to add the option "debug_level": 2 and got the following:

create runtime
libtidl_onnxrt_EP loaded 0x40ab35a0
artifacts_folder                                = /home/root/cl-ort-mymodel/artifacts
debug_level                                     = 2
target_priority                                 = 0
max_pre_empt_delay                              = 340282346638528859811704183484516925440.000000
Final number of subgraphs created are : 1, - Offloaded Nodes - 133, Total Nodes - 133
In TIDL_createStateInfer
Compute on node : TIDLExecutionProvider_TIDL_0_0
************ in TIDL_subgraphRtCreate ************
 TIDL_RT_OVX: Set default TIDLRT params done
Calling appInit() in TIDL-RT!
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   326.436828 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   326.440892 s:  VX_ZONE_INIT:Enabled
   326.440928 s:  VX_ZONE_ERROR:Enabled
   326.440937 s:  VX_ZONE_WARNING:Enabled
   326.442140 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
   326.442377 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
   326.442527 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
   326.442667 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
   326.442681 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
   326.443501 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
TIDL_RT_OVX: Init ...
TIDL_RT_OVX: Mapping config file ...
TIDL_RT_OVX: Mapping config file ... Done. 93976 bytes
TIDL_RT_OVX: Tensors, input = 1, output = 1
Host kernel - 0xffff8ce24f68
TIDL_RT_OVX: Mapping network file
TIDL_RT_OVX: Mapping network file... Done 26481652 bytes
TIDL_RT_OVX: Init done.
TIDL_RT_OVX: Creating graph ...
TIDL_RT_OVX: input_sizes[0] = 896, dim = 224 padL = 0 padR = 0
TIDL_RT_OVX: input_sizes[1] = 200704, dim = 224 padT = 0 padB = 0
TIDL_RT_OVX: input_sizes[2] = 3, dim = 3
TIDL_RT_OVX: input_sizes[3] = 1, dim = 1
TIDL_RT_OVX: input_buffer = 0xffff9c0f7000 150528
TIDL_RT_OVX: Creating graph ... Done.
TIDL_RT_OVX: Verifying TIDL graph ... Done.
************ TIDL_subgraphRtCreate done ************
create done
infer start
 *******   In TIDL_subgraphRtInvoke  ********
TIDL_RT_OVX: Set default TIDLRT tensor done
TIDL_RT_OVX: Set default TIDLRT tensor done
TIDL_RT_OVX: Running Graph ...
TIDL_RT_OVX: input_sizes[0] = 896, dim = 224 padL = 0 padR = 0
TIDL_RT_OVX: input_sizes[1] = 200704, dim = 224 padT = 0 padB = 0
TIDL_RT_OVX: input_sizes[2] = 3, dim = 3
TIDL_RT_OVX: input_sizes[3] = 1, dim = 1
TIDL_RT_OVX : Memcpy Input Buffer
TIDL_RT_OVX: input_buffer = 0xffff9c0f7000 150528
TIDL_RT_OVX: memset_out_tensor_tidlrt_tiovx  ... Done.

It appears to hang after memset_out_tensor_tidlrt_tiovx completes.

Best Regards,

Takayuki

Hi Takayuki-san,

Thank you for providing the data. I miss-informed you about /opt/vx_app_arm_remote_log.out, sorry. Actually, you have to run this in the background and then start running the inference. The logs then will be printed on the terminal. 

Another suggestion is to run the model on host emulation. You can do that using tidl-tools. Just make sure that the TIDL_TOOLS_PATH and LD_LIBRARY_PATH variables are set up correctly as in this page: https://github.com/TexasInstruments/edgeai-tidl-tools . This way we make sure that the compiled model run as expected before trying it on the target. 

If this logs do not provide enough information to figure out the issue, I suggest looking at the compilation log. Please, run the compilation of the model with debug_level 2 and share the logs with us. If you think that this log contains information about your model which you don't want to share publicly, you can sent the it to me in a private e2e message. 

Best regards,

Qutaiba