Other Parts Discussed in Thread: TDA4VH
Tool/software:
Hello Team,
We have done model compilation and when trying to do model inference on TDA4VH platform we are getting the following errors
root@j784s4-evm:/opt/edgeai-tidl-tools# ./bin/Release/ort_main -f model-artifacts/cl-ort-resnet18-v1/artifacts -i test_data/airshow.jpg -a 1
***** Display run Config: start *****
verbose level set to: 3
accelerated mode set to: 1
device mem set to: 1
loop count set to: 1
model path set to:
model artifacts path set to: model-artifacts/cl-ort-resnet18-v1/artifacts
image path set to: test_data/airshow.jpg
device_type set to: cpu
labels path set to: test_data/labels.txt
num of threads set to: 4
num of results set to: 5
num of warmup runs set to: 2
***** Display run Config: end *****
libtidl_onnxrt_EP loaded 0x3ea086f0
terminate called after throwing an instance of 'Ort::Exception'
what(): /root/onnxruntime/onnxruntime/core/providers/tidl/tidl_execution_provider.cc:94 onnxruntime ::TidlExecutionProvider::TidlExecutionProvider(const onnxruntime::TidlExecutionProviderInfo&) status = = true was false.
Aborted (core dumped)
We tried with python as well which generated the following error
root@j784s4-evm:/opt/edgeai-tidl-tools/examples/osrt_python/ort# python3 onnxrt_ep.py
Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
Running 1 Models - ['cl-ort-resnet18-v1_low_latency']
Running_Model : cl-ort-resnet18-v1_low_latency
libtidl_onnxrt_EP loaded 0x332bea60
Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52
APP: Init ... !!!
4194.801576 s: MEM: Init ... !!!
4194.801625 s: MEM: Initialized DMA HEAP (fd=5) !!!
4194.801765 s: MEM: Init ... Done !!!
4194.801785 s: IPC: Init ... !!!
4194.833762 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
4194.841211 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
4194.841307 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
4194.841317 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
4194.841324 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
4194.841883 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0
4194.842005 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1
4194.842104 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2
4194.842223 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3
4194.842239 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
4194.842256 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
4194.865933 s: VX_ZONE_ERROR: [ownContextSendCmd:1001] Command ack message returned failure cmd_status: -1
4194.865979 s: VX_ZONE_ERROR: [ownNodeKernelInit:704] Target kernel, TIVX_CMD_NODE_CREATE failed for node node_136
4194.865988 s: VX_ZONE_ERROR: [ownNodeKernelInit:705] Please be sure the target callbacks have been registered for this core
4194.866003 s: VX_ZONE_ERROR: [ownNodeKernelInit:706] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
4194.866016 s: VX_ZONE_ERROR: [ownGraphNodeKernelInit:793] kernel init for node 0, kernel com.ti.tidl:1:1 ... failed !!!
4194.866056 s: VX_ZONE_ERROR: [ graph_116 ] Node kernel init failed
4194.866066 s: VX_ZONE_ERROR: [ graph_116 ] Graph verify failed
4194.866196 s: VX_ZONE_ERROR: [ownContextSendCmd:1001] Command ack message returned failure cmd_status: -1
4194.866215 s: VX_ZONE_ERROR: [ownNodeKernelInit:704] Target kernel, TIVX_CMD_NODE_CREATE failed for node node_83
4194.866226 s: VX_ZONE_ERROR: [ownNodeKernelInit:705] Please be sure the target callbacks have been registered for this core
4194.866236 s: VX_ZONE_ERROR: [ownNodeKernelInit:706] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
4194.866248 s: VX_ZONE_ERROR: [ownGraphNodeKernelInit:793] kernel init for node 0, kernel com.ti.tidl:1:1 ... failed !!!
4194.866261 s: VX_ZONE_ERROR: [ TIDL subgraph 191 ] Node kernel init failed
4194.866270 s: VX_ZONE_ERROR: [ TIDL subgraph 191 ] Graph verify failed
Traceback (most recent call last):
File "/opt/edgeai-tidl-tools/examples/osrt_python/ort/onnxrt_ep.py", line 602, in <module>
run_model(model, mIdx)
File "/opt/edgeai-tidl-tools/examples/osrt_python/ort/onnxrt_ep.py", line 366, in run_model
sess = rt.InferenceSession(
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 387, in __init__
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 439, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Create state function failed. Return value:-1
How to run the model using TIDLExecutionProvider?
Best Regards
Reshma
