Part Number: PROCESSOR-SDK-AM68A
Tool/software:
Hi,
I am trying to generate model artifacts for an ONNX object detection model using a script located in the TIDL tools x86 container at examples/osrt_python/ort/onnxrt_ep.py. However, I get the following logs and error when I try to run the script after setting my model path in the dir -
root@fc5a52ad3ba2:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py -c
Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
Running 1 Models - ['resnet18_rcnn_azureml']
Running_Model : resnet18_rcnn_azureml
Running shape inference on model ../../../models/public/resnet18_rcnn_azureml.onnx
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] WARNING: 'meta_layers_names_list' is not provided - running OD post processing in ARM mode
Number of OD backbone nodes = 0
Size of odBackboneNodeIds = 0
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 1 | 1 |
| CPU | 11 | x |
-------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------
| Node | Node Name | Reason |
------------------------------------------------------------------------------------------------------------------------------------------
| Unsqueeze | Unsqueeze_1217 | Layer type not supported by TIDL |
| Unsqueeze | Unsqueeze_1218 | Layer type not supported by TIDL |
| ReduceMax | ReduceMax_1209 | Reducing in all dimensions is not supported in TIDL-RT |
| Cast | Cast_1210 | Only supported at the terminal nodes (Input/Output) of the network |
| Mul | Mul_1213 | The variable inputs in Add/Mul/Sub/Div layer must of be same dimensions or broadcast-able |
| Unsqueeze | Unsqueeze_1214 | Layer type not supported by TIDL |
| Add | Add_1215 | The variable inputs in Add/Mul/Sub/Div layer must of be same dimensions or broadcast-able |
| Unsqueeze | Unsqueeze_1216 | Layer type not supported by TIDL |
| NonMaxSuppression | NonMaxSuppression_1221 | Layer type not supported by TIDL |
| Gather | Gather_1223 | Only line gather is supported |
------------------------------------------------------------------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------
| Node | Node Name | Reason |
-----------------------------------------------------------------------------------------
| Identity | /roi_heads/box_roi_pool/Identity | Not supported as an individual operator |
-----------------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
| Node | Node Name | Reason |
-------------------------------------------------------------------------------------------
| Identity | /roi_heads/box_roi_pool/Identity_1 | Not supported as an individual operator |
-------------------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
| Node | Node Name | Reason |
-------------------------------------------------------------------------------------------
| Identity | /roi_heads/box_roi_pool/Identity_2 | Not supported as an individual operator |
-------------------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
| Node | Node Name | Reason |
-------------------------------------------------------------------------------------------
| Identity | /roi_heads/box_roi_pool/Identity_3 | Not supported as an individual operator |
-------------------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 0 | 0 |
| CPU | 1 | x |
-------------------------------------------------------------------------------
----------------------------------------------------------------------
| Node | Node Name | Reason |
----------------------------------------------------------------------
| Identity | Identity_1707 | Not supported as an individual operator |
----------------------------------------------------------------------
============================= [Parsing Completed] =============================
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_06_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 11 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] WARNING: 'meta_layers_names_list' is not provided - running OD post processing in ARM mode
Number of OD backbone nodes = 0
Size of odBackboneNodeIds = 0
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 1 | 1 |
| CPU | 14 | x |
-------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------
| Node | Node Name | Reason |
------------------------------------------------------------------------------------------------------------------------------------------
| Unsqueeze | Unsqueeze_1693 | Layer type not supported by TIDL |
| Unsqueeze | Unsqueeze_1694 | Layer type not supported by TIDL |
| ReduceMax | ReduceMax_1685 | Reducing in all dimensions is not supported in TIDL-RT |
| Cast | Cast_1686 | Only supported at the terminal nodes (Input/Output) of the network |
| Mul | Mul_1689 | The variable inputs in Add/Mul/Sub/Div layer must of be same dimensions or broadcast-able |
| Unsqueeze | Unsqueeze_1690 | Layer type not supported by TIDL |
| Add | Add_1691 | The variable inputs in Add/Mul/Sub/Div layer must of be same dimensions or broadcast-able |
| Unsqueeze | Unsqueeze_1692 | Layer type not supported by TIDL |
| NonMaxSuppression | NonMaxSuppression_1697 | Layer type not supported by TIDL |
| Gather | Gather_1699 | Only line gather is supported |
| Shape | Shape_1701 | Layer type not supported by TIDL |
| Gather | Gather_1702 | Input dimensions must be greater than 1D |
| Equal | Equal_1704 | Layer type not supported by TIDL |
| If | If_1705 | Layer type not supported by TIDL |
------------------------------------------------------------------------------------------------------------------------------------------
============================= [Parsing Completed] =============================
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/root/examples/osrt_python/ort/onnxrt_ep.py", line 223, in run_model
sess = rt.InferenceSession(config['model_path'] ,providers=EP_list, provider_options=[delegate_options, {}], sess_options=so)
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 362, in __init__
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 410, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Failed to add kernel for TIDL_0 com.microsoft TIDLExecutionProvider: Conflicting with a registered kernel with op versions.
Can you help diagnose and rectify this issue.
Thanks,
Suraj
