root@cf0e25348060:/home/root/examples/osrt_python/ort# python3 onnxrt_ep_cnd_topkonly.py -c Skipping import of model optimizer Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider'] Running 1 Models - ['condlanet_FNC_SafetyNoEdge_NewForkV1'] Running_Model : condlanet_FNC_SafetyNoEdge_NewForkV1 Running shape inference on model /home/root/lucid/model_onnx/topk_only.onnx ************Started the Interpreter tidl_tools_path = /home/root/tidl_tools artifacts_folder = ../../../model-artifacts//condlanet_FNC_SafetyNoEdge_NewForkV1/ tidl_tensor_bits = 16 debug_level = 4 num_tidl_subgraphs = 16 tidl_denylist = tidl_denylist_layer_name = tidl_denylist_layer_type = tidl_allowlist_layer_name = model_type = tidl_calibration_accuracy_level = 64 tidl_calibration_options:num_frames_calibration = 3 tidl_calibration_options:bias_calibration_iterations = 3 mixed_precision_factor = -1.000000 model_group_id = 0 power_of_2_quantization = 2 ONNX QDQ Enabled = 0 enable_high_resolution_optimization = 0 pre_batchnorm_fold = 1 add_data_convert_ops = 3 output_feature_16bit_names_list = m_params_16bit_names_list = m_single_core_layers_names_list = Inference mode = 0 Number of cores = 1 reserved_compile_constraints_flag = 1601 partial_init_during_compile = 0 ti_internal_reserved_1 = ========================= [Model Compilation Started] ========================= Model compilation will perform the following stages: 1. Parsing 2. Graph Optimization 3. Quantization & Calibration 4. Memory Planning ============================== [Version Summary] ============================== ------------------------------------------------------------------------------- | TIDL Tools Version | 10_00_04_00 | ------------------------------------------------------------------------------- | C7x Firmware Version | 10_00_02_00 | ------------------------------------------------------------------------------- | Runtime Version | 1.14.0+10000005 | ------------------------------------------------------------------------------- | Model Opset Version | 9 | ------------------------------------------------------------------------------- NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX ============================== [Parsing Started] ============================== [TIDL Import] WARNING: 'meta_layers_names_list' is not provided - running OD post processing in ARM mode Number of OD backbone nodes = 0 Size of odBackboneNodeIds = 0 [TIDL Import] [PARSER] UNSUPPORTED: Layer type not supported by TIDL --- layer type - TopK, Node name - TopK_219 -- [tidl_onnxRtImport_core.cpp, 503] [TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Slice, Node name - Slice_nf_0 -- [tidl_onnxRtImport_core.cpp, 524] [TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Slice, Node name - Slice_nf_1 -- [tidl_onnxRtImport_core.cpp, 524] [TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Reshape, Node name - Reshape_221 -- [tidl_onnxRtImport_core.cpp, 524] [TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Reshape, Node name - Reshape_223 -- [tidl_onnxRtImport_core.cpp, 524] ------------------------- Subgraph Information Summary ------------------------- ------------------------------------------------------------------------------- | Core | No. of Nodes | Number of Subgraphs | ------------------------------------------------------------------------------- | C7x | 4 | 1 | | CPU | 1 | x | ------------------------------------------------------------------------------- ------------------------------------------------------- | Node | Node Name | Reason | ------------------------------------------------------- | TopK | TopK_219 | Layer type not supported by TIDL | ------------------------------------------------------- Running Runtimes GraphViz - /home/root/tidl_tools/tidl_graphVisualiser_runtimes.out ../../../model-artifacts//condlanet_FNC_SafetyNoEdge_NewForkV1//allowedNode.txt ../../../model-artifacts//condlanet_FNC_SafetyNoEdge_NewForkV1//tempDir/graphvizInfo.txt ../../../model-artifacts//condlanet_FNC_SafetyNoEdge_NewForkV1//tempDir/runtimes_visualization.svg ============================= [Parsing Completed] ============================= TIDL_createStateImportFunc Started: Compute on node : TIDLExecutionProvider_TIDL_0_0 0, Slice, 1, 1, 501_nf, 501 1, Reshape, 2, 1, 501, seeds_val 2, Slice, 1, 1, 502_nf, 502 3, Reshape, 2, 1, 502, seeds Input tensor name - 502_nf Input tensor name - 501_nf Output tensor name - seeds Output tensor name - seeds_val ****************************** Start of session ******************* ***************Running_Inference Section ********** This is Lucid Model for image /home/root/lucid/data_onnx/test_images/img0277.png In TIDL_onnxRtImportInit subgraph_name=subgraph_0 Layer 0, subgraph id subgraph_0, name=seeds Layer 1, subgraph id subgraph_0, name=seeds_val Layer 2, subgraph id subgraph_0, name=502_nf Layer 3, subgraph id subgraph_0, name=501_nf ==================== [Optimization for subgraph_0 Started] ==================== In TIDL_runtimesOptimizeNet: LayerIndex = 8, dataIndex = 6 [TIDL Import] [PARSER] WARNING: Requested input data convert layer is not added to the network, It is currently not optimal ----------------------------- Optimization Summary ----------------------------- -------------------------------------------------------------------------- | Layer | Nodes before optimization | Nodes after optimization | -------------------------------------------------------------------------- | TIDL_SliceLayer | 2 | 2 | -------------------------------------------------------------------------- =================== [Optimization for subgraph_0 Completed] =================== In TIDL_runtimesPostProcessNet ************ in TIDL_subgraphRtCreate ************ The soft limit is 10240 The hard limit is 10240 MEM: Init ... !!! MEM: Init ... Done !!! 0.0s: VX_ZONE_INIT:Enabled 0.14s: VX_ZONE_ERROR:Enabled 0.18s: VX_ZONE_WARNING:Enabled 0.3151s: VX_ZONE_INIT:[tivxInit:190] Initialization Done !!! ************ TIDL_subgraphRtCreate done ************ ============= [Quantization & Calibration for subgraph_0 Started] ============= ******* In TIDL_subgraphRtInvoke ******** 0 1.00000 0.00000 255.00000 5 2 1.00000 0.00000 0.00000 6 1 1.00000 0.00010 0.99535 6 3 1.00000 0.00010 0.99535 6 4 1.00000 0.00010 0.00010 6 5 1.00000 0.00000 0.00000 6 6 1.00000 0.00010 0.00010 6 7 1.00000 0.00000 0.00000 6 8 1.00000 0.00010 0.00010 6 9 1.00000 0.00000 0.00000 6 10 1.00000 0.00010 0.00010 6 11 1.00000 0.00000 0.00000 6 13 1.00000 0.00010 0.00010 6 Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Sum of Layer Cycles 0 Sub Graph Stats 20.000000 1847.000000 488.000000 ******* TIDL_subgraphRtInvoke done ******** 2 ***************Running_Benchmark_Section ********** ****************************** Start of session ******************* ***************Running_Inference Section ********** This is Lucid Model for image /home/root/lucid/data_onnx/test_images/img0278.png ******* In TIDL_subgraphRtInvoke ******** 0 1.00000 0.00000 255.00000 5 2 1.00000 0.00000 0.00010 6 1 1.00000 0.00010 0.99535 6 3 1.00000 0.00010 0.99535 6 4 1.00000 0.00010 0.00010 6 5 1.00000 0.00000 0.00010 6 6 1.00000 0.00010 0.00010 6 7 1.00000 0.00000 0.00010 6 8 1.00000 0.00010 0.00010 6 9 1.00000 0.00000 0.00010 6 10 1.00000 0.00010 0.00010 6 11 1.00000 0.00000 0.00010 6 13 1.00000 0.00010 0.00010 6 Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Sum of Layer Cycles 0 Sub Graph Stats 11.000000 1431.000000 397.000000 ******* TIDL_subgraphRtInvoke done ******** 2 ***************Running_Benchmark_Section ********** ****************************** Start of session ******************* ***************Running_Inference Section ********** This is Lucid Model for image /home/root/lucid/data_onnx/test_images/img0279.png ******* In TIDL_subgraphRtInvoke ******** 0 1.00000 0.00000 255.00000 5 2 1.00000 0.00000 0.00010 6 1 1.00000 0.00010 0.99535 6 3 1.00000 0.00010 0.99535 6 4 1.00000 0.00010 0.00010 6 5 1.00000 0.00000 0.00010 6 6 1.00000 0.00010 0.00010 6 7 1.00000 0.00000 0.00010 6 8 1.00000 0.00010 0.00010 6 9 1.00000 0.00000 0.00010 6 10 1.00000 0.00010 0.00010 6 11 1.00000 0.00000 0.00010 6 13 1.00000 0.00010 0.00010 6 Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Sum of Layer Cycles 0 Sub Graph Stats 36.000000 2141.000000 296.000000 ******* TIDL_subgraphRtInvoke done ******** In TIDL_runtimesPostProcessNet -------- Running Calibration in Float Mode to Collect Tensor Statistics -------- [=============================================================================] 100 % ------------------ Fixed-point Calibration Iteration [1 / 3]: ------------------ [=============================================================================] 100 % ------------------ Fixed-point Calibration Iteration [2 / 3]: ------------------ [=============================================================================] 100 % ------------------ Fixed-point Calibration Iteration [3 / 3]: ------------------ [=============================================================================] 100 % ==================== [Quantization & Calibration Completed] ==================== ========================== [Memory Planning Started] ========================== ------------------------- Network Compiler Traces ------------------------------ Wrong data type for layer input 2, 14 Wrong data type for layer output 2, 14 Wrong data type for layer input 7, 38 Wrong data type for layer output 7, 38 Error : Error Code = [TIDL Import] FATAL ERROR: Network Compiler failed to execute - Memory planning failed with return code - 65280 -- [tidl_import_core.cpp, 1002] [TIDL Import] Aborting