Part Number: AM62A7
Tool/software:
Greetings!
This post is a follow-up question to my previous post which has been locked.
I have finally managed to get back to this issue and I think I identified the source of the problem. For my use-case, I would ideally receive a dynamic number of inputs and infer a score to each of these inputs based on a fixed set of attributes associated to these inputs (i.e., a table with multiple rows, each with a fixed number of columns representing the attributes). However, as seen in my previous post, setting the inputs as dynamic when creating the model resulted in the compiler failing to generate the model’s binary. By modifying the model to only process a fixed number of inputs (e.g., 20 inputs every time the model is called), the compiler seem to successfully finish the compilation and generates the model’s binary:
root@ac9ed4158784:/home/root/examples/osrt_python/ort# python3 onnxrt_ep_static.py -c -m model_notdynamic_opset18
Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
Running Model - model_notdynamic_opset18
Running_Model : model_notdynamic_opset18
Running shape inference on model ../../../models/public/model_notdynamic_opset18.onnx
2
tidl_tools_path = /home/root/tidl_tools
artifacts_folder = ../../../model-artifacts/model_notdynamic_opset18/artifacts
tidl_tensor_bits = 16
debug_level = 2
num_tidl_subgraphs = 16
tidl_denylist =
tidl_denylist_layer_name =
tidl_denylist_layer_type =
tidl_allowlist_layer_name =
model_type =
tidl_calibration_accuracy_level = 7
tidl_calibration_options:num_frames_calibration = 2
tidl_calibration_options:bias_calibration_iterations = 5
mixed_precision_factor = -1.000000
model_group_id = 0
power_of_2_quantization = 2
ONNX QDQ Enabled = 0
enable_high_resolution_optimization = 0
pre_batchnorm_fold = 1
add_data_convert_ops = 3
output_feature_16bit_names_list =
m_params_16bit_names_list =
m_single_core_layers_names_list =
Inference mode = 0
Number of cores = 1
reserved_compile_constraints_flag = 1601
partial_init_during_compile = 0
ti_internal_reserved_1 =
========================= [Model Compilation Started] =========================
Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning
============================== [Version Summary] ==============================
-------------------------------------------------------------------------------
| TIDL Tools Version | 10_00_08_00 |
-------------------------------------------------------------------------------
| C7x Firmware Version | 10_00_02_00 |
-------------------------------------------------------------------------------
| Runtime Version | 1.14.0+10000005 |
-------------------------------------------------------------------------------
| Model Opset Version | 18 |
-------------------------------------------------------------------------------
NOTE: The runtime version here specifies ONNXRT_VERSION+TIDL_VERSION
Ex: 1.14.0+1000XXXX -> ONNXRT 1.14.0 and a TIDL_VERSION 10.00.XX.XX
============================== [Parsing Started] ==============================
[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
[TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Gemm, Node name - /network/network.0/Gemm -- [tidl_onnxRtImport_core.cpp, 529]
[TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Relu, Node name - /network/network.1/Relu -- [tidl_onnxRtImport_core.cpp, 529]
[TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Gemm, Node name - /network/network.2/Gemm -- [tidl_onnxRtImport_core.cpp, 529]
[TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Relu, Node name - /network/network.3/Relu -- [tidl_onnxRtImport_core.cpp, 529]
[TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Gemm, Node name - /network/network.4/Gemm -- [tidl_onnxRtImport_core.cpp, 529]
[TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Relu, Node name - /network/network.5/Relu -- [tidl_onnxRtImport_core.cpp, 529]
[TIDL Import] [PARSER] SUPPORTED: Layers type supported by TIDL --- layer type - Gemm, Node name - /network/network.6/Gemm -- [tidl_onnxRtImport_core.cpp, 529]
------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
| Core | No. of Nodes | Number of Subgraphs |
-------------------------------------------------------------------------------
| C7x | 7 | 1 |
| CPU | 0 | x |
-------------------------------------------------------------------------------
Running Runtimes GraphViz - /home/root/tidl_tools/tidl_graphVisualiser_runtimes.out ../../../model-artifacts/model_notdynamic_opset18/artifacts/allowedNode.txt ../../../model-artifacts/model_notdynamic_opset18/artifacts/tempDir/graphvizInfo.txt ../../../model-artifacts/model_notdynamic_opset18/artifacts/tempDir/runtimes_visualization.svg
============================= [Parsing Completed] =============================
TIDL_createStateImportFunc Started:
Compute on node : TIDLExecutionProvider_TIDL_0_0
0, Gemm, 3, 1, input, /network/network.0/Gemm_output_0
1, Relu, 1, 1, /network/network.0/Gemm_output_0, /network/network.1/Relu_output_0
2, Gemm, 3, 1, /network/network.1/Relu_output_0, /network/network.2/Gemm_output_0
3, Relu, 1, 1, /network/network.2/Gemm_output_0, /network/network.3/Relu_output_0
4, Gemm, 3, 1, /network/network.3/Relu_output_0, /network/network.4/Gemm_output_0
5, Relu, 1, 1, /network/network.4/Gemm_output_0, /network/network.5/Relu_output_0
6, Gemm, 3, 1, /network/network.5/Relu_output_0, output
Input tensor name - input
Output tensor name - output
In TIDL_onnxRtImportInit subgraph_name=subgraph_0
Layer 0, subgraph id subgraph_0, name=output
Layer 1, subgraph id subgraph_0, name=input
==================== [Optimization for subgraph_0 Started] ====================
In TIDL_runtimesOptimizeNet: LayerIndex = 9, dataIndex = 8
----------------------------- Optimization Summary -----------------------------
---------------------------------------------------------------------------------
| Layer | Nodes before optimization | Nodes after optimization |
---------------------------------------------------------------------------------
| TIDL_ReLULayer | 3 | 0 |
| TIDL_InnerProductLayer | 4 | 4 |
---------------------------------------------------------------------------------
=================== [Optimization for subgraph_0 Completed] ===================
In TIDL_runtimesPostProcessNet
************ in TIDL_subgraphRtCreate ************
The soft limit is 10240
The hard limit is 10240
MEM: Init ... !!!
MEM: Init ... Done !!!
0.0s: VX_ZONE_INIT:Enabled
0.22s: VX_ZONE_ERROR:Enabled
0.27s: VX_ZONE_WARNING:Enabled
0.1506s: VX_ZONE_INIT:[tivxInit:190] Initialization Done !!!
--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum , Space , Attribute , Alignment , Size(KBytes), BasePtr
0 , DDR Cacheable , Persistent , 128, 19.27 , 0x00000000
1 , DDR Cacheable , Persistent , 128, 0.65 , 0x00000000
2 , DDR Cacheable , Scratch , 128, 16.00 , 0x00000000
3 , DDR Cacheable , Scratch , 128, 4.00 , 0x00000000
4 , DDR Cacheable , Scratch , 128, 56.00 , 0x00000000
5 , DDR Cacheable , Persistent , 128, 263.16 , 0x00000000
6 , DDR Cacheable , Scratch , 128, 3.05 , 0x00000000
7 , DDR Cacheable , Scratch , 128, 0.12 , 0x00000000
8 , DDR Cacheable , Scratch , 128, 1.38 , 0x00000000
9 , DDR Cacheable , Scratch , 128, 4.56 , 0x00000000
10 , DDR Cacheable , Persistent , 128, 302.89 , 0x00000000
11 , DDR Cacheable , Scratch , 128, 512.25 , 0x00000000
12 , DDR Cacheable , Persistent , 128, 0.12 , 0x00000000
13 , DDR Cacheable , Persistent , 128, 6319.57 , 0x00000000
14 , DDR Cacheable , Persistent , 128, 0.00 , 0x00000000
15 , DDR Cacheable , Persistent , 128, 0.12 , 0x00000000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 7503.16
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
To get the actual TIDL memory requirement make sure to run on EVM with
debugTraceLevel = 2
--------------------------------------------
TIDL init call from ivision API
--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum , Space , Attribute , Alignment , Size(KBytes), BasePtr
0 , DDR Cacheable , Persistent , 128, 19.27 , 0x5d756000
1 , DDR Cacheable , Persistent , 128, 0.65 , 0x60dac000
2 , DDR Cacheable , Scratch , 128, 16.00 , 0x5c802000
3 , DDR Cacheable , Scratch , 128, 4.00 , 0x60dab000
4 , DDR Cacheable , Scratch , 128, 56.00 , 0x21a03000
5 , DDR Cacheable , Persistent , 128, 263.16 , 0x1b584000
6 , DDR Cacheable , Scratch , 128, 3.05 , 0x60a09000
7 , DDR Cacheable , Scratch , 128, 0.12 , 0x60267000
8 , DDR Cacheable , Scratch , 128, 1.38 , 0x60266000
9 , DDR Cacheable , Scratch , 128, 4.56 , 0x5de8d000
10 , DDR Cacheable , Persistent , 128, 302.89 , 0x1b538000
11 , DDR Cacheable , Scratch , 128, 512.25 , 0x1b4b7000
12 , DDR Cacheable , Persistent , 128, 0.12 , 0x5d63e000
13 , DDR Cacheable , Persistent , 128, 6319.57 , 0xdb3a8000
14 , DDR Cacheable , Persistent , 128, 0.00 , 0x5d63d000
15 , DDR Cacheable , Persistent , 128, 0.12 , 0x5d201000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 7503.16
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
To get the actual TIDL memory requirement make sure to run on EVM with
debugTraceLevel = 2
--------------------------------------------
Alg Init for Layer # - 1
Alg Init for Layer # - 2
Alg Init for Layer # - 3
Alg Init for Layer # - 4
Alg Init for Layer # - 5
Alg Init for Layer # - 6
PREEMPTION: Adding a new priority object for targetPriority = 0, handle = 0x786d5d756000
PREEMPTION: Now total number of priority objects = 1 at priorityId = 0, with new memRec of base = 0x786d5d63e000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x786d5d756000, return Addr = 0x786d0a60f0b8
************ TIDL_subgraphRtCreate done ************
============= [Quantization & Calibration for subgraph_0 Started] =============
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x786d5d756000
TIDL_activate is called with handle : 0x786d5d756000 - Copying handle of size 19736 from 0x786d5d756000 to 0x786d21a03080
Core 0 Alg Process for Layer # - 0, layer type 0
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x786d60a09000
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x786d60a09100
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x786d60a09500
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x786d60a09780
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x786d60a09880
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x786d61ea5000
Core 0 Alg Process for Layer # - 7, layer type 0
TIDL_process is completed with handle : 0x786d5d756000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 20.000000 520.000000 314.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x786d5d756000
Core 0 Alg Process for Layer # - 0, layer type 0
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x786d60a09000
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x786d60a09100
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x786d60a09500
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x786d60a09780
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x786d60a09880
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x786d61ea5000
Core 0 Alg Process for Layer # - 7, layer type 0
TIDL_process is completed with handle : 0x786d5d756000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 3.000000 182.000000 7019.000000
******* TIDL_subgraphRtInvoke done ********
In TIDL_runtimesPostProcessNet
-------- Running Calibration in Float Mode to Collect Tensor Statistics --------
[=============================================================================] 100 %
------------------ Fixed-point Calibration Iteration [1 / 5]: ------------------
[=============================================================================] 100 %
------------------ Fixed-point Calibration Iteration [2 / 5]: ------------------
[=============================================================================] 100 %
------------------ Fixed-point Calibration Iteration [3 / 5]: ------------------
[=============================================================================] 100 %
------------------ Fixed-point Calibration Iteration [4 / 5]: ------------------
[=============================================================================] 100 %
------------------ Fixed-point Calibration Iteration [5 / 5]: ------------------
[=============================================================================] 100 %
==================== [Quantization & Calibration Completed] ====================
========================== [Memory Planning Started] ==========================
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
========================= [Memory Planning Completed] =========================
======================== Subgraph Compiled Successfully ========================
Completed_Model : 1, Name : model_notdynamic_opset18 , Total time : 1596.71, Offload Time : 0.00 , DDR RW MBs : 0, Output Bin File : py_out_model_notdynamic_opset18data_sample.bin
************ in TIDL_subgraphRtDelete ************
TIDL_deactivate is called with handle : 0x786d5d756000 - Copying handle of size 19736 from 0x786d21a03080 to 0x786d5d756000
MEM: Deinit ... !!!
MEM: Alloc's: 26 alloc's of 14642801 bytes
MEM: Free's : 26 free's of 14642801 bytes
MEM: Open's : 0 allocs of 0 bytes
MEM: Deinit ... Done !!!
However, when trying to run the inference part on the AM62A, I am not sure if the model operations are indeed being offloaded to the hardware accelerators. As you can see in the log, it correctly identifies the number of subgraphs and the nodes to be offloaded. Nevertheless, the print showing the time offloaded to the hardware accelerator is always zero. If I understood it correclty, if the layerCycles is not zero, the operations are being offloaded. However, the benchmarks seem to be saying otherwise.
root@mitysom-am62ax:/opt/edgeai-tidl-tools/examples/osrt_python/ort# python3 onnxrt_ep_e2e.py -m model_notdynamic_opset18
Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
Running Model - model_notdynamic_opset18
Running_Model : model_notdynamic_opset18
/usr/lib/python3.12/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator StandardScaler from version 1.5.2 when using version 1.5.0. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
warnings.warn(
libtidl_onnxrt_EP loaded 0x2c45ead0
artifacts_folder = ../../../model-artifacts/model_notdynamic_opset18/artifacts
debug_level = 2
target_priority = 0
max_pre_empt_delay = 340282346638528859811704183484516925440.000000
Final number of subgraphs created are : 1, - Offloaded Nodes - 7, Total Nodes - 7
In TIDL_createStateInfer
Compute on node : TIDLExecutionProvider_TIDL_0_0
************ in TIDL_subgraphRtCreate ************
APP: Init ... !!!
1365.379095 s: MEM: Init ... !!!
1365.379317 s: MEM: Initialized DMA HEAP (fd=5) !!!
1365.379598 s: MEM: Init ... Done !!!
1365.379707 s: IPC: Init ... !!!
1365.399654 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
1365.405862 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
1365.406440 s: VX_ZONE_INIT:Enabled
1365.406819 s: VX_ZONE_ERROR:Enabled
1365.407007 s: VX_ZONE_WARNING:Enabled
1365.408583 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0
1365.410002 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1
1365.410990 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2
1365.411848 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3
1365.412168 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
1365.412694 s: VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
RT-Profile: TIDLRT_init_profiling
tidlrt_create : 59629932 ns,
tidl_rt_ovx_Init : 36642640 ns,
vxCreateContext : 1960429 ns,
init_tidl_tiovx : 3803597 ns,
create_graph_tidl_tiovx : 585690 ns,
verify_graph_tidl_tiovx : 16295643 ns,
tivxTIDLLoadKernels : 54380 ns,
mapConfig : 716232 ns,
tivxAddKernelTIDL : 116496 ns,
mapNetwork : 2239707 ns,
setCreateParams : 316548 ns,
setArgs : 357179 ns,
vxCreateUserDataObject : 54391 ns,
vxMapUserDataObject : 998320 ns,
memcopy_network_buffer : 1066015 ns,
vxUnmapUserDataObject : 117886 ns,
************ TIDL_subgraphRtCreate done ************
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 78991, 1153, 2364, 10347, 2062, 0, 0, 0, 0, 0, 928, 1, 0, 5083, 0, 0, 0,
2, 91038, 7526, 7526, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 77501, 6550, 6550, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 69519, 1657, 1657, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 94097, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 72414, 1401, 3948, 11740, 2241, 0, 0, 0, 0, 0, 797, 1, 0, 4828, 0, 0, 0,
Sum of Layer Cycles 483560
Sub Graph Stats 108.000000 1587.000000 928.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 80077, 994, 1964, 12634, 2161, 0, 0, 0, 0, 0, 695, 1, 0, 6671, 0, 0, 0,
2, 87979, 7926, 7926, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 77053, 6561, 6561, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 69278, 1516, 1516, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92159, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 71222, 837, 3185, 11792, 2376, 0, 0, 0, 0, 0, 697, 1, 0, 4782, 0, 0, 0,
Sum of Layer Cycles 477768
Sub Graph Stats 66.000000 1330.000000 627.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 76817, 937, 2081, 12153, 1908, 0, 0, 0, 0, 0, 869, 1, 0, 6516, 0, 0, 0,
2, 89545, 7856, 7856, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 76345, 6552, 6552, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68506, 1454, 1454, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 91596, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 70694, 872, 3273, 11271, 2346, 0, 0, 0, 0, 0, 750, 1, 0, 4267, 0, 0, 0,
Sum of Layer Cycles 473503
Sub Graph Stats 117.000000 1320.000000 598.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 77770, 992, 1983, 12338, 1898, 0, 0, 0, 0, 0, 716, 1, 0, 6102, 0, 0, 0,
2, 92091, 7921, 7921, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 78001, 6625, 6625, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68711, 1395, 1395, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 70398, 1144, 3326, 11304, 2386, 0, 0, 0, 0, 0, 529, 1, 0, 4299, 0, 0, 0,
Sum of Layer Cycles 479147
Sub Graph Stats 54.000000 1349.000000 612.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 78462, 928, 2169, 11770, 2100, 0, 0, 0, 0, 0, 966, 1, 0, 6602, 0, 0, 0,
2, 89245, 7919, 7919, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 76260, 6556, 6556, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 71536, 1925, 1925, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92422, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 71042, 1226, 3421, 11844, 2351, 0, 0, 0, 0, 0, 531, 1, 0, 4841, 0, 0, 0,
Sum of Layer Cycles 478967
Sub Graph Stats 61.000000 1336.000000 599.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 79265, 963, 2215, 12532, 2068, 0, 0, 0, 0, 0, 977, 1, 0, 6758, 0, 0, 0,
2, 90809, 7851, 7851, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 76742, 6177, 6177, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 67853, 1398, 1398, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92480, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 71873, 869, 3219, 11306, 2350, 0, 0, 0, 0, 0, 689, 1, 0, 4354, 0, 0, 0,
Sum of Layer Cycles 479022
Sub Graph Stats 60.000000 1324.000000 605.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 80170, 930, 2009, 12855, 1912, 0, 0, 0, 0, 0, 804, 1, 0, 6675, 0, 0, 0,
2, 90907, 7777, 7777, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 76238, 6554, 6554, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 69686, 1397, 1397, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 95052, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 70726, 838, 3189, 11713, 2382, 0, 0, 0, 0, 0, 669, 1, 0, 4709, 0, 0, 0,
Sum of Layer Cycles 482779
Sub Graph Stats 61.000000 1352.000000 594.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 76144, 946, 1869, 9897, 1869, 0, 0, 0, 0, 0, 648, 1, 0, 5107, 0, 0, 0,
2, 89051, 7981, 7981, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 75437, 6501, 6501, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 67242, 1396, 1396, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 93172, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 68802, 828, 3182, 11805, 2380, 0, 0, 0, 0, 0, 694, 1, 0, 4845, 0, 0, 0,
Sum of Layer Cycles 469848
Sub Graph Stats 58.000000 1305.000000 643.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 90620, 899, 2048, 10535, 1856, 0, 0, 0, 0, 0, 873, 1, 0, 5043, 0, 0, 0,
2, 89112, 8065, 8065, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 75917, 6499, 6499, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68540, 1340, 1340, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 91599, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 72069, 1227, 3723, 11169, 2349, 0, 0, 0, 0, 0, 842, 1, 0, 4163, 0, 0, 0,
Sum of Layer Cycles 487857
Sub Graph Stats 65.000000 1318.000000 597.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 78562, 919, 2007, 12153, 1918, 0, 0, 0, 0, 0, 813, 1, 0, 6516, 0, 0, 0,
2, 89256, 7748, 7748, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 76883, 6305, 6305, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68535, 1398, 1398, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 91600, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 72363, 867, 3218, 11848, 2352, 0, 0, 0, 0, 0, 697, 1, 0, 4841, 0, 0, 0,
Sum of Layer Cycles 477199
Sub Graph Stats 57.000000 1314.000000 602.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 77556, 976, 2031, 12107, 1903, 0, 0, 0, 0, 0, 780, 1, 0, 6518, 0, 0, 0,
2, 89589, 7671, 7671, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 77172, 6545, 6545, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 67978, 1398, 1398, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 94076, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 72376, 928, 3284, 11197, 2393, 0, 0, 0, 0, 0, 700, 1, 0, 4299, 0, 0, 0,
Sum of Layer Cycles 478747
Sub Graph Stats 56.000000 1332.000000 611.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 78470, 948, 2037, 12291, 1888, 0, 0, 0, 0, 0, 814, 1, 0, 6100, 0, 0, 0,
2, 89331, 7993, 7993, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 77085, 6550, 6550, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68879, 1393, 1393, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 91753, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 70678, 867, 3218, 11173, 2349, 0, 0, 0, 0, 0, 699, 1, 0, 4166, 0, 0, 0,
Sum of Layer Cycles 476196
Sub Graph Stats 57.000000 1317.000000 596.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 76735, 934, 2010, 11432, 1916, 0, 0, 0, 0, 0, 801, 1, 0, 5797, 0, 0, 0,
2, 90560, 7906, 7906, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 77437, 6036, 6036, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68535, 1336, 1336, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 91863, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 69899, 772, 2959, 11710, 2107, 0, 0, 0, 0, 0, 529, 1, 0, 4803, 0, 0, 0,
Sum of Layer Cycles 475029
Sub Graph Stats 63.000000 1319.000000 599.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 81728, 923, 2064, 13043, 2104, 0, 0, 0, 0, 0, 871, 1, 0, 7064, 0, 0, 0,
2, 88492, 7992, 7992, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 77510, 6510, 6510, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 70172, 1838, 1838, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92750, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 70725, 874, 3060, 11074, 2094, 0, 0, 0, 0, 0, 530, 1, 0, 4352, 0, 0, 0,
Sum of Layer Cycles 481377
Sub Graph Stats 54.000000 1343.000000 612.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 75355, 938, 2041, 12147, 1913, 0, 0, 0, 0, 0, 828, 1, 0, 6854, 0, 0, 0,
2, 88503, 7987, 7987, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 75433, 6492, 6492, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 67120, 1397, 1397, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92533, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 70405, 834, 3184, 11734, 2368, 0, 0, 0, 0, 0, 697, 1, 0, 4929, 0, 0, 0,
Sum of Layer Cycles 469349
Sub Graph Stats 58.000000 1309.000000 665.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 76141, 1026, 2132, 10407, 1932, 0, 0, 0, 0, 0, 831, 1, 0, 5209, 0, 0, 0,
2, 90026, 7991, 7991, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 75091, 6525, 6525, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68433, 1395, 1395, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 93286, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 69874, 835, 3186, 11302, 2385, 0, 0, 0, 0, 0, 698, 1, 0, 4300, 0, 0, 0,
Sum of Layer Cycles 472851
Sub Graph Stats 57.000000 1318.000000 597.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 78741, 961, 2371, 11853, 1909, 0, 0, 0, 0, 0, 1135, 1, 0, 6223, 0, 0, 0,
2, 90458, 7994, 7994, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 75433, 6576, 6576, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68047, 1387, 1387, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92596, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 69578, 833, 3182, 11168, 2390, 0, 0, 0, 0, 0, 676, 1, 0, 4165, 0, 0, 0,
Sum of Layer Cycles 474853
Sub Graph Stats 57.000000 1321.000000 587.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 79057, 878, 2116, 12355, 2141, 0, 0, 0, 0, 0, 963, 1, 0, 6598, 0, 0, 0,
2, 88853, 7924, 7924, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 75436, 6501, 6501, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 68678, 1398, 1398, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92089, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 70400, 821, 3171, 11844, 2399, 0, 0, 0, 0, 0, 698, 1, 0, 4842, 0, 0, 0,
Sum of Layer Cycles 474513
Sub Graph Stats 60.000000 1324.000000 595.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 77323, 964, 2184, 11687, 2096, 0, 0, 0, 0, 0, 945, 1, 0, 6515, 0, 0, 0,
2, 89595, 7992, 7992, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 76347, 6482, 6482, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 70531, 1884, 1884, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 92453, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 72374, 925, 3281, 11380, 2215, 0, 0, 0, 0, 0, 702, 1, 0, 4520, 0, 0, 0,
Sum of Layer Cycles 478623
Sub Graph Stats 62.000000 1315.000000 593.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 79517, 873, 2020, 12982, 1923, 0, 0, 0, 0, 0, 872, 1, 0, 6827, 0, 0, 0,
2, 90142, 8067, 8067, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
3, 77194, 6505, 6505, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
4, 70157, 1893, 1893, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
5, 91809, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
6, 69910, 776, 2961, 11695, 2105, 0, 0, 0, 0, 0, 529, 1, 0, 4828, 0, 0, 0,
Sum of Layer Cycles 478729
Sub Graph Stats 55.000000 1355.000000 613.000000
******* TIDL_subgraphRtInvoke done ********
Predictions : [3, 2, 2, 4, 4, 2, 3, 3, 4, 3, 2, 3, 4, 5, 5, 2, 2, 3, 3, 3]
predicted_score actual_score
0 3 2
1 2 2
2 2 2
3 4 3
4 4 4
5 2 2
6 3 3
7 3 2
8 4 4
9 3 3
10 2 2
11 3 3
12 4 4
13 5 5
14 5 5
15 2 2
16 2 2
17 3 2
18 3 3
19 3 3
Accuracy: 80.00%
Completed_Model : 1, Name : model_notdynamic_opset18 , Total time : 46.49, Offload Time : 0.00 , DDR RW MBs : 0
************ in TIDL_subgraphRtDelete ************
1365.543292 s: VX_ZONE_INIT:[tivxHostDeInitLocal:120] De-Initialization Done for HOST !!!
1365.548162 s: VX_ZONE_INIT:[tivxDeInitLocal:206] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
1365.548845 s: IPC: Deinit ... !!!
1365.549652 s: IPC: DeInit ... Done !!!
1365.549737 s: MEM: Deinit ... !!!
1365.549757 s: DDR_SHARED_MEM: Alloc's: 7 alloc's of 1670636 bytes
1365.549769 s: DDR_SHARED_MEM: Free's : 7 free's of 1670636 bytes
1365.549780 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes
1365.549796 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
Running the same script on the x86 docker gives me the following:
libtidl_onnxrt_EP loaded 0x5d39f021e9a0
artifacts_folder = ../../../model-artifacts/model_notdynamic_opset18/artifacts
debug_level = 2
target_priority = 0
max_pre_empt_delay = 340282346638528859811704183484516925440.000000
Final number of subgraphs created are : 1, - Offloaded Nodes - 7, Total Nodes - 7
In TIDL_createStateInfer
Compute on node : TIDLExecutionProvider_TIDL_0_0
************ in TIDL_subgraphRtCreate ************
The soft limit is 10240
The hard limit is 10240
MEM: Init ... !!!
MEM: Init ... Done !!!
0.0s: VX_ZONE_INIT:Enabled
0.9s: VX_ZONE_ERROR:Enabled
0.10s: VX_ZONE_WARNING:Enabled
0.1797s: VX_ZONE_INIT:[tivxInit:190] Initialization Done !!!
--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum , Space , Attribute , Alignment , Size(KBytes), BasePtr
0 , DDR Cacheable , Persistent , 128, 19.27 , 0x00000000
1 , DDR Cacheable , Persistent , 128, 0.65 , 0x00000000
2 , DDR Cacheable , Scratch , 128, 16.00 , 0x00000000
3 , DDR Cacheable , Scratch , 128, 224.00 , 0x00000000
4 , DDR Cacheable , Scratch , 128, 1024.00 , 0x00000000
5 , DDR Cacheable , Persistent , 128, 260.58 , 0x00000000
6 , DDR Cacheable , Scratch , 128, 1.75 , 0x00000000
7 , DDR Cacheable , Scratch , 128, 1.38 , 0x00000000
8 , DDR Cacheable , Scratch , 128, 5.38 , 0x00000000
9 , DDR Cacheable , Scratch , 128, 5.25 , 0x00000000
10 , DDR Cacheable , Persistent , 128, 302.89 , 0x00000000
11 , DDR Cacheable , Scratch , 128, 512.25 , 0x00000000
12 , DDR Cacheable , Persistent , 128, 0.12 , 0x00000000
13 , DDR Cacheable , Persistent , 128, 1308.80 , 0x00000000
14 , DDR Cacheable , Persistent , 128, 0.00 , 0x00000000
15 , DDR Cacheable , Persistent , 128, 46.25 , 0x00000000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 3728.57
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
To get the actual TIDL memory requirement make sure to run on EVM with
debugTraceLevel = 2
--------------------------------------------
TIDL init call from ivision API
--------------------------------------------
TIDL Memory size requiement (record wise):
MemRecNum , Space , Attribute , Alignment , Size(KBytes), BasePtr
0 , DDR Cacheable , Persistent , 128, 19.27 , 0xb7256000
1 , DDR Cacheable , Persistent , 128, 0.65 , 0xba893000
2 , DDR Cacheable , Scratch , 128, 16.00 , 0xb9c66000
3 , DDR Cacheable , Scratch , 128, 224.00 , 0x75288000
4 , DDR Cacheable , Scratch , 128, 1024.00 , 0x73b00000
5 , DDR Cacheable , Persistent , 128, 260.58 , 0x75246000
6 , DDR Cacheable , Scratch , 128, 1.75 , 0xba892000
7 , DDR Cacheable , Scratch , 128, 1.38 , 0xba4f0000
8 , DDR Cacheable , Scratch , 128, 5.38 , 0xb703d000
9 , DDR Cacheable , Scratch , 128, 5.25 , 0xb6c02000
10 , DDR Cacheable , Persistent , 128, 302.89 , 0x74e6c000
11 , DDR Cacheable , Scratch , 128, 512.25 , 0x74437000
12 , DDR Cacheable , Persistent , 128, 0.12 , 0xb788d000
13 , DDR Cacheable , Persistent , 128, 1308.80 , 0x730b8000
14 , DDR Cacheable , Persistent , 128, 0.00 , 0xb6c01000
15 , DDR Cacheable , Persistent , 128, 46.25 , 0x80003000
--------------------------------------------
Total memory size requirement (space wise):
Mem Space , Size(KBytes)
DDR Cacheable, 3728.57
--------------------------------------------
NOTE: Memory requirement in host emulation can be different from the same on EVM
To get the actual TIDL memory requirement make sure to run on EVM with
debugTraceLevel = 2
--------------------------------------------
Alg Init for Layer # - 1
Alg Init for Layer # - 2
Alg Init for Layer # - 3
Alg Init for Layer # - 4
Alg Init for Layer # - 5
Alg Init for Layer # - 6
PREEMPTION: Adding a new priority object for targetPriority = 0, handle = 0x7646b7256000
PREEMPTION: Now total number of priority objects = 1 at priorityId = 0, with new memRec of base = 0x7646b788d000 and size = 128
PREEMPTION: Requesting context memory addr for handle 0x7646b7256000, return Addr = 0x76466a60f0b8
************ TIDL_subgraphRtCreate done ************
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
TIDL_activate is called with handle : 0x7646b7256000 - Copying handle of size 19736 from 0x7646b7256000 to 0x764673bf2000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 15.000000 292.000000 272.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 2.000000 81.000000 7415.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 2.000000 135.000000 134.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 1.000000 82.000000 87.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 86.000000 67.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 81.000000 69.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 80.000000 58.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 90.000000 62.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 89.000000 62.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 54.000000 78.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 77.000000 67.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 3.000000 74.000000 58.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 79.000000 58.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 94.000000 58.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 79.000000 88.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 1.000000 119.000000 98.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 1.000000 52.000000 117.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 1.000000 45.000000 83.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 53.000000 83.000000
******* TIDL_subgraphRtInvoke done ********
******* In TIDL_subgraphRtInvoke ********
TIDL_process is started with handle : 0x7646b7256000
Core 0 Alg Process for Layer # - 1, layer type 29
Processing Layer # - 1
Core 0 End of Layer # - 1 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 2, layer type 6
Processing Layer # - 2
Executing reference flow for inner product layer
Core 0 End of Layer # - 2 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 3, layer type 6
Processing Layer # - 3
Executing reference flow for inner product layer
Core 0 End of Layer # - 3 with outPtrs[0] = 0x764673b09f00
Core 0 Alg Process for Layer # - 4, layer type 6
Processing Layer # - 4
Executing reference flow for inner product layer
Core 0 End of Layer # - 4 with outPtrs[0] = 0x764673b00000
Core 0 Alg Process for Layer # - 5, layer type 6
Processing Layer # - 5
Executing reference flow for inner product layer
Core 0 End of Layer # - 5 with outPtrs[0] = 0x764673b00280
Core 0 Alg Process for Layer # - 6, layer type 29
Processing Layer # - 6
Core 0 End of Layer # - 6 with outPtrs[0] = 0x7646bb98c000
TIDL_process is completed with Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
Running Model - model_notdynamic_opset18
Running_Model : model_notdynamic_opset18
Predictions : [2, 2, 2, 4, 4, 2, 3, 2, 4, 3, 2, 3, 4, 5, 5, 2, 2, 2, 3, 3]
predicted_score actual_score
0 2 2
1 2 2
2 2 2
3 4 3
4 4 4
5 2 2
6 3 3
7 2 2
8 4 4
9 3 3
10 2 2
11 3 3
12 4 4
13 5 5
14 5 5
15 2 2
16 2 2
17 2 2
18 3 3
19 3 3
Accuracy: 95.00%
Completed_Model : 1, Name : model_notdynamic_opset18 , Total time : 11.27, Offload Time : 0.00 , DDR RW MBs : 0
handle : 0x7646b7256000
Layer, Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles,LayerDeinitCycles,LastBlockCycles, paddingTrigger, paddingWait,LayerWithoutPad,LayerHandleCopy, BackupCycles, RestoreCycles,Multic7xContextCopyCycles,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Sum of Layer Cycles 0
Sub Graph Stats 0.000000 53.000000 78.000000
******* TIDL_subgraphRtInvoke done ********
************ in TIDL_subgraphRtDelete ************
TIDL_deactivate is called with handle : 0x7646b7256000 - Copying handle of size 19736 from 0x764673bf2000 to 0x7646b7256000
MEM: Deinit ... !!!
MEM: Alloc's: 26 alloc's of 5647733 bytes
MEM: Free's : 26 free's of 5647733 bytes
MEM: Open's : 0 allocs of 0 bytes
MEM: Deinit ... Done !!!
Additionally, running the script with debug_level=3 on the board hangs the process with the following log:
root@mitysom-am62ax:/opt/edgeai-tidl-tools/examples/osrt_python/ort# python3 onnxrt_ep_e2e.py -m model_notdynamic_opset18 Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider'] Running Model - model_notdynamic_opset18 Running_Model : model_notdynamic_opset18 /usr/lib/python3.12/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator StandardScaler from version 1.5.2 when using version 1.5.0. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations warnings.warn( libtidl_onnxrt_EP loaded 0x3e29c260 artifacts_folder = ../../../model-artifacts/model_notdynamic_opset18/artifacts debug_level = 3 target_priority = 0 max_pre_empt_delay = 340282346638528859811704183484516925440.000000 Final number of subgraphs created are : 1, - Offloaded Nodes - 7, Total Nodes - 7 In TIDL_createStateInfer Compute on node : TIDLExecutionProvider_TIDL_0_0 ************ in TIDL_subgraphRtCreate ************ APP: Init ... !!! 6090.876017 s: MEM: Init ... !!! 6090.876337 s: MEM: Initialized DMA HEAP (fd=5) !!! 6090.876632 s: MEM: Init ... Done !!! 6090.876746 s: IPC: Init ... !!! 6090.896445 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 6090.902588 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 6090.903378 s: VX_ZONE_INIT:Enabled 6090.903529 s: VX_ZONE_ERROR:Enabled 6090.903622 s: VX_ZONE_WARNING:Enabled 6090.905153 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0 6090.906129 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1 6090.906877 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2 6090.907324 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3 6090.907704 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!! 6090.908083 s: VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!! ************ TIDL_subgraphRtCreate done ************ ******* In TIDL_subgraphRtInvoke ********
Is there a way to confirm that it is actually being offloaded? On top of that, is there a way to compile models to support a dynamic number of inputs?
Customized compilation script:
And the model's entry on the models_config.py file:
"model_notdynamic_opset18": create_model_config(
source=AttrDict(
infer_shape=True,
),
preprocess=AttrDict(
resize=None,
crop=None,
data_layout="NCHW",
resize_with_pad=False,
reverse_channels=False,
),
session=AttrDict(
session_name="onnxrt",
model_path=os.path.join(models_base_path, "model_notdynamic_opset18.onnx"),
input_optimization=True,
),
task_type="other",
extra_info=AttrDict(
num_rows=20,
num_classes=5,
),
),
Thank you for the support.
Best regards,
Giann.
