root@908b5978fca7:/home/root/examples/osrt_python/ort# gdb --args python3 onnxrt_ep.py --compile GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1 Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from python3... (No debugging symbols found in python3) (gdb) run Starting program: /usr/bin/python3 onnxrt_ep.py --compile warning: Error disabling address space randomization: Operation not permitted [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x75e69e000640 (LWP 184)] [New Thread 0x75e69b600640 (LWP 185)] [New Thread 0x75e69ac00640 (LWP 186)] [New Thread 0x75e698200640 (LWP 187)] [New Thread 0x75e693800640 (LWP 188)] [New Thread 0x75e692e00640 (LWP 189)] [New Thread 0x75e68e400640 (LWP 190)] [New Thread 0x75e68ba00640 (LWP 191)] [New Thread 0x75e689000640 (LWP 192)] [New Thread 0x75e686600640 (LWP 193)] [New Thread 0x75e683c00640 (LWP 194)] [New Thread 0x75e683200640 (LWP 195)] [New Thread 0x75e67e800640 (LWP 196)] [New Thread 0x75e67be00640 (LWP 197)] [New Thread 0x75e679400640 (LWP 198)] [New Thread 0x75e676a00640 (LWP 199)] [New Thread 0x75e674000640 (LWP 200)] [New Thread 0x75e673600640 (LWP 201)] [New Thread 0x75e670c00640 (LWP 202)] Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider'] /home/root/model-artifacts/model Running shape inference on model model [New Thread 0x75e65f800640 (LWP 203)] [New Thread 0x75e65ee00640 (LWP 204)] [New Thread 0x75e65e400640 (LWP 205)] [New Thread 0x75e65da00640 (LWP 206)] [New Thread 0x75e65d000640 (LWP 207)] [New Thread 0x75e657e00640 (LWP 208)] [New Thread 0x75e657400640 (LWP 209)] [New Thread 0x75e656a00640 (LWP 210)] [New Thread 0x75e656000640 (LWP 211)] [New Thread 0x75e655600640 (LWP 212)] [New Thread 0x75e654c00640 (LWP 213)] [New Thread 0x75e64be00640 (LWP 214)] [New Thread 0x75e64b400640 (LWP 215)] tidl_tools_path = /home/root/tidl_tools artifacts_folder = /home/root/model-artifacts/model tidl_tensor_bits = 8 debug_level = 2 num_tidl_subgraphs = 16 tidl_denylist = tidl_denylist_layer_name = tidl_denylist_layer_type = tidl_allowlist_layer_name = model_type = tidl_calibration_accuracy_level = 7 tidl_calibration_options:num_frames_calibration = 2 tidl_calibration_options:bias_calibration_iterations = 5 mixed_precision_factor = -1.000000 model_group_id = 0 power_of_2_quantization = 2 ONNX QDQ Enabled = 0 enable_high_resolution_optimization = 0 pre_batchnorm_fold = 1 add_data_convert_ops = 3 output_feature_16bit_names_list = m_params_16bit_names_list = reserved_compile_constraints_flag = 1601 ti_internal_reserved_1 = ****** WARNING : Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options****** Supported TIDL layer type --- [...] Preliminary subgraphs created = 1 Final number of subgraphs created are : 1, - Offloaded Nodes - 124, Total Nodes - 124 [Detaching after vfork from child process 216] Running runtimes graphviz - /home/root/tidl_tools/tidl_graphVisualiser_runtimes.out /home/root/model-artifacts/model/allowedNode.txt /home/root/model-artifacts/model/tempDir/graphvizInfo.txt /home/root/model-artifacts/model/tempDir/runtimes_visualization.svg *** In TIDL_createStateImportFunc *** Compute on node : TIDLExecutionProvider_TIDL_0_0 [...] Input tensor name - input Output tensor name - 501 Output tensor name - output Output tensor name - 499 [New Thread 0x75e63be00640 (LWP 221)] [New Thread 0x75e63b400640 (LWP 222)] [New Thread 0x75e63aa00640 (LWP 223)] [New Thread 0x75e63a000640 (LWP 224)] [New Thread 0x75e639600640 (LWP 225)] [New Thread 0x75e638c00640 (LWP 226)] [New Thread 0x75e628200640 (LWP 227)] [New Thread 0x75e627800640 (LWP 228)] [New Thread 0x75e626e00640 (LWP 229)] [New Thread 0x75e626400640 (LWP 230)] [New Thread 0x75e625a00640 (LWP 231)] [New Thread 0x75e625000640 (LWP 232)] [New Thread 0x75e624600640 (LWP 233)] [New Thread 0x75e623c00640 (LWP 234)] [New Thread 0x75e623200640 (LWP 235)] [New Thread 0x75e622800640 (LWP 236)] [New Thread 0x75e621e00640 (LWP 237)] [New Thread 0x75e621400640 (LWP 238)] [New Thread 0x75e620a00640 (LWP 239)] Graph Domain TO version : 11In TIDL_onnxRtImportInit subgraph_name=499output501 Layer 0, subgraph id 499output501, name=501 Layer 1, subgraph id 499output501, name=output Layer 2, subgraph id 499output501, name=499 Layer 3, subgraph id 499output501, name=input In TIDL_runtimesOptimizeNet: LayerIndex = 128, dataIndex = 125 WARNING: [...] WARNING: [...] WARNING: [...] ************** Frame index 1 : Running float import ************* In TIDL_runtimesPostProcessNet In TIDL_runtimesPostProcessNet 1 In TIDL_runtimesPostProcessNet 2 In TIDL_runtimesPostProcessNet 3 [Detaching after vfork from child process 240] [Detaching after vfork from child process 242] **************************************************** ** ALL MODEL CHECK PASSED ** **************************************************** In TIDL_runtimesPostProcessNet 4 ************ in TIDL_subgraphRtCreate ************ TIDL_RT_OVX: Set default TIDLRT params done Calling appInit() in TIDL-RT! The soft limit is 2048 The hard limit is 2048 MEM: Init ... !!! MEM: Init ... Done !!! 0.0s: VX_ZONE_INIT:Enabled 0.5s: VX_ZONE_ERROR:Enabled 0.7s: VX_ZONE_WARNING:Enabled [New Thread 0x75e616a00640 (LWP 249)] [New Thread 0x75e616000640 (LWP 250)] [New Thread 0x75e615600640 (LWP 251)] [New Thread 0x75e614c00640 (LWP 252)] [New Thread 0x75e614200640 (LWP 253)] [New Thread 0x75e613800640 (LWP 254)] [New Thread 0x75e612e00640 (LWP 255)] [New Thread 0x75e612400640 (LWP 256)] [New Thread 0x75e611a00640 (LWP 257)] [New Thread 0x75e611000640 (LWP 258)] [New Thread 0x75e610600640 (LWP 259)] [New Thread 0x75e60fc00640 (LWP 260)] [New Thread 0x75e60f200640 (LWP 261)] [New Thread 0x75e60e800640 (LWP 262)] [New Thread 0x75e60de00640 (LWP 263)] [New Thread 0x75e60d400640 (LWP 264)] [New Thread 0x75e60ca00640 (LWP 265)] [New Thread 0x75e60c000640 (LWP 266)] [New Thread 0x75e60b600640 (LWP 267)] [New Thread 0x75e60ac00640 (LWP 268)] [New Thread 0x75e60a200640 (LWP 269)] [New Thread 0x75e609800640 (LWP 270)] [New Thread 0x75e608e00640 (LWP 271)] [New Thread 0x75e608400640 (LWP 272)] 0.6024s: VX_ZONE_INIT:[tivxInit:185] Initialization Done !!! TIDL_RT_OVX: Init ... TIDL_RT_OVX: Mapping config file ... TIDL_RT_OVX: Mapping config file ... Done. 37912 bytes TIDL_RT_OVX: Tensors, input = 1, output = 3 Host kernel - 0x75e649c0f658 TIDL_RT_OVX: Mapping network file TIDL_RT_OVX: Mapping network file... Done 97299008 bytes TIDL_RT_OVX: Init done. TIDL_RT_OVX: Creating graph ... TIDL_RT_OVX: input_sizes[0] = 896, dim = 224 padL = 0 padR = 0 TIDL_RT_OVX: input_sizes[1] = 200704, dim = 224 padT = 0 padB = 0 TIDL_RT_OVX: input_sizes[2] = 3, dim = 3 TIDL_RT_OVX: input_sizes[3] = 1, dim = 1 TIDL_RT_OVX: input_buffer = 0x75e683232000 150528 TIDL_RT_OVX: Creating graph ... Done. -------------------------------------------- TIDL Memory size requiement (record wise): MemRecNum , Space , Attribute , Alignment , Size(KBytes), BasePtr 0 , DDR Cacheable , Persistent , 128, 15.25 , 0x00000000 1 , DDR Cacheable , Persistent , 128, 0.64 , 0x00000000 2 , DDR Cacheable , Scratch , 128, 16.00 , 0x00000000 3 , DDR Cacheable , Scratch , 128, 4.00 , 0x00000000 4 , DDR Cacheable , Scratch , 128, 56.00 , 0x00000000 5 , DDR Cacheable , Persistent , 128, 930.75 , 0x00000000 6 , DDR Cacheable , Scratch , 128, 34549.12, 0x00000000 7 , DDR Cacheable , Scratch , 128, 0.12 , 0x00000000 8 , DDR Cacheable , Scratch , 128, 4873.25 , 0x00000000 9 , DDR Cacheable , Scratch , 128, 6500.50 , 0x00000000 10 , DDR Cacheable , Persistent , 128, 929.20 , 0x00000000 11 , DDR Cacheable , Scratch , 128, 512.25 , 0x00000000 12 , DDR Cacheable , Persistent , 128, 0.12 , 0x00000000 13 , DDR Cacheable , Persistent , 128, 95018.69, 0x00000000 14 , DDR Cacheable , Persistent , 128, 0.08 , 0x00000000 -------------------------------------------- Total memory size requirement (space wise): Mem Space , Size(KBytes) DDR Cacheable, 143405.98 -------------------------------------------- NOTE: Memory requirement in host emulation can be different from the same on EVM To get the actual TIDL memory requirement make sure to run on EVM with debugTraceLevel = 2 -------------------------------------------- TIDL init call from ivision API -------------------------------------------- TIDL Memory size requiement (record wise): MemRecNum , Space , Attribute , Alignment , Size(KBytes), BasePtr 0 , DDR Cacheable , Persistent , 128, 15.25 , 0x9ec8e000 1 , DDR Cacheable , Persistent , 128, 0.64 , 0xa1b89000 2 , DDR Cacheable , Scratch , 128, 16.00 , 0x9e009000 3 , DDR Cacheable , Scratch , 128, 4.00 , 0xa1b88000 4 , DDR Cacheable , Scratch , 128, 56.00 , 0x92e34000 5 , DDR Cacheable , Persistent , 128, 930.75 , 0x7be17000 6 , DDR Cacheable , Scratch , 128, 34549.12, 0xffd77000 7 , DDR Cacheable , Scratch , 128, 0.12 , 0xa1940000 8 , DDR Cacheable , Scratch , 128, 4873.25 , 0x5c33d000 9 , DDR Cacheable , Scratch , 128, 6500.50 , 0xff71d000 10 , DDR Cacheable , Persistent , 128, 929.20 , 0x79417000 11 , DDR Cacheable , Scratch , 128, 512.25 , 0x86609000 12 , DDR Cacheable , Persistent , 128, 0.12 , 0xa193f000 13 , DDR Cacheable , Persistent , 128, 95018.69, 0xf9a52000 14 , DDR Cacheable , Persistent , 128, 0.08 , 0xa1098000 -------------------------------------------- Total memory size requirement (space wise): Mem Space , Size(KBytes) DDR Cacheable, 143405.98 -------------------------------------------- NOTE: Memory requirement in host emulation can be different from the same on EVM To get the actual TIDL memory requirement make sure to run on EVM with debugTraceLevel = 2 -------------------------------------------- Alg Init for Layer # - 1 [...] PREEMPTION: Adding a new priority object for targetPriority = 0, handle = 0x75e69ec8e000 PREEMPTION: Now total number of priority objects = 1 at priorityId = 0, with new memRec of base = 0x75e6a193f000 and size = 128 PREEMPTION: Requesting context memory addr for handle 0x75e69ec8e000, return Addr = 0x75e64aa4e678 TIDL_RT_OVX: Verifying TIDL graph ... Done. ************ TIDL_subgraphRtCreate done ************ ******* In TIDL_subgraphRtInvoke ******** TIDL_RT_OVX: Set default TIDLRT tensor done TIDL_RT_OVX: Set default TIDLRT tensor done TIDL_RT_OVX: Set default TIDLRT tensor done TIDL_RT_OVX: Set default TIDLRT tensor done TIDL_RT_OVX: Running Graph ... TIDL_RT_OVX: input_sizes[0] = 896, dim = 224 padL = 0 padR = 0 TIDL_RT_OVX: input_sizes[1] = 200704, dim = 224 padT = 0 padB = 0 TIDL_RT_OVX: input_sizes[2] = 3, dim = 3 TIDL_RT_OVX: input_sizes[3] = 1, dim = 1 TIDL_RT_OVX : Memcpy Input Buffer TIDL_RT_OVX: input_buffer = 0x75e683232000 150528 TIDL_RT_OVX: memset_out_tensor_tidlrt_tiovx ... Done. TIDL_activate is called with handle : 9ec8e000 Core 0 Alg Process for Layer # [...] Thread 53 "python3" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x75e616a00640 (LWP 249)] 0x000075e64889109d in void TIDL_refInnerProductParamBitDepth(TIDL_Obj*, int, void*, void*, void*, float*, float*, float*, int, tidlInnerProductBuffParams_t*) [clone .isra.0] () from /home/root/tidl_tools/libvx_tidl_rt.so (gdb) backtrace #0 0x000075e64889109d in void TIDL_refInnerProductParamBitDepth(TIDL_Obj*, int, void*, void*, void*, float*, float*, float*, int, tidlInnerProductBuffParams_t*) [clone .isra.0] () from /home/root/tidl_tools/libvx_tidl_rt.so #1 0x000075e648896e75 in TIDL_innerProductRefProcess(TIDL_Obj*, sTIDL_AlgLayer_t*, sTIDL_Layer_t*, sTIDL_InnerProductParams_t*, tidlInnerProductBuffParams_t*, void*, void*, void*) () from /home/root/tidl_tools/libvx_tidl_rt.so #2 0x000075e648897f2c in TIDL_innerProductProcessNew(TIDL_NetworkCommonParams*, sTIDL_AlgLayer_t*, sTIDL_Layer_t*, void**, void**, int) () from /home/root/tidl_tools/libvx_tidl_rt.so #3 0x000075e6488dbae2 in WorkloadRefExec_Process(TIDL_Obj*, TIDL_NetworkCommonParams*, sWorkloadUnit_t*, sTIDL_AlgLayer_t*, sTIDL_Layer_t*, void**, void**, int, int) () from /home/root/tidl_tools/libvx_tidl_rt.so #4 0x000075e648838904 in TIDL_process(IVISION_Obj*, IVISION_BufDescList*, IVISION_BufDescList*, IVISION_InArgs*, IVISION_OutArgs*) () from /home/root/tidl_tools/libvx_tidl_rt.so #5 0x000075e648835f7a in tivxKernelTIDLProcess () from /home/root/tidl_tools/libvx_tidl_rt.so #6 0x000075e648824411 in ownTargetKernelExecute () from /home/root/tidl_tools/libvx_tidl_rt.so #7 0x000075e648822bb7 in ownTargetNodeDescNodeExecuteTargetKernel () from /home/root/tidl_tools/libvx_tidl_rt.so #8 0x000075e6488235d9 in ownTargetTaskMain () from /home/root/tidl_tools/libvx_tidl_rt.so #9 0x000075e64883137c in tivxTaskMain () from /home/root/tidl_tools/libvx_tidl_rt.so #10 0x000075e6a1efcac3 in start_thread (arg=) at ./nptl/pthread_create.c:442 #11 0x000075e6a1f8da04 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100