Command : python3 tflrt_delegate.py in Dir : examples/osrt_python/tfl Started Number of subgraphs:1 , 34 nodes delegated out of 34 nodes APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=6) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3291.623065 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3291.623156 s: VX_ZONE_INIT:Enabled 3291.623159 s: VX_ZONE_ERROR:Enabled 3291.623161 s: VX_ZONE_WARNING:Enabled 3291.626036 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3291.631264 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3292.655146 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3292.659497 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 7 alloc's of 7607170 bytes DDR_SHARED_MEM: Free's : 7 free's of 7607170 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Number of subgraphs:1 , 81 nodes delegated out of 81 nodes APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=6) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3292.972593 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3292.972609 s: VX_ZONE_INIT:Enabled 3292.972611 s: VX_ZONE_ERROR:Enabled 3292.972614 s: VX_ZONE_WARNING:Enabled 3292.972981 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3292.973005 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3311.052032 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3311.056373 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 7 alloc's of 7145076 bytes DDR_SHARED_MEM: Free's : 7 free's of 7145076 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Number of subgraphs:1 , 107 nodes delegated out of 107 nodes APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=6) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3311.732352 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3311.732365 s: VX_ZONE_INIT:Enabled 3311.732368 s: VX_ZONE_ERROR:Enabled 3311.732370 s: VX_ZONE_WARNING:Enabled 3311.732772 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3311.732798 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3322.899088 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3322.903436 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 10 alloc's of 20444424 bytes DDR_SHARED_MEM: Free's : 10 free's of 20444424 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Number of subgraphs:1 , 129 nodes delegated out of 129 nodes APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=6) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3323.390885 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3323.390899 s: VX_ZONE_INIT:Enabled 3323.390901 s: VX_ZONE_ERROR:Enabled 3323.390904 s: VX_ZONE_WARNING:Enabled 3323.391283 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3323.391308 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3334.558246 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115]Running 4 Models - ['cl-tfl-mobilenet_v1_1.0_224', 'ss-tfl-deeplabv3_mnv2_ade20k_float', 'od-tfl-ssd_mobilenet_v2_300_float', 'od-tfl-ssdlite_mobiledet_dsp_320x320_coco'] Running_Model : cl-tfl-mobilenet_v1_1.0_224 , 0 0.691726 warplane, military plane ,, 1 0.181373 missile ,, 2 0.109571 projectile, missile ,, 3 0.006352 cannon ,, 4 0.005370 aircraft carrier, carrier, flattop, attack aircraft carrier , Saving image to ../../../output_images/ Completed_Model : 1, Name : cl-tfl-mobilenet_v1_1.0_224 , Total time : 1.72, Offload Time : 1.70 , DDR RW MBs : 18446744073709.52, Output File : py_out_cl-tfl-mobilenet_v1_1.0_224_airshow.jpg Running_Model : ss-tfl-deeplabv3_mnv2_ade20k_float Saving image to ../../../output_images/ Completed_Model : 2, Name : ss-tfl-deeplabv3_mnv2_ade20k_float , Total time : 10.55, Offload Time : 10.54 , DDR RW MBs : 18446744073709.52, Output File : py_out_ss-tfl-deeplabv3_mnv2_ade20k_float_ADE_val_00001801.jpg Running_Model : od-tfl-ssd_mobilenet_v2_300_float Saving image to ../../../output_images/ Completed_Model : 3, Name : od-tfl-ssd_mobilenet_v2_300_float , Total time : 6.35, Offload Time : 6.33 , DDR RW MBs : 18446744073709.52, Output File : py_out_od-tfl-ssd_mobilenet_v2_300_float_ADE_val_00001801.jpg Running_Model : od-tfl-ssdlite_mobiledet_dsp_320x320_coco Saving image to ../../../output_images/ Completed_Model : 4, Name : od-tfl-ssdlite_mobiledet_dsp_320x320_coco , Total time : 5.61, Offload Time : 5.59 , DDR RW MBs : 18446744073709.52, Output File : py_out_od-tfl-ssdlite_mobiledet_dsp_320x320_coco_ADE_val_00001801.jpg De-Initialization Done for HOST !!! 3334.559806 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 10 alloc's of 10800264 bytes DDR_SHARED_MEM: Free's : 10 free's of 10800264 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Command : python3 onnxrt_ep.py in Dir : examples/osrt_python/ort Started Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider'] Running 3 Models - ['cl-ort-resnet18-v1', 'ss-ort-deeplabv3lite_mobilenetv2', 'od-ort-ssd-lite_mobilenetv2_fpn'] Running_Model : cl-ort-resnet18-v1 , 0 23.847309 warplane, military plane ,, 1 22.507572 aircraft carrier, carrier, flattop, attack aircraft carrier ,, 2 19.024258 projectile, missile ,, 3 18.756310 missile ,, 4 15.808890 airliner , Saving image to ../../../output_images/ Completed_Model : 1, Name : cl-ort-resnet18-v1 , Total time : 2.62, Offload Time : 2.51 , DDR RW MBs : 0, Output File : py_out_cl-ort-resnet18-v1_airshow.jpg Running_Model : ss-ort-deeplabv3lite_mobilenetv2 Saving image to ../../../output_images/ Completed_Model : 2, Name : ss-ort-deeplabv3lite_mobilenetv2 , Total time : 9.39, Offload Time : 9.22 , DDR RW MBs : 0, Output File : py_out_ss-ort-deeplabv3lite_mobilenetv2_ADE_val_00001801.jpg Running_Model : od-ort-ssd-lite_mobilenetv2_fpn 2 Saving image to ../../../output_images/ Completed_Model : 3, Name : od-ort-ssd-lite_mobilenetv2_fpn , Total time : 6.34, Offload Time : 6.10 , DDR RW MBs : 0, Output File : py_out_od-ort-ssd-lite_mobilenetv2_fpn_ADE_val_00001801.jpg libtidl_onnxrt_EP loaded 0x17955000 Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=5) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3337.565660 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3337.565711 s: VX_ZONE_INIT:Enabled 3337.565714 s: VX_ZONE_ERROR:Enabled 3337.565716 s: VX_ZONE_WARNING:Enabled 3337.566789 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3337.566926 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3338.613612 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3338.617997 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 7 alloc's of 15581092 bytes DDR_SHARED_MEM: Free's : 7 free's of 15581092 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! libtidl_onnxrt_EP loaded 0x17955000 Final number of subgraphs created are : 1, - Offloaded Nodes - 124, Total Nodes - 124 APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=5) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3339.174326 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3339.174338 s: VX_ZONE_INIT:Enabled 3339.174341 s: VX_ZONE_ERROR:Enabled 3339.174343 s: VX_ZONE_WARNING:Enabled 3339.174927 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3339.174952 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3355.570767 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3355.575163 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 7 alloc's of 7296436 bytes DDR_SHARED_MEM: Free's : 7 free's of 7296436 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! libtidl_onnxrt_EP loaded 0x17955000 Final number of subgraphs created are : 1, - Offloaded Nodes - 482, Total Nodes - 482 APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=5) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3356.537065 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3356.537077 s: VX_ZONE_INIT:Enabled 3356.537080 s: VX_ZONE_ERROR:Enabled 3356.537082 s: VX_ZONE_WARNING:Enabled 3356.537509 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3356.537538 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3369.445315 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3369.449703 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 8 alloc's of 7256996 bytes DDR_SHARED_MEM: Free's : 8 free's of 7256996 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Command : python3 dlr_inference_example.py in Dir : examples/osrt_python/tvm_dlr Started 2024-02-19 14:13:45,145 INFO Could not find libdlr.so in model artifact. Using dlr from /usr/dlr/libdlr.so APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=5) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3376.394962 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3376.395017 s: VX_ZONE_INIT:Enabled 3376.395020 s: VX_ZONE_ERROR:Enabled 3376.395023 s: VX_ZONE_WARNING:Enabled 3376.395726 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3376.396257 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! 3376.618019 s: VX_ZONE_ERROR:[ownContextSendCmd:822] Command ack message returned failure cmd_status: -1 3376.618027 s: VX_ZONE_ERROR:[ownContextSendCmd:862] tivxEventWait() failed. 3376.618031 s: VX_ZONE_ERROR:[ownNodeKernelInit:584] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode 3376.618033 s: VX_ZONE_ERROR:[ownNodeKernelInit:585] Please be sure the target callbacks have been registered for this core 3376.618036 s: VX_ZONE_ERROR:[ownNodeKernelInit:586] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel 3376.618040 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl:1:1 ... failed !!! 3376.618055 s: VX_ZONE_ERROR:[vxVerifyGraph:2059] Node kernel init failed 3376.618058 s: VX_ZONE_ERROR:[vxVerifyGraph:2113] Graph verify failed TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!! TIDL_RT_OVX: ERROR: Verify OpenVX graph failed 3376.637550 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3376.641913 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 7 alloc's of 28308270 bytes DDR_SHARED_MEM: Free's : 7 free's of 28308270 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! [14:13:51] /home/kumar/work/tvm_dlr_91/neo-ai-dlr/src/dlr.cc:343: Error: [14:13:51] /home/kumar/work/tvm_dlr_91/neo-ai-dlr/3rdparty/tvm/src/runtime/contrib/tidl/tidl_runtime.cc:260: Failed to initialize TIDLRT for subgraph 0 Stack trace: [bt] (0) /usr/dlr/libdlr.so(+0x13d844) [0xffff8534d844] [bt] (1) /usr/dlr/libdlr.so(tvm::runtime::detail::LogFatal::Entry::Finalize()+0x6c) [0xffff8524e8fc] [bt] (2) /usr/dlr/libdlr.so(tvm::runtime::TIDLJ7Module::GetFunction(std::__cxx11::basic_string, std::allocator > const&, tvm::runtime::ObjectPtr const&)+0x7e4) [0xffff8531794c] [bt] (3) /usr/dlr/libdlr.so(+0x140084) [0xffff85350084] [bt] (4) /usr/dlr/libdlr.so(tvm::runtime::ConstLoaderModuleNode::GetFunction(std::__cxx11::basic_string, std::allocator > const&, tvm::runtime::ObjectPtr const&)+0x328) [0xffff85265750] [bt] (5) /usr/dlr/libdlr.so(+0x140084) [0xffff85350084] [bt] (6) /usr/dlr/libdlr.so(+0x1a5f00) [0xffff853b5f00] [bt] (7) /usr/dlr/libdlr.so(+0x1a9aa0) [0xffff853b9aa0] [bt] (8) /usr/dlr/libdlr.so(+0x1ab27c) [0xffff853bb27c] 2024-02-19 14:13:51,597 ERROR error in DLRModel instantiation Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/dlr/api.py", line 89, in __init__ self._impl = DLRModelImpl(model_path, dev_type, dev_id, error_log_file, use_default_dlr) File "/usr/lib/python3.10/site-packages/dlr/dlr_model.py", line 79, in __init__ self._check_call(self._lib.CreateDLRModel(byref(self.handle), File "/usr/lib/python3.10/site-packages/dlr/dlr_model.py", line 160, in _check_call raise DLRError(self._lib.DLRGetLastError().decode('ascii')) dlr.dlr_model.DLRError Running Inference on Model - ../../../model-artifacts/cl-dlr-tflite_inceptionnetv3_device Traceback (most recent call last): File "/home/projects/edgeai-tidl-tools/examples/osrt_python/tvm_dlr/dlr_inference_example.py", line 199, in model_create_and_run(model_output_directory, 'input', File "/home/projects/edgeai-tidl-tools/examples/osrt_python/tvm_dlr/dlr_inference_example.py", line 164, in model_create_and_run model = DLRModel(model_dir, 'cpu') File "/usr/lib/python3.10/site-packages/dlr/api.py", line 92, in __init__ raise ex File "/usr/lib/python3.10/site-packages/dlr/api.py", line 89, in __init__ self._impl = DLRModelImpl(model_path, dev_type, dev_id, error_log_file, use_default_dlr) File "/usr/lib/python3.10/site-packages/dlr/dlr_model.py", line 79, in __init__ self._check_call(self._lib.CreateDLRModel(byref(self.handle), File "/usr/lib/python3.10/site-packages/dlr/dlr_model.py", line 160, in _check_call raise DLRError(self._lib.DLRGetLastError().decode('ascii')) dlr.dlr_model.DLRError Command : bash run_tfl_models.sh in Dir : scripts/osrt_cpp_scripts/ Started cl-tfl-mobilenet_v1_1.0_224 ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/cl-tfl-mobilenet_v1_1.0_224/ image path set to: test_data/airshow.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [14:13:51.000.000000]:INFO:[runInference:0199] Loaded model model-artifacts/cl-tfl-mobilenet_v1_1.0_224//mobilenet_v1_1.0_224.tflite [14:13:51.000.000029]:INFO:[runInference:0201] resolved reporter [14:13:51.089.089590]:INFO:[runInference:0213] tensors size: 93 [14:13:51.089.089621]:INFO:[runInference:0214] nodes size: 34 [14:13:51.089.089629]:INFO:[runInference:0215] number of inputs: 1 [14:13:51.089.089636]:INFO:[runInference:0216] number of outputs: 1 [14:13:51.089.089641]:INFO:[runInference:0217] input(0) name: input [14:13:51.089.089649]:INFO:[runInference:0246] input: 87 [14:13:51.089.089655]:INFO:[runInference:0252] TIDL delegate mode Number of subgraphs:1 , 34 nodes delegated out of 34 nodes APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=6) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3377.130821 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3377.130897 s: VX_ZONE_INIT:Enabled 3377.130900 s: VX_ZONE_ERROR:Enabled 3377.130902 s: VX_ZONE_WARNING:Enabled 3377.131672 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3377.131832 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! [14:13:52.255.255482]:INFO:[runInference:0262] ModifyGraphWithDelegate - Done [14:13:52.255.255519]:INFO:[runInference:0309] device mem enabled [14:13:52.283.283607]:INFO:[preprocImage:0115] template NHWC [14:13:52.284.284024]:INFO:[runInference:0382] interpreter->Invoke - Started [14:13:52.320.320843]:INFO:[runInference:0402] interpreter->Invoke - Done [14:13:52.320.320872]:INFO:[runInference:0404] average time:1.621300 ms [14:13:52.320.320881]:INFO:[prepClassificationResult:0127] preparing clasification result [14:13:52.332.332476]:INFO:[prepClassificationResult:0169] 0.705051: 896 :warplane, military plane [14:13:52.332.332501]:INFO:[prepClassificationResult:0169] 0.184867: 658 :missile [14:13:52.332.332510]:INFO:[prepClassificationResult:0169] 0.094558: 745 :projectile, missile [14:13:52.332.332520]:INFO:[prepClassificationResult:0169] 0.006475: 472 :cannon [14:13:52.332.332527]:INFO:[prepClassificationResult:0169] 0.003915: 404 :aircraft carrier, carrier, flattop, attack aircraft carrier [14:13:52.336.336160]:INFO:[runInference:0491] saving image result file [14:13:52.342.342662]:INFO:[runInference:0541] Completed_Model : 0, Name : cl-tfl-mobilenet_v1_1.0_224, Total time : 1.621300, Offload Time : 0 , DDR RW MBs : 0, Output File : cpp_out_cl-tfl-mobilenet_v1_1.0_224.jpg 3377.283757 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3377.288171 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 9 alloc's of 7761830 bytes DDR_SHARED_MEM: Free's : 9 free's of 7761830 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! od-tfl-ssd_mobilenet_v2_300_float ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/od-tfl-ssd_mobilenet_v2_300_float/ image path set to: test_data/ADE_val_00001801.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [14:13:52.000.000000]:INFO:[runInference:0199] Loaded model model-artifacts/od-tfl-ssd_mobilenet_v2_300_float//ssd_mobilenet_v2_300_float.tflite [14:13:52.000.000030]:INFO:[runInference:0201] resolved reporter [14:13:52.136.136311]:INFO:[runInference:0213] tensors size: 272 [14:13:52.136.136347]:INFO:[runInference:0214] nodes size: 107 [14:13:52.136.136356]:INFO:[runInference:0215] number of inputs: 1 [14:13:52.136.136363]:INFO:[runInference:0216] number of outputs: 4 [14:13:52.136.136369]:INFO:[runInference:0217] input(0) name: normalized_input_image_tensor [14:13:52.136.136377]:INFO:[runInference:0246] input: 0 [14:13:52.136.136383]:INFO:[runInference:0252] TIDL delegate mode Number of subgraphs:1 , 107 nodes delegated out of 107 nodes APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=6) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3377.591268 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3377.591347 s: VX_ZONE_INIT:Enabled 3377.591350 s: VX_ZONE_ERROR:Enabled 3377.591352 s: VX_ZONE_WARNING:Enabled 3377.592128 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3377.592878 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! [14:13:52.343.343500]:INFO:[runInference:0262] ModifyGraphWithDelegate - Done [14:13:52.343.343544]:INFO:[runInference:0309] device mem enabled [14:13:52.434.434367]:INFO:[preprocImage:0115] template NHWC [14:13:52.435.435108]:INFO:[runInference:0382] interpreter->Invoke - Started [14:13:52.569.569312]:INFO:[runInference:0402] interpreter->Invoke - Done [14:13:52.569.569345]:INFO:[runInference:0404] average time:6.055250 ms [14:13:52.569.569355]:INFO:[runInference:0428] detected objects:2 [14:13:52.569.569393]:INFO:[prepDetectionResult:0448] preparing detection result [14:13:52.569.569410]:INFO:[prepDetectionResult:0513] box with score:0.684622, threshold set to:0.300000 [14:13:52.569.569419]:INFO:[prepDetectionResult:0513] box with score:0.499985, threshold set to:0.300000 [14:13:52.569.569488]:INFO:[runInference:0491] saving image result file [14:13:52.575.575973]:INFO:[runInference:0541] Completed_Model : 0, Name : od-tfl-ssd_mobilenet_v2_300_float, Total time : 6.055250, Offload Time : 0 , DDR RW MBs : 0, Output File : cpp_out_od-tfl-ssd_mobilenet_v2_300_float.jpg 3377.917813 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3377.922239 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 15 alloc's of 20714988 bytes DDR_SHARED_MEM: Free's : 15 free's of 20714988 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! ss-tfl-deeplabv3_mnv2_ade20k_float ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/ image path set to: test_data/ADE_val_00001801.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [14:13:52.000.000000]:INFO:[runInference:0199] Loaded model model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float//deeplabv3_mnv2_ade20k_float.tflite [14:13:52.000.000031]:INFO:[runInference:0201] resolved reporter [14:13:53.088.088515]:INFO:[runInference:0213] tensors size: 208 [14:13:53.088.088546]:INFO:[runInference:0214] nodes size: 81 [14:13:53.088.088553]:INFO:[runInference:0215] number of inputs: 1 [14:13:53.088.088560]:INFO:[runInference:0216] number of outputs: 1 [14:13:53.088.088566]:INFO:[runInference:0217] input(0) name: MobilenetV2/MobilenetV2/input [14:13:53.088.088574]:INFO:[runInference:0246] input: 0 [14:13:53.088.088581]:INFO:[runInference:0252] TIDL delegate mode Number of subgraphs:1 , 81 nodes delegated out of 81 nodes APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=6) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3378.163452 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3378.163528 s: VX_ZONE_INIT:Enabled 3378.163531 s: VX_ZONE_ERROR:Enabled 3378.163534 s: VX_ZONE_WARNING:Enabled 3378.164908 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3378.164986 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! [14:13:53.237.237969]:INFO:[runInference:0262] ModifyGraphWithDelegate - Done [14:13:53.238.238005]:INFO:[runInference:0309] device mem enabled [14:13:53.304.304965]:INFO:[preprocImage:0115] template NHWC [14:13:53.307.307059]:INFO:[runInference:0382] interpreter->Invoke - Started [14:13:53.534.534572]:INFO:[runInference:0402] interpreter->Invoke - Done [14:13:53.534.534611]:INFO:[runInference:0404] average time:10.287300 ms [14:13:53.534.534621]:INFO:[prepSegResult:0086] preparing segmentation result [14:13:53.537.537399]:INFO:[runInference:0491] saving image result file [14:13:53.543.543698]:INFO:[runInference:0541] Completed_Model : 0, Name : ss-tfl-deeplabv3_mnv2_ade20k_float, Total time : 10.287300, Offload Time : 0 , DDR RW MBs : 0, Output File : cpp_out_ss-tfl-deeplabv3_mnv2_ade20k_float.jpg 3378.515964 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3378.519378 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 9 alloc's of 8193780 bytes DDR_SHARED_MEM: Free's : 9 free's of 8193780 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! /home/projects/edgeai-tidl-tools/scripts/osrt_cpp_scripts Command : bash run_onnx_models.sh in Dir : scripts/osrt_cpp_scripts/ Started cl-ort-resnet18-v1 ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/cl-ort-resnet18-v1/ image path set to: test_data/airshow.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [14:13:53.000.000000]:INFO:[runInference:0316] accelerated mode [14:13:53.002.002385]:INFO:[runInference:0319] artifacts: model-artifacts/cl-ort-resnet18-v1/ libtidl_onnxrt_EP loaded 0x23e28120 Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=5) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 3380.362452 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 3380.362532 s: VX_ZONE_INIT:Enabled 3380.362536 s: VX_ZONE_ERROR:Enabled 3380.362538 s: VX_ZONE_WARNING:Enabled 3380.363328 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!! 3380.364060 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! [14:13:55.1738.738320]:INFO:[runInference:0342] Loaded model model-artifacts/cl-ort-resnet18-v1//resnet18_opset9.onnx [14:13:55.1738.738394]:INFO:[printTensorInfo:0216] number of inputs:1 [14:13:55.1738.738402]:INFO:[printTensorInfo:0217] number of outputs: 1 [14:13:55.1738.738409]:INFO:[printTensorInfo:0218] input(0) name: input.1Net_IN [14:13:55.1738.738414]:INFO:[printTensorInfo:0223] Input 0 : name=input.1Net_IN [14:13:55.1738.738423]:INFO:[printTensorInfo:0230] Input 0 : type=2 [14:13:55.1738.738430]:INFO:[printTensorInfo:0233] Input 0 : num_dims=4 [14:13:55.1738.738437]:INFO:[printTensorInfo:0236] Input 0 : dim 0=1 [14:13:55.1738.738443]:INFO:[printTensorInfo:0236] Input 0 : dim 1=3 [14:13:55.1738.738448]:INFO:[printTensorInfo:0236] Input 0 : dim 2=224 [14:13:55.1738.738455]:INFO:[printTensorInfo:0236] Input 0 : dim 3=224 [14:13:55.1744.744697]:INFO:[preprocImage:0127] template NCHW [14:13:55.1797.797631]:INFO:[runInference:0474] invoked [14:13:55.1797.797663]:INFO:[runInference:0475] average time: 2.445450 ms [14:13:55.1797.797671]:INFO:[prepClassificationResult:0093] preparing classification result [14:13:55.1801.801873]:INFO:[prepClassificationResult:0137] 22.507572: 895 warplane, military plane [14:13:55.1801.801897]:INFO:[prepClassificationResult:0137] 21.435783: 403 aircraft carrier, carrier, flattop, attack aircraft carrier [14:13:55.1801.801906]:INFO:[prepClassificationResult:0137] 16.880678: 744 projectile, missile [14:13:55.1801.801918]:INFO:[prepClassificationResult:0137] 16.880678: 404 airliner [14:13:55.1801.801925]:INFO:[prepClassificationResult:0137] 16.612732: 657 missile [14:13:55.1808.808135]:INFO:[runInference:0607] Completed_Model : 0, Name : cl-ort-resnet18-v1, Total time : 2.445450, Offload Time : 0 , DDR RW MBs : 0, Output File : cpp_out_cl-ort-resnet18-v1.jpg 3380.501550 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 3380.506040 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 10 alloc's of 15739812 bytes DDR_SHARED_MEM: Free's : 10 free's of 15739812 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!! od-ort-ssd-lite_mobilenetv2_fpn