/data/user/edgeai-tidl-tools # python3 ./scripts/gen_test_report.py Command : python3 tflrt_delegate.py in Dir : examples/osrt_python/tfl Started Number of subgraphs:1 , 34 nodes delegated out of 34 nodes APP: Init ... !!! 667.770899 s: MEM: Init ... !!! 667.770947 s: MEM: Initialized DMA HEAP (fd=7) !!! 667.771128 s: MEM: Init ... Done !!! 667.771144 s: IPC: Init ... !!! 667.774794 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 667.779866 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 667.780035 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 667.780044 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 667.780049 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 667.781188 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 667.781490 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 667.781770 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 667.782040 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 667.782053 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 667.782076 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 670.158299 s: IPC: Deinit ... !!! 670.158863 s: IPC: DeInit ... Done !!! 670.158885 s: MEM: Deinit ... !!! 670.158895 s: DDR_SHARED_MEM: Alloc's: 7 alloc's of 6201004 bytes 670.158899 s: DDR_SHARED_MEM: Free's : 7 free's of 6201004 bytes 670.158904 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 670.158914 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Number of subgraphs:1 , 81 nodes delegated out of 81 nodes APP: Init ... !!! 670.174947 s: MEM: Init ... !!! 670.174998 s: MEM: Initialized DMA HEAP (fd=7) !!! 670.175138 s: MEM: Init ... Done !!! 670.175144 s: IPC: Init ... !!! 670.178775 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 670.182616 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 670.182635 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 670.182640 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 670.182644 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 670.183005 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 670.183140 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 670.183223 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 670.183286 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 670.183292 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 670.183297 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 717.920326 s: IPC: Deinit ... !!! 717.921205 s: IPC: DeInit ... Done !!! 717.921218 s: MEM: Deinit ... !!! 717.921222 s: DDR_SHARED_MEM: Alloc's: 7 alloc's of 5801340 bytes 717.921227 s: DDR_SHARED_MEM: Free's : 7 free's of 5801340 bytes 717.921230 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 717.921241 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Number of subgraphs:1 , 107 nodes delegated out of 107 nodes APP: Init ... !!! 718.103717 s: MEM: Init ... !!! 718.103773 s: MEM: Initialized DMA HEAP (fd=7) !!! 718.103915 s: MEM: Init ... Done !!! 718.103923 s: IPC: Init ... !!! 718.107471 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 718.111379 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 718.111399 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 718.111404 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 718.111408 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 718.111816 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 718.111933 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 718.112013 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 718.112074 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 718.112080 s: VX_ZONE_INFO: [tivxInRunning 4 Models - ['cl-tfl-mobilenet_v1_1.0_224', 'ss-tfl-deeplabv3_mnv2_ade20k_float', 'od-tfl-ssd_mobilenet_v2_300_float', 'od-tfl-ssdlite_mobiledet_dsp_320x320_coco'] Running_Model : cl-tfl-mobilenet_v1_1.0_224 , 0 0.726901 warplane, military plane ,, 1 0.160935 missile ,, 2 0.097488 projectile, missile ,, 3 0.005643 aircraft carrier, carrier, flattop, attack aircraft carrier ,, 4 0.004772 cannon , Saving image to ../../../output_images/ Saving output tensor to ../../../output_binaries/ Completed_Model : 1, Name : cl-tfl-mobilenet_v1_1.0_224 , Total time : 4.50, Offload Time : 4.47 , DDR RW MBs : 7, Output Image File : py_out_cl-tfl-mobilenet_v1_1.0_224_airshow.jpg, Output Bin File : py_out_cl-tfl-mobilenet_v1_1.0_224_airshow.bin Running_Model : ss-tfl-deeplabv3_mnv2_ade20k_float Saving image to ../../../output_images/ Saving output tensor to ../../../output_binaries/ Completed_Model : 2, Name : ss-tfl-deeplabv3_mnv2_ade20k_float , Total time : 40.21, Offload Time : 40.17 , DDR RW MBs : 150, Output Image File : py_out_ss-tfl-deeplabv3_mnv2_ade20k_float_ADE_val_00001801.jpg, Output Bin File : py_out_ss-tfl-deeplabv3_mnv2_ade20k_float_ADE_val_00001801.bin Running_Model : od-tfl-ssd_mobilenet_v2_300_float Saving image to ../../../output_images/ Saving output tensor to ../../../output_binaries/ Completed_Model : 3, Name : od-tfl-ssd_mobilenet_v2_300_float , Total time : 22.98, Offload Time : 22.94 , DDR RW MBs : 77, Output Image File : py_out_od-tfl-ssd_mobilenet_v2_300_float_ADE_val_00001801.jpg, Output Bin File : py_out_od-tfl-ssd_mobilenet_v2_300_float_ADE_val_00001801.bin Running_Model : od-tfl-ssdlite_mobiledet_dsp_320x320_coco Saving image to ../../../output_images/ Saving output tensor to ../../../output_binaries/ Completed_Model : 4, Name : od-tfl-ssdlite_mobiledet_dsp_320x320_coco , Total time : 14.14, Offload Time : 14.10 , DDR RW MBs : 21, Output Image File : py_out_od-tfl-ssdlite_mobiledet_dsp_320x320_coco_ADE_val_00001801.jpg, Output Bin File : py_out_od-tfl-ssdlite_mobiledet_dsp_320x320_coco_ADE_val_00001801.bin itLocal:202] Initialization Done !!! 718.112129 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 748.540884 s: IPC: Deinit ... !!! 748.541350 s: IPC: DeInit ... Done !!! 748.541362 s: MEM: Deinit ... !!! 748.541367 s: DDR_SHARED_MEM: Alloc's: 10 alloc's of 19216528 bytes 748.541372 s: DDR_SHARED_MEM: Free's : 10 free's of 19216528 bytes 748.541376 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 748.541385 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Number of subgraphs:1 , 129 nodes delegated out of 129 nodes APP: Init ... !!! 748.670847 s: MEM: Init ... !!! 748.670915 s: MEM: Initialized DMA HEAP (fd=7) !!! 748.671068 s: MEM: Init ... Done !!! 748.671075 s: IPC: Init ... !!! 748.674969 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 748.679230 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 748.679253 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 748.679257 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 748.679261 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 748.679677 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 748.679818 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 748.679945 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 748.680050 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 748.680056 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 748.680061 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 778.655930 s: IPC: Deinit ... !!! 778.656581 s: IPC: DeInit ... Done !!! 778.656596 s: MEM: Deinit ... !!! 778.656601 s: DDR_SHARED_MEM: Alloc's: 10 alloc's of 9624072 bytes 778.656606 s: DDR_SHARED_MEM: Free's : 10 free's of 9624072 bytes 778.656610 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 778.656624 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Command : python3 onnxrt_ep.py in Dir : examples/osrt_python/ort Started Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider'] Running 2 Models - ['cl-ort-resnet18-v1', 'od-ort-ssd-lite_mobilenetv2_fpn'] Running_Model : cl-ort-resnet18-v1 , 0 23.847309 warplane, military plane ,, 1 22.507572 aircraft carrier, carrier, flattop, attack aircraft carrier ,, 2 19.024258 projectile, missile ,, 3 18.756310 missile ,, 4 15.808890 airliner , Saving image to ../../../output_images/ Saving output tensor to ../../../output_binaries/ Completed_Model : 1, Name : cl-ort-resnet18-v1 , Total time : 6.73, Offload Time : 6.54 , DDR RW MBs : 14, Output Image File : py_out_cl-ort-resnet18-v1_airshow.jpg, Output Bin File : py_out_cl-ort-resnet18-v1_airshow.bin Running_Model : od-ort-ssd-lite_mobilenetv2_fpn 2 Saving image to ../../../output_images/ Saving output tensor to ../../../output_binaries/ Completed_Model : 2, Name : od-ort-ssd-lite_mobilenetv2_fpn , Total time : 31.92, Offload Time : 31.53 , DDR RW MBs : 76, Output Image File : py_out_od-ort-ssd-lite_mobilenetv2_fpn_ADE_val_00001801.jpg, Output Bin File : py_out_od-ort-ssd-lite_mobilenetv2_fpn_ADE_val_00001801.bin libtidl_onnxrt_EP loaded 0x1d466330 Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 APP: Init ... !!! 782.065151 s: MEM: Init ... !!! 782.065213 s: MEM: Initialized DMA HEAP (fd=5) !!! 782.065408 s: MEM: Init ... Done !!! 782.065434 s: IPC: Init ... !!! 782.069469 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 782.074849 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 782.075012 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 782.075020 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 782.075025 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 782.076161 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 782.076651 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 782.076987 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 782.077272 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 782.077284 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 782.077316 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 784.680611 s: IPC: Deinit ... !!! 784.681185 s: IPC: DeInit ... Done !!! 784.681209 s: MEM: Deinit ... !!! 784.681219 s: DDR_SHARED_MEM: Alloc's: 7 alloc's of 13664460 bytes 784.681224 s: DDR_SHARED_MEM: Free's : 7 free's of 13664460 bytes 784.681228 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 784.681240 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! libtidl_onnxrt_EP loaded 0x1d466330 Final number of subgraphs created are : 1, - Offloaded Nodes - 478, Total Nodes - 478 APP: Init ... !!! 786.244378 s: MEM: Init ... !!! 786.244440 s: MEM: Initialized DMA HEAP (fd=5) !!! 786.244716 s: MEM: Init ... Done !!! 786.244730 s: IPC: Init ... !!! 786.248420 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 786.252860 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 786.252889 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 786.252893 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 786.252898 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 786.253412 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 786.253585 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 786.254144 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 786.254305 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 786.254314 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 786.254319 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 823.566623 s: IPC: Deinit ... !!! 823.567115 s: IPC: DeInit ... Done !!! 823.567127 s: MEM: Deinit ... !!! 823.567132 s: DDR_SHARED_MEM: Alloc's: 8 alloc's of 6224532 bytes 823.567137 s: DDR_SHARED_MEM: Free's : 8 free's of 6224532 bytes 823.567140 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 823.567150 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! Command : bash run_tfl_models.sh in Dir : scripts/osrt_cpp_scripts/ Started cl-tfl-mobilenet_v1_1.0_224 ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/cl-tfl-mobilenet_v1_1.0_224/artifacts/ image path set to: test_data/airshow.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [06:00:30.000.000000]:INFO:[runInference:0231] Loaded model model-artifacts/cl-tfl-mobilenet_v1_1.0_224//model/mobilenet_v1_1.0_224.tflite [06:00:30.000.000074]:INFO:[runInference:0233] resolved reporter [06:00:30.045.045609]:INFO:[runInference:0245] tensors size: 93 [06:00:30.045.045688]:INFO:[runInference:0246] nodes size: 34 [06:00:30.045.045706]:INFO:[runInference:0247] number of inputs: 1 [06:00:30.045.045724]:INFO:[runInference:0248] number of outputs: 1 [06:00:30.045.045741]:INFO:[runInference:0249] input(0) name: input [06:00:30.045.045763]:INFO:[runInference:0278] input: 87 [06:00:30.045.045782]:INFO:[runInference:0284] TIDL delegate mode Number of subgraphs:1 , 34 nodes delegated out of 34 nodes APP: Init ... !!! 824.309146 s: MEM: Init ... !!! 824.309280 s: MEM: Initialized DMA HEAP (fd=6) !!! 824.309551 s: MEM: Init ... Done !!! 824.309581 s: IPC: Init ... !!! 824.316389 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 824.322823 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 824.323015 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 824.323026 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 824.323032 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 824.324155 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 824.324457 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 824.324802 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 824.325020 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 824.325028 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 824.325059 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO [06:00:30.169.169233]:INFO:[runInference:0294] ModifyGraphWithDelegate - Done [06:00:30.169.169321]:INFO:[runInference:0341] device mem enabled [06:00:30.208.208857]:INFO:[preprocImage:0115] template NHWC [06:00:30.211.211911]:INFO:[runInference:0414] interpreter->Invoke - Started [06:00:30.328.328939]:INFO:[runInference:0434] interpreter->Invoke - Done [06:00:30.329.329019]:INFO:[runInference:0436] average time:5.210250 ms [06:00:30.329.329110]:INFO:[prepClassificationResult:0147] preparing clasification result [06:00:30.342.342347]:INFO:[prepClassificationResult:0201] 0.693905: 896 :warplane, military plane [06:00:30.342.342508]:INFO:[prepClassificationResult:0201] 0.181945: 658 :missile [06:00:30.342.342540]:INFO:[prepClassificationResult:0201] 0.109916: 745 :projectile, missile [06:00:30.342.342559]:INFO:[prepClassificationResult:0201] 0.005386: 472 :cannon [06:00:30.342.342577]:INFO:[prepClassificationResult:0201] 0.004556: 404 :aircraft carrier, carrier, flattop, attack aircraft carrier [06:00:30.344.344794]:INFO:[runInference:0559] saving image result file [06:00:30.357.357671]:INFO:[runInference:0609] Completed_Model : 0, Name : cl-tfl-mobilenet_v1_1.0_224, Total time : 5.210250, Offload Time : 0 , DDR RW MBs : 0, Output Image File : cpp_out_cl-tfl-mobilenet_v1_1.0_224.jpg, Output Bin File : cpp_out_cl-tfl-mobilenet_v1_1.0_224.bin APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 824.591116 s: IPC: Deinit ... !!! 824.591684 s: IPC: DeInit ... Done !!! 824.591710 s: MEM: Deinit ... !!! 824.591720 s: DDR_SHARED_MEM: Alloc's: 9 alloc's of 6355664 bytes 824.591724 s: DDR_SHARED_MEM: Free's : 9 free's of 6355664 bytes 824.591729 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 824.591743 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! ss-tfl-deeplabv3_mnv2_ade20k_float ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/artifacts/ image path set to: test_data/ADE_val_00001801.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [06:00:30.000.000000]:INFO:[runInference:0231] Loaded model model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float//model/deeplabv3_mnv2_ade20k_float.tflite [06:00:30.000.000073]:INFO:[runInference:0233] resolved reporter [06:00:30.037.037469]:INFO:[runInference:0245] tensors size: 208 [06:00:30.037.037542]:INFO:[runInference:0246] nodes size: 81 [06:00:30.037.037560]:INFO:[runInference:0247] number of inputs: 1 [06:00:30.037.037577]:INFO:[runInference:0248] number of outputs: 1 [06:00:30.037.037594]:INFO:[runInference:0249] input(0) name: MobilenetV2/MobilenetV2/input [06:00:30.037.037613]:INFO:[runInference:0278] input: 0 [06:00:30.037.037629]:INFO:[runInference:0284] TIDL delegate mode Number of subgraphs:1 , 81 nodes delegated out of 81 nodes APP: Init ... !!! 824.707273 s: MEM: Init ... !!! 824.707354 s: MEM: Initialized DMA HEAP (fd=6) !!! 824.707545 s: MEM: Init ... Done !!! 824.707564 s: IPC: Init ... !!! 824.711447 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 824.716189 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 824.716338 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 824.716343 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 824.716347 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 824.717361 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 824.717524 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 824.717640 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 824.717795 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 824.717801 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 824.717819 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO [06:00:31.141.141835]:INFO:[runInference:0294] ModifyGraphWithDelegate - Done [06:00:31.141.141903]:INFO:[runInference:0341] device mem enabled [06:00:31.288.288200]:INFO:[preprocImage:0115] template NHWC [06:00:31.302.302397]:INFO:[runInference:0414] interpreter->Invoke - Started [06:00:32.1279.279586]:INFO:[runInference:0434] interpreter->Invoke - Done [06:00:32.1279.279669]:INFO:[runInference:0436] average time:44.391949 ms [06:00:32.1279.279761]:INFO:[prepSegResult:0087] preparing segmentation result [06:00:32.1287.287857]:INFO:[runInference:0559] saving image result file [06:00:32.1300.300332]:INFO:[runInference:0609] Completed_Model : 0, Name : ss-tfl-deeplabv3_mnv2_ade20k_float, Total time : 44.391949, Offload Time : 0 , DDR RW MBs : 0, Output Image File : cpp_out_ss-tfl-deeplabv3_mnv2_ade20k_float.jpg, Output Bin File : cpp_out_ss-tfl-deeplabv3_mnv2_ade20k_float.bin APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 825.965409 s: IPC: Deinit ... !!! 825.965991 s: IPC: DeInit ... Done !!! 825.966019 s: MEM: Deinit ... !!! 825.966030 s: DDR_SHARED_MEM: Alloc's: 9 alloc's of 6850044 bytes 825.966035 s: DDR_SHARED_MEM: Free's : 9 free's of 6850044 bytes 825.966039 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 825.966052 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! /data/user/edgeai-tidl-tools/scripts/osrt_cpp_scripts Command : bash run_onnx_models.sh in Dir : scripts/osrt_cpp_scripts/ Started cl-ort-resnet18-v1 ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/cl-ort-resnet18-v1/artifacts/ image path set to: test_data/airshow.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [06:00:32.000.000000]:INFO:[runInference:0341] accelerated mode [06:00:32.001.001300]:INFO:[runInference:0344] artifacts: model-artifacts/cl-ort-resnet18-v1//artifacts libtidl_onnxrt_EP loaded 0xd070750 Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 APP: Init ... !!! 827.744108 s: MEM: Init ... !!! 827.744169 s: MEM: Initialized DMA HEAP (fd=5) !!! 827.744361 s: MEM: Init ... Done !!! 827.744381 s: IPC: Init ... !!! 827.748192 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 827.753613 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 827.753774 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 827.753781 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 827.753785 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 827.754666 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 827.754828 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 827.754970 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 827.755087 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 827.755093 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 827.755109 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO [06:00:34.1666.666737]:INFO:[runInference:0367] Loaded model model-artifacts/cl-ort-resnet18-v1//model/resnet18_opset9.onnx [06:00:34.1666.666850]:INFO:[printTensorInfo:0241] number of inputs:1 [06:00:34.1666.666869]:INFO:[printTensorInfo:0242] number of outputs: 1 [06:00:34.1666.666885]:INFO:[printTensorInfo:0243] input(0) name: input.1Net_IN [06:00:34.1666.666901]:INFO:[printTensorInfo:0248] Input 0 : name=input.1Net_IN [06:00:34.1666.666920]:INFO:[printTensorInfo:0255] Input 0 : type=2 [06:00:34.1666.666938]:INFO:[printTensorInfo:0258] Input 0 : num_dims=4 [06:00:34.1666.666954]:INFO:[printTensorInfo:0261] Input 0 : dim 0=1 [06:00:34.1666.666969]:INFO:[printTensorInfo:0261] Input 0 : dim 1=3 [06:00:34.1666.666984]:INFO:[printTensorInfo:0261] Input 0 : dim 2=224 [06:00:34.1667.667000]:INFO:[printTensorInfo:0261] Input 0 : dim 3=224 [06:00:34.1678.678836]:INFO:[preprocImage:0127] template NCHW [06:00:34.1823.823846]:INFO:[runInference:0499] invoked [06:00:34.1823.823939]:INFO:[runInference:0500] average time: 6.799100 ms [06:00:34.1824.824035]:INFO:[prepClassificationResult:0094] preparing classification result [06:00:34.1835.835374]:INFO:[prepClassificationResult:0145] 23.043467: 895 warplane, military plane [06:00:34.1835.835433]:INFO:[prepClassificationResult:0145] 21.703730: 403 aircraft carrier, carrier, flattop, attack aircraft carrier [06:00:34.1835.835455]:INFO:[prepClassificationResult:0145] 18.488363: 744 projectile, missile [06:00:34.1835.835474]:INFO:[prepClassificationResult:0145] 18.488363: 657 missile [06:00:34.1835.835492]:INFO:[prepClassificationResult:0145] 15.808890: 404 airliner [06:00:34.1849.849663]:INFO:[runInference:0670] Completed_Model : 0, Name : cl-ort-resnet18-v1, Total time : 6.799100, Offload Time : 0 , DDR RW MBs : 0, Output Image File : cpp_out_cl-ort-resnet18-v1.jpg, Output Bin File : cpp_out_cl-ort-resnet18-v1.bin APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 828.035284 s: IPC: Deinit ... !!! 828.036602 s: IPC: DeInit ... Done !!! 828.036634 s: MEM: Deinit ... !!! 828.036788 s: DDR_SHARED_MEM: Alloc's: 10 alloc's of 13823180 bytes 828.036795 s: DDR_SHARED_MEM: Free's : 10 free's of 13823180 bytes 828.036799 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 828.036810 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! od-ort-ssd-lite_mobilenetv2_fpn ***** Display run Config: start ***** verbose level set to: 1 accelerated mode set to: 1 device mem set to: 1 loop count set to: 20 model path set to: model artifacts path set to: model-artifacts/od-ort-ssd-lite_mobilenetv2_fpn/artifacts/ image path set to: test_data/ADE_val_00001801.jpg device_type set to: cpu labels path set to: test_data/labels.txt num of threads set to: 4 num of results set to: 5 num of warmup runs set to: 2 ***** Display run Config: end ***** [06:00:34.000.000000]:INFO:[runInference:0341] accelerated mode [06:00:34.000.000082]:INFO:[runInference:0344] artifacts: model-artifacts/od-ort-ssd-lite_mobilenetv2_fpn//artifacts libtidl_onnxrt_EP loaded 0xd05c590 Final number of subgraphs created are : 1, - Offloaded Nodes - 478, Total Nodes - 478 APP: Init ... !!! 830.141177 s: MEM: Init ... !!! 830.141232 s: MEM: Initialized DMA HEAP (fd=5) !!! 830.141426 s: MEM: Init ... Done !!! 830.141445 s: IPC: Init ... !!! 830.145391 s: IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 830.150078 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 830.150226 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR 830.150231 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING 830.150236 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO 830.151363 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 830.151535 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 830.151679 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 830.151808 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 830.151814 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!! 830.151830 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO [06:00:36.1934.934973]:INFO:[runInference:0367] Loaded model model-artifacts/od-ort-ssd-lite_mobilenetv2_fpn//model/ssd-lite_mobilenetv2_fpn.onnx [06:00:36.1935.935088]:INFO:[printTensorInfo:0241] number of inputs:1 [06:00:36.1935.935106]:INFO:[printTensorInfo:0242] number of outputs: 2 [06:00:36.1935.935122]:INFO:[printTensorInfo:0243] input(0) name: inputNet_IN [06:00:36.1935.935138]:INFO:[printTensorInfo:0248] Input 0 : name=inputNet_IN [06:00:36.1935.935158]:INFO:[printTensorInfo:0255] Input 0 : type=2 [06:00:36.1935.935176]:INFO:[printTensorInfo:0258] Input 0 : num_dims=4 [06:00:36.1935.935192]:INFO:[printTensorInfo:0261] Input 0 : dim 0=1 [06:00:36.1935.935207]:INFO:[printTensorInfo:0261] Input 0 : dim 1=3 [06:00:36.1935.935222]:INFO:[printTensorInfo:0261] Input 0 : dim 2=512 [06:00:36.1935.935237]:INFO:[printTensorInfo:0261] Input 0 : dim 3=512 [06:00:36.2048.048457]:INFO:[preprocImage:0127] template NCHW [06:00:37.2754.754218]:INFO:[runInference:0499] invoked [06:00:37.2754.754304]:INFO:[runInference:0500] average time: 33.445351 ms [06:00:37.2755.755565]:INFO:[prepDetectionResult:0448] preparing detection result [06:00:37.2756.756293]:INFO:[prepDetectionResult:0513] box with score:0.694876, threshold set to:0.500000 [06:00:37.2770.770600]:INFO:[runInference:0670] Completed_Model : 0, Name : od-ort-ssd-lite_mobilenetv2_fpn, Total time : 33.445351, Offload Time : 0 , DDR RW MBs : 0, Output Image File : cpp_out_od-ort-ssd-lite_mobilenetv2_fpn.jpg, Output Bin File : cpp_out_od-ort-ssd-lite_mobilenetv2_fpn.bin APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! 831.099977 s: IPC: Deinit ... !!! 831.100594 s: IPC: DeInit ... Done !!! 831.100626 s: MEM: Deinit ... !!! 831.100783 s: DDR_SHARED_MEM: Alloc's: 12 alloc's of 7016840 bytes 831.100791 s: DDR_SHARED_MEM: Free's : 12 free's of 7016840 bytes 831.100795 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes 831.100806 s: MEM: Deinit ... Done !!! APP: Deinit ... Done !!! /data/user/edgeai-tidl-tools/scripts/osrt_cpp_scripts [{'Sl No.': 0, 'Runtime': 'tfl-py', 'Name': 'cl-tfl-mobilenet_v1_1.0_224', 'Output Image File': 'py_out_cl-tfl-mobilenet_v1_1.0_224_airshow.jpg', 'Output Bin File': 'py_out_cl-tfl-mobilenet_v1_1.0_224_airshow.bin', 'Total time': '4.50', 'Offload Time': '4.47', 'Ref Total Time': '3.9', 'Ref Offload Time': '3.88', 'Total Time Diff': ' 0.60', 'Total Time Diff(%)': '15.38%', 'DDR RW MBs': '7', 'Functional Status': 'PASS', 'Performance Status': 'FAIL', 'Info': 'Actual time more than Ref time by 0.02ms or 2%'}, {'Sl No.': 1, 'Runtime': 'tfl-py', 'Name': 'ss-tfl-deeplabv3_mnv2_ade20k_float', 'Output Image File': 'py_out_ss-tfl-deeplabv3_mnv2_ade20k_float_ADE_val_00001801.jpg', 'Output Bin File': 'py_out_ss-tfl-deeplabv3_mnv2_ade20k_float_ADE_val_00001801.bin', 'Total time': '40.21', 'Offload Time': '40.17', 'Ref Total Time': '34.98', 'Ref Offload Time': '34.7', 'Total Time Diff': ' 5.23', 'Total Time Diff(%)': '14.95%', 'DDR RW MBs': '150', 'Functional Status': 'PASS', 'Performance Status': 'FAIL', 'Info': 'Actual time more than Ref time by 0.02ms or 2%'}, {'Sl No.': 2, 'Runtime': 'tfl-py', 'Name': 'od-tfl-ssd_mobilenet_v2_300_float', 'Output Image File': 'py_out_od-tfl-ssd_mobilenet_v2_300_float_ADE_val_00001801.jpg', 'Output Bin File': 'py_out_od-tfl-ssd_mobilenet_v2_300_float_ADE_val_00001801.bin', 'Total time': '22.98', 'Offload Time': '22.94', 'Ref Total Time': '21.3', 'Ref Offload Time': '21.26', 'Total Time Diff': ' 1.68', 'Total Time Diff(%)': ' 7.89%', 'DDR RW MBs': '77', 'Functional Status': 'PASS', 'Performance Status': 'FAIL', 'Info': 'Actual time more than Ref time by 0.02ms or 2%'}, {'Sl No.': 3, 'Runtime': 'ort-py', 'Name': 'cl-ort-resnet18-v1', 'Output Image File': 'py_out_cl-ort-resnet18-v1_airshow.jpg', 'Output Bin File': 'py_out_cl-ort-resnet18-v1_airshow.bin', 'Total time': '6.73', 'Offload Time': '6.54', 'Ref Total Time': '9.6', 'Ref Offload Time': '0', 'Total Time Diff': '-2.87', 'Total Time Diff(%)': '-29.90%', 'DDR RW MBs': '14', 'Functional Status': 'PASS', 'Performance Status': 'PASS', 'Info': ''}, {'Sl No.': 4, 'Runtime': 'ort-py', 'Name': 'od-ort-ssd-lite_mobilenetv2_fpn', 'Output Image File': 'py_out_od-ort-ssd-lite_mobilenetv2_fpn_ADE_val_00001801.jpg', 'Output Bin File': 'py_out_od-ort-ssd-lite_mobilenetv2_fpn_ADE_val_00001801.bin', 'Total time': '31.92', 'Offload Time': '31.53', 'Ref Total Time': '30', 'Ref Offload Time': '0', 'Total Time Diff': ' 1.92', 'Total Time Diff(%)': ' 6.40%', 'DDR RW MBs': '76', 'Functional Status': 'PASS', 'Performance Status': 'FAIL', 'Info': 'Actual time more than Ref time by 0.02ms or 2%'}, {'Sl No.': 5, 'Runtime': 'tfl-cpp', 'Name': 'ss-tfl-deeplabv3_mnv2_ade20k_float', 'Output Image File': 'cpp_out_ss-tfl-deeplabv3_mnv2_ade20k_float.jpg', 'Output Bin File': 'cpp_out_ss-tfl-deeplabv3_mnv2_ade20k_float.bin', 'Total time': '44.391949', 'Offload Time': '0', 'Ref Total Time': '38.67', 'Ref Offload Time': '0', 'Total Time Diff': ' 5.72', 'Total Time Diff(%)': '14.80%', 'DDR RW MBs': '0', 'Functional Status': 'PASS', 'Performance Status': 'FAIL', 'Info': 'Actual time more than Ref time by 0.02ms or 2%'}, {'Sl No.': 6, 'Runtime': 'tfl-cpp', 'Name': 'cl-tfl-mobilenet_v1_1.0_224', 'Output Image File': 'cpp_out_cl-tfl-mobilenet_v1_1.0_224.jpg', 'Output Bin File': 'cpp_out_cl-tfl-mobilenet_v1_1.0_224.bin', 'Total time': '5.210250', 'Offload Time': '0', 'Ref Total Time': '4.4', 'Ref Offload Time': '0', 'Total Time Diff': ' 0.81', 'Total Time Diff(%)': '18.41%', 'DDR RW MBs': '0', 'Functional Status': 'PASS', 'Performance Status': 'FAIL', 'Info': 'Actual time more than Ref time by 0.02ms or 2%'}, {'Sl No.': 7, 'Runtime': 'ort-cpp', 'Name': 'cl-ort-resnet18-v1', 'Output Image File': 'cpp_out_cl-ort-resnet18-v1.jpg', 'Output Bin File': 'cpp_out_cl-ort-resnet18-v1.bin', 'Total time': '6.799100', 'Offload Time': '0', 'Ref Total Time': '9.2', 'Ref Offload Time': '0', 'Total Time Diff': '-2.40', 'Total Time Diff(%)': '-26.10%', 'DDR RW MBs': '0', 'Functional Status': 'PASS', 'Performance Status': 'PASS', 'Info': ''}, {'Sl No.': 8, 'Runtime': 'ort-cpp', 'Name': 'od-ort-ssd-lite_mobilenetv2_fpn', 'Output Image File': 'cpp_out_od-ort-ssd-lite_mobilenetv2_fpn.jpg', 'Output Bin File': 'cpp_out_od-ort-ssd-lite_mobilenetv2_fpn.bin', 'Total time': '33.445351', 'Offload Time': '0', 'Ref Total Time': '32', 'Ref Offload Time': '0', 'Total Time Diff': ' 1.45', 'Total Time Diff(%)': ' 4.52%', 'DDR RW MBs': '0', 'Functional Status': 'PASS', 'Performance Status': 'FAIL', 'Info': 'Actual time more than Ref time by 0.02ms or 2%'}] Func Pass: 9 Func Fail: 0 Perf Pass: 2 Perf Fail: 7 Please refer to the output_images and output_binaries directory for generated outputs TEST DONE!