X64 Architecture Running 4 Models - ['cl-tfl-mobilenet_v1_1.0_224', 'ss-tfl-deeplabv3_mnv2_ade20k_float', 'od-tfl-ssd_mobilenet_v2_300_float', 'od-tfl-ssdlite_mobiledet_dsp_320x320_coco'] Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/cl-tfl-mobilenet_v1_1.0_224/tempDir/86_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 903.57 .... ..... ... .... ..... # 1 . .. T 960.47 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/cl-tfl-mobilenet_v1_1.0_224/tempDir/86_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 642.42 .... ..... ... .... ..... # 1 . .. T 597.44 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/cl-tfl-mobilenet_v1_1.0_224/tempDir/86_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 555.69 .... ..... ... .... ..... # 1 . .. T 842.92 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssd_mobilenet_v2_300_float/tempDir/264_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/cl-tfl-mobilenet_v1_1.0_224/tempDir/86_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 590.45 .... ..... ... .... ..... # 1 . .. T 627.72 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/cl-tfl-mobilenet_v1_1.0_224/tempDir/86_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 695.93 .... ..... ... .... ..... # 1 . .. Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssdlite_mobiledet_dsp_320x320_coco/tempDir/321_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 619.84 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/cl-tfl-mobilenet_v1_1.0_224/tempDir/86_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 3495.13 .... ..... ... .... ..... # 1 . .. T 664.43 .... ..... ... .... ..... # 1 . .. T 604.25 .... ..... ... .... ..... ------------------ Network Compiler Traces ----------------------------- successful Memory allocation successful Workload Creation Running_Model : cl-tfl-mobilenet_v1_1.0_224 Completed_Model : 1, Name : cl-tfl-mobilenet_v1_1.0_224 , Total time : 6258.77, Offload Time : 0.00 , DDR RW MBs : 18446744073709.55, Output File : py_out_cl-tfl-mobilenet_v1_1.0_224_ADE_val_00001801.jpg T 2981.58 .... ..... ... .... ..... T 4368.99 .... ..... ... .... ..... # 1 . .. Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssd_mobilenet_v2_300_float/tempDir/264_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 1369.71 .... ..... ... .... ..... # 1 . .. T 1427.43 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssd_mobilenet_v2_300_float/tempDir/264_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 4648.49 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssdlite_mobiledet_dsp_320x320_coco/tempDir/321_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 1437.67 .... ..... ... .... ..... # 1 . .. T 1359.41 .... ..... ... .... ..... T 1948.35 .... ..... ... .... ..... # 1 . .. Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssd_mobilenet_v2_300_float/tempDir/264_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 1392.91 .... ..... ... .... ..... # 1 . .. T 2017.41 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssdlite_mobiledet_dsp_320x320_coco/tempDir/321_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 1559.30 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssd_mobilenet_v2_300_float/tempDir/264_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 2165.24 .... ..... ... .... ..... # 1 . .. T 1494.44 .... ..... ... .... ..... # 1 . .. T 2172.60 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssdlite_mobiledet_dsp_320x320_coco/tempDir/321_tidl_io_.qunat_stats_config.txt T 1475.41 .... ..... ... .... ..... Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssd_mobilenet_v2_300_float/tempDir/264_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 1886.10 .... ..... ... .... ..... # 1 . .. T 1433.16 .... ..... ... .... ..... # 1 . .. T 1602.57 .... ..... ... .... ..... T 2236.32 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/tempDir/201_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssdlite_mobiledet_dsp_320x320_coco/tempDir/321_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. ------------------ Network Compiler Traces ----------------------------- successful Memory allocation successful Workload Creation Running_Model : od-tfl-ssd_mobilenet_v2_300_float Completed_Model : 3, Name : od-tfl-ssd_mobilenet_v2_300_float , Total time : 16900.14, Offload Time : 0.00 , DDR RW MBs : 18446744073709.55, Output File : py_out_od-tfl-ssd_mobilenet_v2_300_float_ADE_val_00001801.jpg T 1972.80 .... ..... ... .... ..... # 1 . .. T 1823.95 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/od-tfl-ssdlite_mobiledet_dsp_320x320_coco/tempDir/321_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 1819.05 .... ..... ... .... ..... # 1 . .. T 2048.29 .... ..... ... .... ..... ------------------ Network Compiler Traces ----------------------------- successful Memory allocation successful Workload Creation TIDL Meta PipeLine (Proto) File : ../../../models/public/ssdlite_mobiledet_dsp_320x320_coco_20200519.prototxt Number of OD backbone nodes = 112 Size of odBackboneNodeIds = 112 Preliminary number of subgraphs:1 , 129 nodes delegated out of 129 nodes TF Meta PipeLine (Proto) File : ../../../models/public/ssdlite_mobiledet_dsp_320x320_coco_20200519.prototxt num_classes : 91 y_scale : 10.000000 x_scale : 10.000000 w_scale : 5.000000 h_scale : 5.000000 num_keypoints : 5.000000 score_threshold : 0.600000 iou_threshold : 0.450000 max_detections_per_class : 200 max_total_detections : 100 scales, height_stride, width_stride, height_offset, width_offset 0.2000000, -1.0000000, -1.0000000, -1.0000000, -1.0000000 0.3500000, -1.0000000, -1.0000000, -1.0000000, -1.0000000 0.5000000, -1.0000000, -1.0000000, -1.0000000, -1.0000000 0.6500000, -1.0000000, -1.0000000, -1.0000000, -1.0000000 0.8000000, -1.0000000, -1.0000000, -1.0000000, -1.0000000 0.9500000, -1.0000000, -1.0000000, -1.0000000, -1.0000000 aspect_ratios 1.0000000 2.0000000 0.5000000 3.0000000 0.3333000 Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal ************** Frame index 1 : Running float import ************* **************************************************** ** ALL MODEL CHECK PASSED ** **************************************************** The soft limit is 2048 The hard limit is 2048 MEM: Init ... !!! MEM: Init ... Done !!! 0.0s: VX_ZONE_INIT:Enabled 0.7s: VX_ZONE_ERROR:Enabled 0.8s: VX_ZONE_WARNING:Enabled 0.2468s: VX_ZONE_INIT:[tivxInit:185] Initialization Done !!! ************ Frame index 1 : Running float inference **************** ************ Frame index 2 : Running fixed point mode for calibration **************** Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 0 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 0 completed ************************ ***************** Calibration iteration number 1 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 1 completed ************************ ***************** Calibration iteration number 2 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 2 completed ************************ ***************** Calibration iteration number 3 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 3 completed ************************ ***************** Calibration iteration number 4 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 4 completed ************************ Empty prototxt path, running calibration **************************************************** ** ALL MODEL CHECK Running_Model : od-tfl-ssdlite_mobiledet_dsp_320x320_coco Completed_Model : 4, Name : od-tfl-ssdlite_mobiledet_dsp_320x320_coco , Total time : 21115.02, Offload Time : 0.00 , DDR RW MBs : 18446744073709.55, Output File : py_out_od-tfl-ssdlite_mobiledet_dsp_320x320_coco_ADE_val_00001801.jpg T 12435.50 .... ..... ... .... ..... # 1 . .. T 10438.78 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/tempDir/201_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 4707.15 .... ..... ... .... ..... # 1 . .. T 5080.44 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/tempDir/201_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 4410.99 .... ..... ... .... ..... # 1 . .. T 4703.71 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/tempDir/201_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 4822.36 .... ..... ... .... ..... # 1 . .. T 4051.73 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/tempDir/201_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 4891.99 .... ..... ... .... ..... # 1 . .. T 4881.33 .... ..... ... .... ..... Processing config file #0 : /home/abhilash/edgeai-tidl-tools/model-artifacts/ss-tfl-deeplabv3_mnv2_ade20k_float/tempDir/201_tidl_io_.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 4353.13 .... ..... ... .... ..... # 1 . .. T 5619.70 .... ..... ... .... ..... ------------------ Network Compiler Traces ----------------------------- successful Memory allocation successful Workload Creation Preliminary number of subgraphs:1 , 81 nodes delegated out of 81 nodes Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal ************** Frame index 1 : Running float import ************* INFORMATION: [TIDL_ResizeLayer] ResizeBilinear_TIDL_0 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] ResizeBilinear_TIDL_1 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] ResizeBilinear Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] decoder/ResizeBilinear Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] ResizeBilinear_1 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize. **************************************************** ** 5 WARNINGS 0 ERRORS ** **************************************************** The soft limit is 2048 The hard limit is 2048 MEM: Init ... !!! MEM: Init ... Done !!! 0.0s: VX_ZONE_INIT:Enabled 0.8s: VX_ZONE_ERROR:Enabled 0.9s: VX_ZONE_WARNING:Enabled 0.3760s: VX_ZONE_INIT:[tivxInit:185] Initialization Done !!! ************ Frame index 1 : Running float inference **************** ************ Frame index 2 : Running fixed point mode for calibration **************** Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 0 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 0 completed ************************ ***************** Calibration iteration number 1 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 1 completed ************************ ***************** Calibration iteration number 2 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 2 completed ************************ ***************** Calibration iteration number 3 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 3 completed ************************ ***************** Calibration iteration number 4 started ************************ Empty prototxt path, running calibration ~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~ ***************** Calibration iteration number 4 completed ************************ Empty prototxt path, running calibration INFORMATION: [TIDL_ResizeLayer] ResizeBilinear_TIDL_0 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For Running_Model : ss-tfl-deeplabv3_mnv2_ade20k_float Completed_Model : 2, Name : ss-tfl-deeplabv3_mnv2_ade20k_float , Total time : 52779.48, Offload Time : 0.00 , DDR RW MBs : 18446744073709.55, Output File : py_out_ss-tfl-deeplabv3_mnv2_ade20k_float_ADE_val_00001801.jpg run python3 tflrt_delegate.py Running 4 Models - ['cl-tfl-mobilenet_v1_1.0_224', 'ss-tfl-deeplabv3_mnv2_ade20k_float', 'od-tfl-ssd_mobilenet_v2_300_float', 'od-tfl-ssdlite_mobiledet_dsp_320x320_coco'] Running_Model : cl-tfl-mobilenet_v1_1.0_224 , 0 0.691726 warplane, military plane ,, 1 0.181373 missile ,, 2 0.109571 projectile, missile ,, 3 0.006352 cannon ,, 4 0.005370 aircraft carrier, carrier, flattop, attack aircraft carrier , Saving image to ../../../output_images/ Completed_Model : 1, Name : cl-tfl-mobilenet_v1_1.0_224 , Total time : 179.95, Offload Time : 179.94 , DDR RW MBs : 18446744073709.55, Output File : py_out_cl-tfl-mobilenet_v1_1.0_224_airshow.jpg Running_Model : od-tfl-ssd_mobilenet_v2_300_float Saving image to ../../../output_images/ Completed_Model : 3, Name : od-tfl-ssd_mobilenet_v2_300_float , Total time : 465.09, Offload Time : 465.08 , DDR RW MBs : 18446744073709.55, Output File : py_out_od-tfl-ssd_mobilenet_v2_300_float_ADE_val_00001801.jpg Running_Model : od-tfl-ssdlite_mobiledet_dsp_320x320_coco Saving image to ../../../output_images/ Completed_Model : 4, Name : od-tfl-ssdlite_mobiledet_dsp_320x320_coco , Total time : 823.78, Offload Time : 823.77 , DDR RW MBs : 18446744073709.55, Output File : py_out_od-tfl-ssdlite_mobiledet_dsp_320x320_coco_ADE_val_00001801.jpg Running_Model : ss-tfl-deeplabv3_mnv2_ade20k_float Saving image to ../../../output_images/ Completed_Model : 2, Name : ss-tfl-deeplabv3_mnv2_ade20k_float , Total time : 1930.89, Offload Time : 1930.88 , DDR RW MBs : 18446744073709.55, Output File : py_out_ss-tfl-deeplabv3_mnv2_ade20k_float_ADE_val_00001801.jpg Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider'] Running 3 Models - ['cl-ort-resnet18-v1', 'od-ort-ssd-lite_mobilenetv2_fpn', 'ss-ort-deeplabv3lite_mobilenetv2'] ssd is meta arch name Number of OD backbone nodes = 159 Size of odBackboneNodeIds = 159 Preliminary subgraphs created = 1 Final number of subgraphs created are : 1, - Offloaded Nodes - 482, Total Nodes - 482 Graph Domain TO version : 11ERROR : ONNX RT data type : 19 not supported by TIDL **************************************************** ** All the Input Tensor Dimensions has to be greater then Zero ** DIM Error - For Tensor 0, Dim 3 is 0 ****************************************************