• State
  • Locked Locked
  • Replies 10 replies
  • Subscribers 98 subscribers
  • Views 1114 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

TDA4VM: yolov5s compiled by TIDL segmentation fault

TDA4VM: yolov5 compiled by TIDL inferring on TDA4 output nothing

jeff
Prodigy 31 points
Part Number: TDA4VM

The compiled yolov5 is deployed on the TDA4, however, it can not output the correct result. 

In comparison, this model can output correct results in TIDL simulation inferring on my server.

The two images below show the outcome between TIDL simulation and TDA4 inferring, using the same image.

  • 0
    Prodigy 31 points

    To help better figure out the problem, I also upload the TIDL compile environment and the param.yaml, below.

    A. the TIDL compile environment

    yolov5_onnx_weed_detector.rar

    1. yolo_onnx_inference_tidl.py

    the program is used to compile the onnx yolov5 model, especially, 

    calibration_frames = 50

    calibration_iterations = 50
    2. Run.log :  log is produced in the compile
     
    3. Artifacts :  compile result.
     
    B. param.yaml
    the model import configuration in TDA4 board
    Fullscreen param.yaml.txt Download 
    postprocess:
  data_layout: NCHW
  detection_thr: 0.3
preprocess:
  crop:
  - 640
  - 640
  data_layout: NCHW
  mean:
  - 0
  - 0
  - 0
  resize:
  - 640
  - 640
  scale:
  - 0.003921568627
  - 0.003921568627
  - 0.003921568627
  reverse_channels: false
session:
  input_mean: null
  input_scale: null
  artifacts_folder: 'artifacts'
  model_folder: model
  model_path: yolov5s6_ti_lite_640_weedDetect.onnx
  session_name: onnxrt
target_device: pc
task_type: detection


# calibration_dataset:
#   name: coco
#   num_classes: 2
#   num_frames: 50
#   path: ./dependencies/datasets/coco
#   shuffle: true
#   split: val2017
input_dataset:
  name: coco
  num_classes: 2
  # num_frames: 5000
  # path: ./dependencies/datasets/coco
  # shuffle: false
  # split: val2017
metric:
  label_offset_pred:
    0: 1
    1: 2
    2: 3
#     3: 4
#     4: 5
#     5: 6
#     6: 7
#     7: 8
#     8: 9
#     9: 10
#     10: 11
#     11: 13
#     12: 14
#     13: 15
#     14: 16
#     15: 17
#     16: 18
#     17: 19
#     18: 20
#     19: 21
#     20: 22
#     21: 23
#     22: 24
#     23: 25
#     24: 27
#     25: 28
#     26: 31
#     27: 32
#     28: 33
#     29: 34
#     30: 35
#     31: 36
#     32: 37
#     33: 38
#     34: 39
#     35: 40
#     36: 41
#     37: 42
#     38: 43
#     39: 44
#     40: 46
#     41: 47
#     42: 48
#     43: 49
#     44: 50
#     45: 51
#     46: 52
#     47: 53
#     48: 54
#     49: 55
#     50: 56
#     51: 57
#     52: 58
#     53: 59
#     54: 60
#     55: 61
#     56: 62
#     57: 63
#     58: 64
#     59: 65
#     60: 67
#     61: 70
#     62: 72
#     63: 73
#     64: 74
#     65: 75
#     66: 76
#     67: 77
#     68: 78
#     69: 79
#     70: 80
#     71: 81
#     72: 82
#     73: 84
#     74: 85
#     75: 86
#     76: 87
#     77: 88
#     78: 89
#     79: 90
#     80: 91
#   run_dir: /data/ssd/files/a0393608/work/code/ti/edgeai-algo/edgeai-benchmark/work_dirs/modelartifacts/8bits/od-8100_onnxrt_coco_edgeai-yolov5_yolov5s6_640_ti_lite_37p4_56p0_onnx
# model_info:
#   metric_reference:
#     accuracy_ap[.5:.95]%: 37.4
# postprocess:
#   detection_thr: 0.01
#   formatter:
#     dst_indices:
#     - 4
#     - 5
#     name: DetectionBoxSL2BoxLS
#     src_indices:
#     - 5
#     - 4
#   ignore_index: null
#   normalized_detections: false
#   resize_with_pad: true
#   save_output: false
#   shuffle_indices: null
#   squeeze_axis: null
# preprocess:
#   add_flip_image: false
#   backend: cv2
#   crop: 640
#   data_layout: NCHW
#   interpolation: null
#   pad_color:
#   - 114
#   - 114
#   - 114
#   resize: 640
#   resize_with_pad: true
#   reverse_channels: false
# session:
#   artifacts_folder: artifacts
#   dir_tree_depth: 3
#   extra_inputs: null
#   input_data_layout: NCHW
#   input_mean: null
#   input_optimization: true
#   input_scale: null
#   input_shape:
#     imagesNet_IN:
#     - 1
#     - 3
#     - 640
#     - 640
#   model_file: /data/ssd/files/a0393608/work/code/ti/edgeai-algo/edgeai-benchmark/work_dirs/modelartifacts/8bits/od-8100_onnxrt_coco_edgeai-yolov5_yolov5s6_640_ti_lite_37p4_56p0_onnx/model/yolov5s6_640_ti_lite_37p4_56p0.onnx
#   model_folder: model
#   model_id: od-8100
#   model_path: model/yolov5s6_640_ti_lite_37p4_56p0.onnx
#   model_type: null
#   num_inputs: 1
#   num_tidl_subgraphs: 16
#   output_shape: null
#   run_dir: od-8100_onnxrt_coco_edgeai-yolov5_yolov5s6_640_ti_lite_37p4_56p0_onnx
#   runtime_options:
#     accuracy_level: 1
#     advanced_options:activation_clipping: 1
#     advanced_options:add_data_convert_ops: 3
#     advanced_options:bias_calibration: 1
#     advanced_options:calibration_frames: 50
#     advanced_options:calibration_iterations: 50
#     advanced_options:channel_wise_quantization: 0
#     advanced_options:high_resolution_optimization: 0
#     advanced_options:output_feature_16bit_names_list: 370, 680, 990, 1300
#     advanced_options:params_16bit_names_list: ''
#     advanced_options:pre_batchnorm_fold: 1
#     advanced_options:quantization_scale_type: 0
#     advanced_options:weight_clipping: 1
#     artifacts_folder: /data/ssd/files/a0393608/work/code/ti/edgeai-algo/edgeai-benchmark/work_dirs/modelartifacts/8bits/od-8100_onnxrt_coco_edgeai-yolov5_yolov5s6_640_ti_lite_37p4_56p0_onnx/artifacts
#     debug_level: 0
#     import: 'no'
#     object_detection:meta_arch_type: 6
#     object_detection:meta_layers_names_list: /data/ssd/files/a0393608/work/code/ti/edgeai-algo/edgeai-benchmark/work_dirs/modelartifacts/8bits/od-8100_onnxrt_coco_edgeai-yolov5_yolov5s6_640_ti_lite_37p4_56p0_onnx/model/yolov5s6_640_ti_lite_metaarch.prototxt
#     platform: J7
#     priority: 0
#     tensor_bits: 8
#     tidl_tools_path: /data/ssd/files/a0393608/work/code/ti/edgeai-algo/edgeai-benchmark/tidl_tools
#     version: '8.2'
#   session_name: onnxrt
#   supported_devices: null
#   target_device: pc
#   tensor_bits: 8
#   tidl_offload: true
#   tidl_tools_path: /data/ssd/files/a0393608/work/code/ti/edgeai-algo/edgeai-benchmark/tidl_tools
#   work_dir: ./work_dirs/modelartifacts/8bits
# task_type: detection
  • 0 in reply to jeff
    TI__Intellectual 2406 points

    Hi Jeff,

    How are you running this model? Can you try using the same script for running both import and  inference? You just need to disable the import part while running in EVM.

    Regards, Debapriya

  • 0 in reply to Debapriya Maji
    Prodigy 31 points

    Hi Debapriya,

    I run this compiled model using python demo app_edgai.py, with the config file object_detection.yaml.  In this config, I add a model_path pointing to the path containing the param.yaml and artifacts folder, as shown.

    jeff said:
    3. Artifacts :  compile result.
     
    B. param.yaml
    the model import configuration in TDA4 board

    Regards, Jeff

  • 0 in reply to jeff
    TI__Intellectual 2406 points

    Hi Jeff,

    Can you try running the same script for import as well as inference. I am attaching a modified script for your reference.

    You can just run the script with the following flags. Let me know the result of it,

    python yolo_onnx_inference_tidl.py --compile

    python yolo_onnx_inference_tidl.py --inference

    Fullscreen 5857.yolo_onnx_inference_tidl.py Download 
    # dependencies
# Anaconda Python 3.7 for Linux - download and install from: https://www.anaconda.com/distribution/
# pytorch, torchvision - install using: conda install pytorch torchvision -c pytorch
# timm - install using: pip install timm

import os
import numpy as np
import cv2
import argparse
import onnxruntime as rt


parser = argparse.ArgumentParser()
parser.add_argument("--model_path", type=str, default="./yolov5s6_640_ti_lite_37p4_56p0.onnx")
parser.add_argument("--model_prototxt", type=str, default="./yolov5s6_640_ti_lite_metaarch.prototxt")
parser.add_argument("--img_calib_path", type=str, default="./sample_calib_ips_640x640.txt")
parser.add_argument("--img_path", type=str, default="./sample_ips_640x640.txt")
parser.add_argument("--dst_path", type=str, default="./sample_ops_onnxrt_det")
parser.add_argument("--compile", default=False, action="store_true")
parser.add_argument("--inference", default=False, action="store_true")

args = parser.parse_args()
output_dir =  "./artifacts/"
os.makedirs(output_dir, exist_ok=True)

if args.compile:
    for root, dirs, files in os.walk(output_dir, topdown=False):
        [os.remove(os.path.join(root, f)) for f in files]
        [os.rmdir(os.path.join(root, d)) for d in dirs]

calibration_frames = 2
calibration_iterations = 5

compile_options = {
"artifacts_folder": "./artifacts/",
"tensor_bits":8,
"accuracy_level":1,
#"debug_level": 3,
"advanced_options:calibration_frames": calibration_frames,
"advanced_options:calibration_iterations": calibration_iterations,
"advanced_options:output_feature_16bit_names_list" : "370, 680, 990, 1300",
'object_detection:meta_layers_names_list' : args.model_prototxt,
'object_detection:meta_arch_type' : 6,
"ti_internal_nc_flag" : 1601,
#"add_data_convert_ops" : 3,
}

if args.inference:
    compile_options["tidl_tools_path"] = ""
else:
    compile_options["tidl_tools_path"] = os.environ["TIDL_TOOLS_PATH"]

so = rt.SessionOptions()
# EP_list = ['TIDLCompilationProvider','CPUExecutionProvider']
# sess = rt.InferenceSession(args.model_path ,providers=EP_list, provider_options=[compile_options, {}], sess_options=so)
# input_details = sess.get_inputs()


_CLASS_COLOR_MAP = [
    (0, 0, 255) , # Person (blue).
    (255, 0, 0) ,  # Bear (red).
    (0, 255, 0) ,  # Tree (lime).
    (255, 0, 255) ,  # Bird (fuchsia).
    (0, 255, 255) ,  # Sky (aqua).
    (255, 255, 0) ,  # Cat (yellow).
]


def read_img(img_file, img_mean=127.5, img_scale=1/127.5):
    img = cv2.imread(img_file)[:, :, ::-1]
    img = cv2.resize(img, (640,640), interpolation=cv2.INTER_LINEAR)
    img = (img - img_mean) * img_scale
    img = np.asarray(img, dtype=np.float32)
    img = np.expand_dims(img,0)
    img = img.transpose(0,3,1,2)
    return img


def model_import_image_list_tidl(model_path, img_path=None, mean=None, scale=None):
    "model compilation"
    img_file_list = list(open(img_path))[:calibration_frames]
    EP_list = ['TIDLCompilationProvider','CPUExecutionProvider']
    sess = rt.InferenceSession(model_path ,providers=EP_list, provider_options=[compile_options, {}], sess_options=so)

    input_name = sess.get_inputs()[0].name
    for img_index, img_file  in enumerate(img_file_list):
        img_file = img_file.split(' ')[0].rstrip()
        input = read_img(img_file, mean, scale)
        output = sess.run([], {input_name: input})


def model_infer_image_list_tidl(model_path, img_path=None, mean=None, scale=None, dst_path=None):
    "inference on sample images"
    os.makedirs(args.dst_path, exist_ok=True)
    EP_list = ['TIDLExecutionProvider','CPUExecutionProvider']
    sess = rt.InferenceSession(model_path ,providers=EP_list, provider_options=[compile_options, {}], sess_options=so)
    input_name = sess.get_inputs()[0].name
    img_file_list = list(open(img_path))
    max_index = 20
    print("Starting Inference")
    for img_index, img_file  in enumerate(img_file_list):
        print(f"{img_index+1}/{len(img_file_list)}")
        img_file = img_file.split(' ')[0].rstrip()
        input = read_img(img_file, mean, scale)
        output = sess.run([], {input_name: input})
        output = np.squeeze(output[0])
        dst_file = os.path.join(dst_path, os.path.basename(img_file))
        post_process(img_file, dst_file, output, score_threshold=0.5)



def post_process(img_file, dst_file, output, score_threshold=0.3):
    """
    Draw bounding boxes on the input image. Dump boxes in a txt file.
    """
    det_bboxes, det_scores, det_labels = output[:, 0:4], output[:, 4], output[:, 5]
    img = cv2.imread(img_file)
    #To generate color based on det_label, to look into the codebase of Tensorflow object detection api.
    dst_txt_file = dst_file.replace('png', 'txt')
    f = open(dst_txt_file, 'wt')
    for idx in range(len(det_bboxes)):
        det_bbox = det_bboxes[idx]
        if det_scores[idx]>0:
            f.write("{:8.0f} {:8.5f} {:8.5f} {:8.5f} {:8.5f} {:8.5f}\n".format(det_labels[idx], det_scores[idx], det_bbox[1], det_bbox[0], det_bbox[3], det_bbox[2]))
        if det_scores[idx]>score_threshold:
            color_map = _CLASS_COLOR_MAP[int(det_labels[idx]) %len(_CLASS_COLOR_MAP)]
            img = cv2.rectangle(img, (int(det_bbox[0]), int(det_bbox[1])), (int(det_bbox[2]), int(det_bbox[3])), color_map[::-1], 2)
            cv2.putText(img, "id:{}".format(int(det_labels[idx])), (int(det_bbox[0]+5),int(det_bbox[1])+15), cv2.FONT_HERSHEY_SIMPLEX, 0.5, color_map[::-1], 2)
            cv2.putText(img, "score:{:2.1f}".format(det_scores[idx]), (int(det_bbox[0] + 5), int(det_bbox[1]) + 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, color_map[::-1], 2)
    cv2.imwrite(dst_file, img)
    f.close()



def main():
    if args.compile:
        model_import_image_list_tidl(model_path=args.model_path, img_path=args.img_calib_path,
                                   mean=0.0, scale=0.00392156862745098)
    elif args.inference:
        model_infer_image_list_tidl(model_path=args.model_path, img_path=args.img_path,
                               mean=0.0, scale=0.00392156862745098,
                               dst_path=args.dst_path)


if __name__== "__main__":
    main()

    Regards, Debapriya

  • 0 in reply to Debapriya Maji
    Prodigy 31 points

    Hi Debapriya,

    I try to run your script on my server as well as run infer on the EVM, the running shown as below:

    A. the compile log, run on my server

    Fullscreen 6052.compile.log Download 
    TIDL Meta PipeLine (Proto) File  : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/yolov5s6_ti_lite_640_weedDetect.prototxt  
yolo_v3
yolo_v3
Number of OD backbone nodes = 192 
Size of odBackboneNodeIds = 192 

Preliminary subgraphs created = 1 
Final number of subgraphs created are : 1, - Offloaded Nodes - 295, Total Nodes - 295 
TIDL Meta PipeLine (Proto) File  : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/yolov5s6_ti_lite_640_weedDetect.prototxt  
yolo_v3
yolo_v3

 ************** Frame index 1 : Running float import ************* 
INFORMATION: [TIDL_ResizeLayer] Resize_107 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_123 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_139 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          4 WARNINGS          0 ERRORS          **
****************************************************
 0.0s:  VX_ZONE_INIT:Enabled
 0.13s:  VX_ZONE_ERROR:Enabled
 0.14s:  VX_ZONE_WARNING:Enabled
 0.589s:  VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!

**********  Frame Index 1 : Running float inference **********

**********  Frame Index 2 : Running fixed point mode for calibration **********

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/artifacts/tempDir/detections_tidl_io_.qunat_stats_config.txt 
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

 #    0 . .. T    6967.04  .... ..... ... .... .....
#    1 . .. T    6983.33  .... ..... ... .... .....
~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/artifacts/tempDir/detections_tidl_io_.qunat_stats_config.txt 
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

 #    0 . .. T    4293.90  .... ..... ... .... .....
#    1 . .. T    4227.38  .... ..... ... .... .....
 
 
 *****************   Calibration iteration number 0 completed ************************ 
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/artifacts/tempDir/detections_tidl_io_.qunat_stats_config.txt 
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

 #    0 . .. T    4247.92  .... ..... ... .... .....
#    1 . .. T    4237.39  .... ..... ... .... .....
 
 
 *****************   Calibration iteration number 1 completed ************************ 
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/artifacts/tempDir/detections_tidl_io_.qunat_stats_config.txt 
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T    4220.41  .... ..... ... .... .....
#    1 . .. T    4198.97  .... ..... ... .... .....
 
 
 *****************   Calibration iteration number 2 completed ************************ 
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/artifacts/tempDir/detections_tidl_io_.qunat_stats_config.txt 
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

 #    0 . .. T    4217.31  .... ..... ... .... .....
#    1 . .. T    4199.89  .... ..... ... .... .....
 
 
 *****************   Calibration iteration number 3 completed ************************ 
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/jeff/XAG_working/TI/edgeai-tidl-tools/model-artifacts/yolov5_onnx_weed_detector/artifacts/tempDir/detections_tidl_io_.qunat_stats_config.txt 
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T    4226.08  .... ..... ... .... .....
#    1 . .. T    4218.58  .... ..... ... .... .....
 
 
 *****************   Calibration iteration number 4 completed ************************ 
 
 
 

------------------ Network Compiler Traces -----------------------------
successful Memory allocation
substitute string tidl_net_ not found
INFORMATION: [TIDL_ResizeLayer] Resize_107 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_123 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_139 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
****************************************************
**          3 WARNINGS          0 ERRORS          **
****************************************************

    B. the infer log, run on the EVM board

    Fullscreen EVMinfer.log Download 
    Starting Inference
1/1
libtidl_onnxrt_EP loaded 0x2ba38520 
Final number of subgraphs created are : 1, - Offloaded Nodes - 295, Total Nodes - 295 
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=4) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  1225.526937 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  1225.526983 s:  VX_ZONE_INIT:Enabled
  1225.526985 s:  VX_ZONE_ERROR:Enabled
  1225.526988 s:  VX_ZONE_WARNING:Enabled
  1225.527723 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
  1225.527911 s:  VX_ZONE_INIT:[tivxHostInitLocal:86] Initialization Done for HOST !!!
  1225.724478 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:100] De-Initialization Done for HOST !!!
  1225.728805 s:  VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
IPC: Deinit ... !!!
IPC: DeInit ... Done !!!
MEM: Deinit ... !!!
MEM: Alloc's: 7 alloc's of 15348981 bytes 
MEM: Free's : 7 free's  of 15348981 bytes 
MEM: Open's : 0 allocs  of 0 bytes 
MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!

    No matter on my server or on the EVM, the onnx model performance remain consist, the result shown as below:

  • 0 in reply to Debapriya Maji
    Prodigy 31 points

    Hi Debapriya,

    I did some modifications for your script following the setup in run_times.py, shown as below:

    def model_infer_image_list_tidl(model_path, img_path=None, mean=None, scale=None, dst_path=None):
    "inference on sample images"
    os.makedirs(args.dst_path, exist_ok=True)
    runtime_options = {
            "tidl_platform": "J7",
            "tidl_version": "7.2",
            "tidl_tools_path": "null",
            "artifacts_folder": srcDir+"/artifacts/",
            "tensor_bits": 8,
            "import": 'no',
    }
    EP_list = ['TIDLExecutionProvider','CPUExecutionProvider']
    # sess = rt.InferenceSession(model_path ,providers=EP_list, provider_options=[compile_options, {}], sess_options=so)
    sess = rt.InferenceSession(model_path ,providers=EP_list, provider_options=[runtime_options, {}], sess_options=so)
    input_name = sess.get_inputs()[0].name
    img_file_list = list(open(img_path))
    max_index = 20
    print("Starting Inference")
    for img_index, img_file  in enumerate(img_file_list):
        print(f"{img_index+1}/{len(img_file_list)}")
        img_file = img_file.split(' ')[0].rstrip()
        input = read_img(img_file, mean, scale)
        start = time()
        output = sess.run([], {input_name: input})
        end = time()
        print('dl-inference', (end - start)*1000)
        output = np.squeeze(output[0])
        dst_file = os.path.join(dst_path, os.path.basename(img_file))
        post_process(img_file, dst_file, output, score_threshold=0.5)

    and the result remain still as before.

    jeff said:
    the result shown as below:

    However, if I run this compiled model using python demo app_edgeai.py or C++ demo app_edgeai, none of them can produce the correct result.

    BTW, I have noticed the output tensor order should be [x1y1 x2y2 score id] instead of [x1y1 x2y2 id score] in the postprocess, and change it in the program, but still not working, which makes me so confused.

    So, according to the result I post, can I make a conclusion that the compiled model with the artifacts file actually runs properly in the EVM accelerated by the MMA correctly, but somewhere goes wrong in the preprocess or postprocess ??

  • 0 in reply to jeff
    TI__Intellectual 2406 points

    Hi Jeff,

    Given that the model inferred correct output with this script  "yolo_onnx_inference_tidl.py",  we can tell that the compilation and inference are happening successfully. 

    There is probably something wrong in the param.yaml file. Let me look into it and get back to you.

    Regards, Debapriya

  • 0 in reply to Debapriya Maji
    TI__Intellectual 2406 points

    Hi jeff,

    I can see that many of the fields in params.yaml is disabled. Even the input_mean and  input_scale under the sess are set to Null. I have updated the params.yaml file with the expected input_mean and input_scale. Can you please try with this params.yaml and let me know your observation.

    https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/detection_5F00_yolov5s6_5F00_param.yaml

    Regards, Debapriya

  • 0 in reply to Debapriya Maji
    Prodigy 31 points

    Hi Debapriya,

    Thank you for your jobs, and after changing the input_mean and  input_scale value in my params.yaml, it works, brilliant !! 

    However, may I ask how the input_mean and input_scale are calculated? and what roles they play in the preprocess? 

    Regards, Jeff

  • 0 in reply to jeff
    TI__Intellectual 2406 points

    HI Jeff,

    The mean and variance values are defined in the training framework. We just perform the operation (x-mean)*scale on the input x before feeding the n/w.

    Regards, Debapriya