SK-AM68: How to rebuild a model to TI_SOC (AM68a)

Hi,

Could you help me understand the exact issue here.

You have precompiled model artifacts i suppose they are named yolov5m6_640_ti_lite_44p1_62p9 and you want to run the model inference over Host Emulation ?

Thanks for the reply.

I want to run yolov5m6_640_ti_lite_44p1_62p9 on AM68a.
Can't I rebuild the model for AM68 on the host PC?

Thanks for the reply.

I looked at benchmark-on-ti-soc. Please let me check.
The param.yaml that I posted is the result of the STEP3 run.
Is it correct to understand that the file created on X86_PC can be converted for AM68a by moving it to AM68a and adding the execution of the make command?

・STEP1@X86_PC : Reference ONNX model -> download
wget yolov5_yolov5m6_640_ti_lite_44p1_62p9_onnx

・STEP2@X86_PC : Generate Model -> model-artifacts
python3 onnxrt_ep.py -c -m yolov5m6_640_ti_lite_44p1_62p9

・STEP3@AM68a : Host to TI_SOC
scp model-artifacts and models

・STEP4@AM68a :
mkdir build && cd build
cmake ../examples && make -j && cd ..
python3 ./scripts/gen_test_report.py

I copied edgeai-tidl-tools into AM68a and ran build, but it failed on the way.　@STEP4

---(cmake ../examples/ && make -j && cd..)--
/opt/edgeai-tidl-tools/exmaples/osrt_cpp/dlr/dlr_main.h:75:10 fatal error: dlr.h: No such or directory
---

Can you share model compilation logs ?

And list of directories created as part of model artifacts ?

You can use ls command in linux inside model artifacts folder.

Thanks for the reply.

Share the model compilation and ls logs.
Is it necessary to install OSRT_CPP on X86_PC?

---(ls logs@Host=X86_PC)---

Dockerfile
LICENSE
README.md
dockers
docs
entrypoint.sh
examples
export.log
gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu
gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
ls.log
model-artifacts
models
output_images
requirements_pc.txt
run_docker.sh
run_export_docker.sh
scripts
setup.sh
test_data
ti-cgt-c7000_3.1.0.LTS
ti_cgt_c7000_3.1.0.LTS_linux-x64_installer.bin
tidl_tools

---

---(model compilation log@Host=X86_PC)---

python3 onnxrt_ep.py -c -m yolov5m6_640_ti_lite_44p1_62p9

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['yolov5m6_640_ti_lite_44p1_62p9']


Processing config file #0 : /home/root/model-artifacts/yolov5m6_640_ti_lite_44p1_62p9/tempDir/detections_tidl_io_.qunat_stats_config.txt
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T  116687.81  .... ..... ... .... .....
#    1 . .. T  116615.32  .... ..... ... .... .....
Processing config file #0 : /home/root/model-artifacts/yolov5m6_640_ti_lite_44p1_62p9/tempDir/detections_tidl_io_.qunat_stats_config.txt
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T  107999.75  .... ..... ... .... .....
#    1 . .. T  107513.14  .... ..... ... .... .....
Processing config file #0 : /home/root/model-artifacts/yolov5m6_640_ti_lite_44p1_62p9/tempDir/detections_tidl_io_.qunat_stats_config.txt
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T  108074.89  .... ..... ... .... .....
#    1 . .. T  107758.42  .... ..... ... .... .....
Processing config file #0 : /home/root/model-artifacts/yolov5m6_640_ti_lite_44p1_62p9/tempDir/detections_tidl_io_.qunat_stats_config.txt
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T  106517.25  .... ..... ... .... .....
#    1 . .. T  106850.45  .... ..... ... .... .....
Processing config file #0 : /home/root/model-artifacts/yolov5m6_640_ti_lite_44p1_62p9/tempDir/detections_tidl_io_.qunat_stats_config.txt
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T  107163.97  .... ..... ... .... .....
#    1 . .. T  107449.07  .... ..... ... .... .....
Processing config file #0 : /home/root/model-artifacts/yolov5m6_640_ti_lite_44p1_62p9/tempDir/detections_tidl_io_.qunat_stats_config.txt
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T  106681.21  .... ..... ... .... .....
#    1 . .. T  106543.32  .... ..... ... .... .....
------------------ Network Compiler Traces -----------------------------
NC running for device: 1
Running with OTF buffer optimizations
successful Memory allocation
yolo_v3 is meta arch name
yolo_v3
Number of OD backbone nodes = 263
Size of odBackboneNodeIds = 263

Preliminary subgraphs created = 1
Final number of subgraphs created are : 1, - Offloaded Nodes - 391, Total Nodes - 391
TIDL Meta PipeLine (Proto) File  : ../../../models/public/yolov5m6_640_ti_lite_metaarch.prototxt  
yolo_v3
yolo_v3
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal

 ************** Frame index 1 : Running float import *************
INFORMATION: [TIDL_ResizeLayer] Resize_151 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_171 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_191 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
****************************************************
**          3 WARNINGS          0 ERRORS          **
****************************************************
The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INIT:Enabled
 0.7s:  VX_ZONE_ERROR:Enabled
 0.8s:  VX_ZONE_WARNING:Enabled
 0.4069s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!

**********  Frame Index 1 : Running float inference **********

**********  Frame Index 2 : Running fixed point mode for calibration **********
Empty prototxt path, running calibration

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

 
 
 *****************   Calibration iteration number 0 started ************************
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

 
 
 *****************   Calibration iteration number 0 completed ************************
 
 
 

 
 
 *****************   Calibration iteration number 1 started ************************
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

 
 
 *****************   Calibration iteration number 1 completed ************************
 
 
 

 
 
 *****************   Calibration iteration number 2 started ************************
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

 
 
 *****************   Calibration iteration number 2 completed ************************
 
 
 

 
 
 *****************   Calibration iteration number 3 started ************************
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

 
 
 *****************   Calibration iteration number 3 completed ************************
 
 
 

 
 
 *****************   Calibration iteration number 4 started ************************
 
 
 

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

 
 
 *****************   Calibration iteration number 4 completed ************************
 
 
 
INFORMATION: [TIDL_ResizeLayer] Resize_151 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_171 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] Resize_191 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize f
Running_Model :  yolov5m6_640_ti_lite_44p1_62p9  


Running shape inference on model ../../../models/public/yolov5m6_640_ti_lite_44p1_62p9.onnx


 
Completed_Model :     1, Name : yolov5m6_640_ti_lite_44p1_62p9                    , Total time :  782281.85, Offload Time :  121075.07 , DDR RW MBs : 0, Output File : py_out_yolov5m6_640_ti_lite_44p1_62p9_ADE_val_00001801.jpg
 ---

Due to regional holiday, the expert handling this thread  is currently out of office. Please expect a 1~2 day delay in responses.

Apologies for the delay, and thank you for you patience.

Thanks for the reply.

I understood that this problem was due to my lack of awareness.
(Converting for target using edgeai-benchmark.)

Thanks for the note. The expert handling this thread will get back with the response soon.

Thanks.

Using edgeai-benchmark, yolov5m6_640_ti_lite_44p1_62p9 (@pretrained_models) was converted to AM68A and run on EVM.
The results showed no errors, but slow processing times and no detection display when objects were placed.
---
Created : yolov5m6_640_ti_lite_44p1_62p9 = 150~160 ms
Example : ONR-OD-8220-yolox-s-lite-mmdet-coco-640x640 = 12ms
---
In/Out of yolov5m6_640_ti_lite_44p1_62p9 and ONR-OD-8220 as reference.
yolov5m6_640_ti_lite_44p1_62p9 : In=1x1x3x640x640, Out=1x1x200x6
ONR-OD-8220: input = 1x1x3x640x640, output = 1x1x200, 1x1x200x5

To run on AM68A's app_edgeai, do I need two separate outputs, one for the detection position and one for the label?

Thanks for the reply.

I will post param.yaml of yolov5_yolov5m6_640_ti_lite_44p1_62p9_onnx copied to AM68A. Please let me know if the path is incorrect.

-- Running log@AM68A --

root@am68a-sk:/opt/edgeai-gst-apps# ./apps_python/app_edgeai.py ./configs/object_detection.yaml
libtidl_onnxrt_EP loaded 0x990d710
Final number of subgraphs created are : 1, - Offloaded Nodes - 391, Total Nodes - 391
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   109.367789 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   109.370937 s:  VX_ZONE_INIT:Enabled
   109.370970 s:  VX_ZONE_ERROR:Enabled
   109.370977 s:  VX_ZONE_WARNING:Enabled
   109.373461 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   109.375392 s:  VX_ZONE_INIT:[tivxHostInitLocal:96] Initialization Done for HOST !!!
==========[INPUT PIPELINE(S)]==========

[PIPE-0]


 +---------------------------------------------------------------------------------+
 | Object Detection                                                                |
 +---------------------------------------------------------------------------------+
 +---------------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                                  |
 | Model Name: od-8120_onnxrt_coco_edgeai-yolov5_yolov5m6_640_ti_lite_44p1_62p9_onn|
 | Model Type: detection                                                           |
 +---------------------------------------------------------------------------------+
 | dl-inference                     :       154.47 ms   from  35    samples        |
 | total time                       :       164.80 ms   from  34    samples        |
 | framerate                        :         6.07 fps  from  34    samples        |
 +---------------------------------------------------------------------------------+

   118.499349 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:110] De-Initialization Done for HOST !!!
   118.504019 s:  VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
IPC: Deinit ... !!!
IPC: DeInit ... Done !!!
MEM: Deinit ... !!!
DDR_SHARED_MEM: Alloc's: 66 alloc's of 150610604 bytes
DDR_SHARED_MEM: Free's : 66 free's  of 150610604 bytes
DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes
MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!

---

---(param.yaml@od-8120_onnxrt_coco_edgeai-yolov5_yolov5m6_640_ti_lite_44p1_62p9_onnx)---

task_type: detection
dataset_category: coco
calibration_dataset:
  num_classes: 90
  num_frames: 25
  name: coco
  path: ./dependencies/datasets/coco
  split: val2017
  shuffle: true
  dataset_info: dataset.yaml
input_dataset:
  num_classes: 90
  num_frames: 5000
  name: coco
  path: ./dependencies/datasets/coco
  split: val2017
  shuffle: false
  dataset_info: dataset.yaml
preprocess:
  resize: 640
  crop: 640
  data_layout: NCHW
  reverse_channels: false
  backend: cv2
  interpolation: null
  add_flip_image: false
  resize_with_pad: true
  pad_color:
  - 114
  - 114
  - 114
session:
  session_name: onnxrt
  work_dir: ./work_dirs/modelartifacts/AM68A/8bits
  target_machine: pc
  target_device: AM68A
  run_suffix: null
  tidl_offload: true
  input_optimization: true
  input_data_layout: NCHW
  input_mean: null
  input_scale: null
  run_dir_tree_depth: 3
  runtime_options:
    platform: J7
    version: '9.0'
    tidl_tools_path: /home/parallels/Desktop/TIDL/edgeai-benchmark/tools/AM68A/tidl_tools
    artifacts_folder: artifacts
    tensor_bits: 8
    import: 'yes'
    accuracy_level: 1
    debug_level: 0
    priority: 0
    advanced_options:high_resolution_optimization: 0
    advanced_options:pre_batchnorm_fold: 1
    advanced_options:calibration_frames: 25
    advanced_options:calibration_iterations: 25
    advanced_options:quantization_scale_type: 0
    advanced_options:activation_clipping: 1
    advanced_options:weight_clipping: 1
    advanced_options:bias_calibration: 1
    advanced_options:output_feature_16bit_names_list: 228, 498, 808, 1118, 1428
    advanced_options:params_16bit_names_list: ''
    advanced_options:add_data_convert_ops: 3
    ti_internal_nc_flag: 83886080
    object_detection:confidence_threshold: 0.3
    object_detection:top_k: 200
    object_detection:meta_arch_type: 6
    object_detection:meta_layers_names_list: model/yolov5m6_640_ti_lite_metaarch.prototxt
    advanced_options:quant_params_proto_path: model/yolov5m6_640_ti_lite_44p1_62p9_qparams.prototxt
  model_path: model/yolov5m6_640_ti_lite_44p1_62p9.onnx
  tidl_tools_path: /home/parallels/Desktop/TIDL/edgeai-benchmark/tools/AM68A/tidl_tools
  run_dir: /home/parallels/Desktop/TIDL/edgeai-benchmark/work_dirs/modelartifacts/AM68A/8bits/od-8120_onnxrt_coco_edgeai-yolov5_yolov5m6_640_ti_lite_44p1_62p9_onnx
  num_tidl_subgraphs: 16
  model_id: od-8120
  model_type: null
  input_details:
  - name: imagesNet_IN
    shape:
    - 1
    - 3
    - 640
    - 640
    type: tensor(uint8)
  output_details:
  - name: detections
    shape:
    - 160
    - 6
    type: tensor(float)
  num_inputs: 1
  extra_inputs: null
  tensor_bits: 8
  quant_params_proto_path: true
  supported_machines: null
  artifacts_folder: artifacts
  model_folder: model
  model_file: model/yolov5m6_640_ti_lite_44p1_62p9.onnx
postprocess:
  detection_threshold: 0.3
  save_output: false
  formatter:
    src_indices:
    - 5
    - 4
    dst_indices:
    - 4
    - 5
    name: DetectionBoxSL2BoxLS
  resize_with_pad: true
  normalized_detections: false
  shuffle_indices: null
  squeeze_axis: null
  ignore_index: null
metric:
  label_offset_pred:
    0: 1
    1: 2
    2: 3
    3: 4
    4: 5
    5: 6
    6: 7
    7: 8
    8: 9
    9: 10
    10: 11
    11: 13
    12: 14
    13: 15
    14: 16
    15: 17
    16: 18
    17: 19
    18: 20
    19: 21
    20: 22
    21: 23
    22: 24
    23: 25
    24: 27
    25: 28
    26: 31
    27: 32
    28: 33
    29: 34
    30: 35
    31: 36
    32: 37
    33: 38
    34: 39
    35: 40
    36: 41
    37: 42
    38: 43
    39: 44
    40: 46
    41: 47
    42: 48
    43: 49
    44: 50
    45: 51
    46: 52
    47: 53
    48: 54
    49: 55
    50: 56
    51: 57
    52: 58
    53: 59
    54: 60
    55: 61
    56: 62
    57: 63
    58: 64
    59: 65
    60: 67
    61: 70
    62: 72
    63: 73
    64: 74
    65: 75
    66: 76
    67: 77
    68: 78
    69: 79
    70: 80
    71: 81
    72: 82
    73: 84
    74: 85
    75: 86
    76: 87
    77: 88
    78: 89
    79: 90
    80: 91
model_info:
  metric_reference:
    accuracy_ap[.5:.95]%: 44.1
  model_shortlist: null

---

Please let us know if this issue is resolved ?

So we can close the thread.

Sorry, I accidentally pressed the resolve button.
Not resolved.
Please check param.yaml.

Hi,

is this param.yaml file generated as part of artifact generation ?

You can refer param.yaml files that are there as part of other object detection models.

Thanks for the reply.

The submitted param.yaml was generated on X86_PC as part of the deliverables using edgeai-benchmark.
All generated data was copied to the AM68A and executed.

We are concerned that other object detection demos, including ONR-OD-8220-yolox-s-lite-mmdet-coco-640x640, generate two or more output_details, but only one for this model.
(see: yolov5m6_640_ti_lite_44p1_62p9.onnx)

Hi,

Understood the concern here.

Can you share the zip file with generated artifacts and onnx model ?

Thanks for the reply.

The results of rebuilding for AM68 on X86_PC can be viewed from the URL below.
(will be deleted after 1 week)

od-8120_onnxrt_coco_edgeai-yolov5_yolov5m6_640_ti_lite_44p1_62p9_onnx

Can you attach the zip file here, external links are not accessible to us.

Please tell me how to upload a ZIP file on this forum.

Could you check the same here : https://e2e.ti.com/support/site-support-group/site-support/f/site-support-forum/761619/faq-how-do-i-attach-a-file-or-link-to-my-post#:~:text=Click%20the%20%22Upload%22%20button%20to,instead%20of%20the%20file%20name.

yolov5_yolov5m6_640_ti_lite_44p1_62p9_onnx.zip

Thanks or the attachment.

Let me check the same and get back to you. 

Hi Kojima San

Can you confirm this observation that, the param.yaml file generated as part of benchmark repo compilation process for yolov5 is not working in edgeai linux sdk ?

If thats the case I can go ahead and file JIRA for this, let me know.

Hi,

Late reply due to illness.
The results of the run have already been presented in the Oct.25 post.
Nothing was displayed on the screen.
(In running, I modified config/object_detection.yaml for AM68).

Please let me know about the submission to JIRA.

Thank you for posting you response.

Currently am Out of Office, Please expect the delay in my responses until 20 Nov.

Thank you for your kind patience.

Hi, I await your reply.

Hi,

Apologies for delayed reply as I was on leave followed by bandwidth issue.

I have file the jira for this issue, adding jira link for TI's internal tracking purpose.

https://jira.itg.ti.com/browse/EDGEAI_ALGO-46