• State Resolved
  • Locked Locked
  • Replies 30 replies
  • Subscribers 102 subscribers
  • Views 853 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: Custom model compilation using edgeai tidl tools

Ramaseshan Subramanian
Intellectual 490 points
Part Number: TDA4VM

Hello TI Forum

I am using yolov8 to perform object detection on TDA4VM, I am using edgeai tidl tools to obtain artifacts that can be ported using edgeai gst apps. But when I try to compile the model there are some issues:

Model dictionary:

Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
{'yolov8' : {'artifacts_folder': '/home/mugu/edgeai-tidl-tools/model/',
'model_folder': '/home/mugu/edgeai-tidl-tools/model/',
'model_path': model_file_name,
'session_name': config['session_name']} ,
'task_type' : model_type,
'target_device': 'pc',
'postprocess':{'data_layout' : layout },
'preprocess' :{'data_layout' : layout ,
'mean':config['mean'],
'scale':config['scale'],
'resize':640,
'crop':640
}
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Contents in artifacts folder:

yolov8.zip

Issue:

  • 0
    TI__Mastermind 24041 points

    Hi,

    Due to limited bandwidth am unable to look into this thread at the moment.

    I will try to circle back in 1 week.

    Thank you for your patience.

  • 0 in reply to Pratik Kedar
    Intellectual 490 points

    HI Pratik

    I could understand the issue from you side, kindly take your time and respond us back as soon as possible.
    Thanks in advance

  • 0 in reply to Ramaseshan Subramanian
    TI__Mastermind 24041 points

    Thanks for understanding. 

  • 0 in reply to Pratik Kedar
    Intellectual 490 points

    Hello Pratik
    Any updates on this. I have uploaded the bin and some files produced during compilation in the zip, you can also find the model inside.

    yolo.zip
    thanks

  • 0
    Intellectual 285 points

    Hi Ramaseshan,

    I thought that the model has 3 operators that are not supported by TIDL. See the figure below:

    -- Joy

  • 0 in reply to Joy Yang
    Intellectual 490 points

    Hello Joy Yang

    I had put them on deny list and tried out the compilation, but I am facing a exception issue which I have mentioned in the starting, can you take a glance, and kindly try to look at our model, if there is an issue

    Thanks in advance

  • 0 in reply to Joy Yang
    Intellectual 490 points

    Also I could find more number of subgraph created, can you guide as to solve that too, is there any tool to replace layers graphically? 

    Thank you

  • 0 in reply to Joy Yang
    Intellectual 490 points

    Hello
    We have replaced unsupported ops of tidl with supported tidl layers, but still the compilation stops and crashes.
    I have included the model link and logs for your reference.
    Thanks!

    custom_model_logs.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    mugu@mugu-ASUS:~/edgeai-tidl-tools$ python3 compile_model.py /home/mugu/edgeai-tidl-tools/last.onnx /home/mugu/edgeai-tidl-tools/test_data /home/mugu/edgeai-tidl-tools/model-artifacts/
    Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
    tidl_tools_path = /home/mugu/edgeai-tidl-tools/tidl_tools
    artifacts_folder = /home/mugu/edgeai-tidl-tools/model-artifacts/tidl_output
    tidl_tensor_bits = 8
    debug_level = 3
    num_tidl_subgraphs = 16
    tidl_denylist =
    tidl_denylist_layer_name =
    tidl_denylist_layer_type =
    tidl_allowlist_layer_name =
    model_type =
    tidl_calibration_accuracy_level = 7
    tidl_calibration_options:num_frames_calibration = 21
    tidl_calibration_options:bias_calibration_iterations = 5
    mixed_precision_factor = -1.000000
    model_group_id = 0
    power_of_2_quantization = 2
    ONNX QDQ Enabled = 0
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Model link: 1145.model.zip

  • 0 in reply to Ramaseshan Subramanian
    TI__Mastermind 24041 points

    Hi, Thanks for continued interest in this issue.

    However due to limited bandwidth, i will circle back the in 1 week.

    Thanks 

  • 0 in reply to Pratik Kedar
    Intellectual 490 points

    Hello
    We have some deadlines, kindly look out the issues and reply us soon!
    Thank you

  • 0 in reply to Ramaseshan Subramanian
    TI__Mastermind 24041 points

    Hi,

    Regarding prioritization of this issue can you please connect with local FAE.

    Thanks

  • 0 in reply to Pratik Kedar
    Intellectual 490 points

    Hello 

    Where can I get the link or email for Local FAE? If possible kindly share here for reference. 

    Thank you

  • 0 in reply to Ramaseshan Subramanian
    TI__Mastermind 24041 points

    Hi,

    You can connect with your business team regarding same.

    Thanks

  • 0 in reply to Pratik Kedar
    Intellectual 490 points

    Hello 

    Is the TI team is available now, can anyone take over this issue kindly? 

    thank you

  • 0 in reply to Manu Mathew
    TI__Genius 11266 points

    Per the discussion at: https://github.com/TexasInstruments/edgeai-modeloptimization/issues/2

    Attached a sample yolov8 lite model and training script changes.

    Also includes TIDL compilation options used in edgeai-benchmark in the config yaml file corresponding to thsi config: https://github.com/TexasInstruments/edgeai-benchmark/blob/main/configs/detection_v2.py#L169

  • 0 in reply to Manu Mathew
    Intellectual 490 points

    Hello 

    Can you please elaborate the changes you have made in yolov8 for my reference? 

    Thank you

  • 0 in reply to Manu Mathew
    Intellectual 490 points

    Hello 

    One last doubt in my custom yolov8 model I had 3maxpool layers with kernal shape 5. So I replaced it with 3 maxpool layers with kernal shape 3. But in your example I could see 6 maxpool layers with 3 kernal shape. Can you please assist me regarding this. Also kindly look out my custom model zip I attached in this thread up

    thank you

  • 0 in reply to Ramaseshan Subramanian
    TI__Genius 11266 points

    kernel_size 3 is what is supported for Maxpool currently in TIDL.

    This was trained and by using the option --model-surgery 2 to the train.py script that I shared and the model optimization tool takes care of the layer changes such as changing activation layers from SiLU to ReLU and breaking down large Maxpool to 3x3.

    Other wise you can compare both the onnx graphs and understand the exact changes.

  • 0 in reply to Manu Mathew
    Intellectual 490 points

    Hello  
    How many 3x3 max pool layers is required to replace one 5x5 maxpool layer?

    Thanks

  • +1 in reply to Ramaseshan Subramanian
    Intellectual 285 points

    Hi 

    Two 3x3 max pool is required to replace one 5x5 maxpool. See more details at https://github.com/TexasInstruments/edgeai-yolov5.

    --Joy

  • +1 in reply to Joy Yang
    TI__Genius 11266 points

    Corrected typo above but posting again to avoid confusion: changing activation layers from SiLU to ReLU and breaking down large Maxpool to 3x3.

  • 0 in reply to Manu Mathew
    Intellectual 490 points

    Hello
    Thanks for the reply!
    One last question:
    1. Can we use artifacts generated from tidl tools to run inference using edgeai gst apps for my custom pretrained model? My model has been successfully compiled using tidl tools.
    2. I have model with number of channel as 1, I try to compile the model using edgeai benchmark, But I ended up by getting shape mismatch error. How can I can add or modify model dict to get number of channels as 1?

    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    'imageseg-3': dict(
    task_type='segmentation',
    calibration_dataset=imageseg_calib_dataset,
    input_dataset=imageseg_val_dataset,
    preprocess=preproc_transforms.get_transform_jai((128,96), (128,96), backend='cv2', interpolation=cv2.INTER_LINEAR),
    session=sessions.ONNXRTSession(**jai_session_cfg,
    runtime_options=settings.runtime_options_onnx_np2(),
    model_path=f'/home/mugu/edgeai-benchmark/model/seat.onnx'),
    postprocess=postproc_transforms.get_transform_segmentation_onnx(),
    model_info=dict(metric_reference={'accuracy_mean_iou%':57.77})
    )
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Thanks in advance

  • 0 in reply to Ramaseshan Subramanian
    TI__Genius 11266 points

    >>> Can we use artifacts generated from tidl tools

    What exactly do you mean by this? Did you compile by using edgeai-benchmark? If so generally it will work in edgeai-gst-apps. However, yolov8 has special post processing that is not yet support in edgeai-gist-apps - so you will have to modify the post processing in edgeai-gist-apps to support yolov8.

    The logic is under the flag logits_bbox_to_bbox_ls in edgeai-benchmark postprocess - the logic under that flag is not yet implemented in edgeai-gst-apps:

    https://github.com/TexasInstruments/edgeai-benchmark/blob/main/edgeai_benchmark/postprocess/__init__.py#L67

  • 0 in reply to Manu Mathew
    Intellectual 490 points

    Hello
    Here are the exact query:

    1) Can we use artifacts compiled from edgeai tidl tools to deploy on target using gst apps?
    2) I have successfully compiled my segmentation model using tidl tools, Can I compile the same model  using edgeai benchmark too?I could see there is default preprocessing function processing images in RGB scale i.e, Number of channels=3. But my model has gray scale, i.e number of channels=1.

    Thanks

  • 0 in reply to Ramaseshan Subramanian
    TI__Genius 11266 points

    The binaries generated in edgeai-tidl-tools and edgeai-benchmark are same, but currently there are some differences, which you can manually fix:

    (1) The directly structure should be same as what is generated by edgeai-benchmark (i.e. with the model folder and the artifacts folder)

    (2) param.yaml should be there in the format that edgeai-benchmark expects.

    I suspect that these details may be a bit overwhelming for you. Why not use edgeai-benchmark itself to compile? Especially since there is an example YOLOV8 model in edgeai-benchark. 

  • 0 in reply to Manu Mathew
    Intellectual 490 points

    Hello 

    This question is related to another model which is a pretrained segmentation model, this model has number of input channels as 1. I could see default configuration dictionary in edgeai benchmark preprocess images in RGB format. How can I changes to GRAY scaling that is num channel as 1

    Thanks

  • 0 in reply to Ramaseshan Subramanian
    TI__Genius 11266 points

    Can you please open another e2e thread for that - mixing these two in same thread might be confusing.