• State
  • +1 person also asked this people also asked this
  • Locked Locked
  • Replies 26 replies
  • Subscribers 102 subscribers
  • Views 1285 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: YOLOv8m compilation for TDA4VM

Mileva Simic
Prodigy 80 points
Part Number: TDA4VM

Tool/software:

Hi TI team,

 

We are working on deploying yolov8m to the TDA4VM. We are using Processor SDK J721 9.2.

Can you please check our steps, our understanding of the edgeai-tensorlab and advise how to proceed?

 

In the scope of edgeai-tensorlab, we are using the following:

0. Inputs:

  • Model is yolov8m onnx (default model from ultralytics converted to onnx, opset 11).
  • For the prototxt file we used attached file – yolov8m_onnx.prototxt (manually generated, based on the yolov8s prototxt from the online model)

1. edgeai-benchmark :When we set tidl_offload to false, it seems that everything is fine and we receive the result shown in the attached log – offload_false.log.
2. edgeai-benchmark : Setting tidl-offload to true

  • Verifying that environment is OK: We successfully ran YOLOv8s lite with the existing configuration in the detection_v2.py file (we found the attached model from previous discussions on the forum (link). Model is also deployed to the board successfully.
  • YOLOv8m:
    1. We tried to compile our YOLOv8m model from Ultralytics, converted into ONNX following yolov8s example. We used the same configuration and prototxt (with only the necessary changes based on our model graph, name of output node) because both of them trained on the COCO dataset.
    2. With our current setup, we receive the error shown in the yolov8m.log.
    3. Do you see some issues with this approach? Do you have some suggestions on how to create the appropriate prototxt file automatically?
    4. We noticed that to overcome issues related to the unsupported layers of YOLOv8m, we have to perform a "surgery" on the model in the model optimizer. We are proceeding now with this step to adapt our model for the benchmark. Our expectation is that this would give us model for the edgeai_benchmark, which would then be able to generate the necessary files for the model deployment on board. Correct?

            5. For the surgery we are using default function for replacement.


            6. Considering that the surgery functions also perform model training with 100 epochs, which requires a considerable amount of time, we would like to know what are the ways we could shorten or omit that training              process, in order to first perform model verification, and will train properly afterwards?


Thanks in advance.

  • 0
    Prodigy 80 points

    We followed the steps for surgery from the following discussion link, but so far we do not have the expected results, can you tell us if this is the correct way?

  • 0 in reply to Mileva Simic
    TI__Genius 11406 points

    Can you see if this helps: github.com/.../7

  • 0 in reply to Manu Mathew
    Prodigy 80 points

    Thanks for your response. We followed all the steps as you instructed, we cloned and installed mmyolo, applied the patch and attempted to run the script with one of the yolov8 config files from the mmyolo repository, but we had the following error: log.txt

    Also, if problem is with package versions could you tell us what are the recommended versions of the packages that are used? We used the following :

    Thanks in advance.

  • 0 in reply to Mileva Simic
    TI__Prodigy 10 points

    We are looking into the issue. We will inform you shortly.

  • 0
    TI__Prodigy 260 points

    There is some bug in model-surgery_v2, we are looking to fix it. Can you try using model-surgery_v1, passing the argument as " --model-surgery 1 ". This should work fine.

  • 0 in reply to Rekib Zaman
    Prodigy 80 points

    When I run surgery v1 with the argument you told me, I get the following error: FileNotFoundError: [Errno 2] No such file or directory: 'data/coco/annotations/instances_train2017.json', the entirre log: log.txt . Can you help me solve this error, it seems to me that this is being called from some Python package, where should I position instances_train2017.json so that it recognizes it? 

    Thanks in advance.

  • 0
    TI__Prodigy 260 points

    The error occured because, your current working directory is mmyolo/tools and the data folder is inside mmyolo. You can run it directly from the root directory, i.e. mmyolo.

    Run this :

    python3 tools/train.py configs/yolov5/yolov5_m-p6-v62_syncbn_fast_8xb16-300e_coco.py --model-surgery 1

  • 0 in reply to Rekib Zaman
    Prodigy 80 points

    Within the mmyolo directory there is no data directory with the dataset by default, for this reason I downloaded the dataset from the following site link and placed it inside the directory. However, I get ValueError: Key img_path is not in available keys, the entire log : log.txt . I assume that there is some error in the dataset, can you give me advice where could I import dataset from because I don't have access fro download to the official COCO site? Thanks in advance.

  • 0 in reply to Mileva Simic
    Prodigy 80 points

    I found scirpt download_dataset.py in mmyolo that will do it automatically, sorry for my hasty question.

  • 0 in reply to Mileva Simic
    TI__Genius 11406 points

    Glad that the issue is resolved.

  • 0 in reply to Manu Mathew
    Prodigy 80 points

    I downloaded the dataset by running the following command: 

    python3 tools/misc/download_dataset.py --dataset-name coco2017

    and unzipped it, but I get the same ValueError as I mentioned in the message above. Can you help me?

  • 0
    TI__Prodigy 260 points

    Can you try reinstalling this particular version : 

    pip install albumentations==1.3.1

  • 0 in reply to Rekib Zaman
    Prodigy 80 points

    That was the problem, now i managed to run the training script, thanks a lot. Is it possible to set the number of epochs as a command line argument because I couldn't find a place in the code where I could explicitly set the number? By default it takes a very long time, I would like to reduce the training duration.

  • 0 in reply to Mileva Simic
    TI__Prodigy 260 points

    You can set max_epochs = {some number} , in the config file

  • 0 in reply to Rekib Zaman
    Prodigy 80 points

    You mean into yolov8_m_syncbn_fast_8xb16-500e_coco.py for example?

  • 0 in reply to Mileva Simic
    Prodigy 80 points

    I managed to change the number of epochs in one of the configuration files, specifically in yolov8_s_syncbn_fast_8xb16-500e_coco.py. I got the following files as a result.

    :

    Can you tell me the best way to run this model on the board, how to prepare it for edgeai-benchmark which will generate everything needed? Is it best to convert it to onnx and if so, which way is best?

  • 0 in reply to Mileva Simic
    TI__Genius 11406 points

    Can you go through this note to understand how to train the model and export the onnx and ptototxt file. 

    https://github.com/TexasInstruments/edgeai-tensorlab/issues/7

    You can go through the reamde to understand further.

    Once the onnx and prototxt are exported, you can use the custom model compilation flow in edgeai-benchmark to compile the model.

  • 0 in reply to Manu Mathew
    Prodigy 80 points

    Okay, thanks. I followed all the steps from your link, as the output of the process started with this command 

    python3 tools/train.py <configfile> --model-surgery 1

    I get a .pth model. Also, within the mmyolo repository I saw that mmdeploy (github) is used to convert models (in my case from .pth to .onnx), but this part bothers me:

    I need to convert yolov8, especially yolov8m. Does this mean there is no way to convert for yolov8?

    Also, I'm interested in which pipeline was used to get yolov8s lite, whose zip is available in the discussion at the following link (I managed to put it in the input of the edgeai-benchmark and run it on the board with all the built artifacts)? I hope you can help me how to do same with yolov8m. Thanks in advance.

  • 0 in reply to Mileva Simic
    TI__Genius 11406 points

    We plan to publish our fork of mmdeploy soon as part of the upcoming update of edgeai-tensorlab - hopefully towards end of next week.

    Regarding yolov8m - we have not looked carefully into that flavor, but it is possible that the only change required may be in the input size (resize, crop) - and that too only if it requires a different input size compared to yolov8s

  • 0 in reply to Manu Mathew
    Prodigy 80 points

    Thank you very much, please let me know about new updates. Also, I would be grateful if you could share with me the piplines for getting yolov8s lite (whose zip is in the link on the forum), how you got it and what environments you used because it is the only model I managed to run on the board. that it is not included in the zoo model that already comes with the SDK?

  • 0 in reply to Mileva Simic
    TI__Genius 11406 points

    Hi Mileva,

    Can you take a look here and see if this is what you are looking for:

    github.com/.../detection_v2.py

  • 0 in reply to Manu Mathew
    Prodigy 80 points

    Thanks for the suggestion, but I already went through this code, I used this configuration to import the yolov8s lite model that I downloaded from the link mentioned above. My question is actually, how do you get the optimized onnx model and the prototxt file so that it can be used for the benchmark like in the link you sent me? How did you get that model and is the yolov8m I need even supported? 

  • 0 in reply to Mileva Simic
    TI__Genius 11406 points

    Okay, I understand your question now. Can you take a look at this writeup. It explains, how these model is trained and exported:

    https://github.com/TexasInstruments/edgeai-tensorlab/issues/7

  • 0 in reply to Manu Mathew
    Prodigy 80 points

    As I mentioned earlier, I followed all the steps from your link, as the output of the process started with this command 

    python3 tools/train.py <configfile> --model-surgery 1

    I get a .pth model. Also, within the mmyolo repository I saw that mmdeploy (github) is used to convert models (in my case from .pth to .onnx), but this part bothers me:

    I need to convert yolov8, especially yolov8m. Does this mean there is no way to convert for yolov8?

    Also, I'm interested in which pipeline was used to get yolov8s lite, whose zip is available in the discussion at the following link (I managed to put it in the input of the edgeai-benchmark and run it on the board with all the built artifacts)? I hope you can help me how to do same with yolov8m. Thanks in advance.

  • 0 in reply to Mileva Simic
    TI__Prodigy 260 points

    If you are using the patch update of mmyolo as mentioned by Manu, it supports onnx export within mmyolo itself and mmdeploy is not required. 

    In order to export the model to .onnx format you can use the following script - projects/easydeploy/tools/export_onnx.py

    here is an example to run the script : 

    """

    python projects/easydeploy/tools/export_onnx.py \
    configs/yolov6/yolov6_n_syncbn_fast_8xb32-400e_coco.py \
    work_dirs/yolov6_n_syncbn_fast_8xb32-400e_coco/epoch_60.pth \
    --work-dir work_dirs/output_dir \
    --img-size 640 640 \
    --batch 1 \
    --device cpu \
    --simplify \
    --opset 11 \
    --pre-topk 1000 \
    --keep-topk 100 \
    --iou-threshold 0.65 \
    --score-threshold 0.25 \
    --export-type YOLOv5 \
    --model-surgery 1

    """"

    You can keep export-type as YOLOv5

  • 0 in reply to Rekib Zaman
    Prodigy 80 points

    Thank you very much for your quick reply. When I run the same command only for another model, I get following error: 

     

    Do you have suggestion how to solve it? I believe that it is important that save_model be from mmyolo because of the model itself. Also, I checked mmyolo/utils: