SK-AM62A-LP: edgeai model maker error during compilation

Hello, an idea to resolve this issue please ?

Hi Mickael,

It is our pleasure to help. Sorry for the late response as yesterday was a public holiday in the US.

There is not much to conclude from the provided data. Please consider the following:

• Make sure that you include the target platform and config file when running modelmaker. This is an example: 

  ./run_modelmaker.sh AM62A config_detection.yaml

• It might be that the compilation option is not enabled in the confi_file.yaml. This is an example of that file: https://github.com/TexasInstruments/edgeai-tensorlab/blob/main/edgeai-modelmaker/config_detection.yaml. The last section is for compilation. Please make sure that it is enabled. 
• Were you able to run any of the default example before running this project?

Please, include your confi.yaml file in your next reply if these suggestions did not solve your issue. 

Best regards,

Qutaiba

Hi and thank you for your answer.

Yes i run this line in edgeai_modelmaker

./run_modelmaker.sh AM62A config_detection.yaml

And the content of config_detection.yaml

common:
    target_module: 'vision'
    task_type: 'detection'
    target_device: 'AM62A'
    # run_name can be any string, but there are some special cases:
    # {date-time} will be replaced with datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
    # {model_name} will be replaced with the name of the model
    run_name: '{date-time}/{model_name}'

dataset:
    # enable/disable dataset loading
    enable: True #False
    # max_num_files: [750, 250] #None

    # Object Detection Dataset Examples:
    # -------------------------------------
    # Example 1, (known datasets): 'widerface_detection', 'pascal_voc0712', 'coco_detection', 'udacity_selfdriving', 'tomato_detection', 'tiscapes2017_driving'
    # dataset_name: widerface_detection
    # -------------------------------------
    # Example 2, give a dataset name and input_data_path.
    # input_data_path could be a path to zip file, tar file, folder OR http, https link to zip or tar files
    # for input_data_path these are provided with this repository as examples:
    #    'http://software-dl.ti.com/jacinto7/esd/modelzoo/08_06_00_01/datasets/tiscapes2017_driving.zip'
    #    'http://software-dl.ti.com/jacinto7/esd/modelzoo/08_06_00_01/datasets/animal_detection.zip'
    # -------------------------------------
    # Example 3, give image folders with annotation files (require list with values for both train and val splits)
    # dataset_name: coco_detection
    # input_data_path: ["./data/projects/coco_detection/dataset/train2017",
    #                        "./data/projects/coco_detection/dataset/val2017"]
    # input_annotation_path: ["./data/projects/coco_detection/dataset/annotations/instances_train2017.json",
    #                        "./data/projects/coco_detection/dataset/annotations/instances_val2017.json"]
    # -------------------------------------
    dataset_name: parking_ploum_aug_5k
    input_data_path: '../../datasets/parking_weather_night_aug_5k.zip'

training:
    # enable/disable training
    enable: True #False

    # Object Detection model chosen can be changed here if needed
    # options are: 'yolox_s_lite', 'yolox_tiny_lite', 'yolox_nano_lite', 'yolox_pico_lite', 'yolox_femto_lite'
    model_name: 'yolox_s_lite'

    training_epochs: 30 #30
    batch_size: 8 #32
    learning_rate: 0.001
    num_gpus: 1 #1 #4

compilation:
    # enable/disable compilation
    enable: True #False
    tensor_bits: 8 #16 #32

I change the branch of edgeai-tensorlab to r9.02

But same error. Model training ok (on gpu version) but error with model artifacts ...

Hi Mickael, 

The config file you are using looks fine. The error message indicates that the artifacts are not available at "/opt/code/edgeai-modelmaker/data/projects/resized_640_640_weather_night_aug_3k/run/20240829-150353/yolox_s_lite/compilation/AM62A/work/od-8220/artifacts". I think that there might be some directory name issue here.  I am trying to regenerate the issue at my side. Meanwhile, would you please provide the directory tree under :opt/code/edgeai-modelmaker/data/projects/ ?

Best regards,

Qutaiba

Hi Qutaiba,

It seems that ONNX GraphSurgeon are missing from package, installed it.
Then, this is the log that i have when i launch the trainning

2024-09-05 09:20:44,697 - mmdet - INFO - Epoch(val) [1][1082]   bbox_mAP: 0.2370, bbox_mAP_50: 0.4190, bbox_mAP_75: 0.2250, bbox_mAP_s: 0.0820, bbox_mAP_m: 0.2700, bbox_mAP_l: 0.4830, bbox_mAP_copypaste: 0.237 0.419 0.225 0.082 0.270 0.483
/home/ubuntu/.pyenv/versions/py310/lib/python3.10/site-packages/mmcv/onnx/info.py:19: UserWarning: DeprecationWarning: This function will be deprecated in future. Welcome to use the unified model deployment toolbox MMDeploy: https://github.com/open-mmlab/mmdeploy
  warnings.warn(msg)
/home/ubuntu/.pyenv/versions/py310/lib/python3.10/site-packages/mmcv/tensorrt/init_plugins.py:51: UserWarning: DeprecationWarning: This function will be deprecated in future. Welcome to use the unified model deployment toolbox MMDeploy: https://github.com/open-mmlab/mmdeploy
  warnings.warn(msg)
/home/ubuntu/.pyenv/versions/py310/lib/python3.10/site-packages/torch/onnx/symbolic_opset9.py:1248: UserWarning: This model contains a squeeze operation on dimension 1. If the model is intended to be used with dynamic input shapes, please use opset version 11 to export the model.
  warnings.warn(
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Successfully exported ONNX model: /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training/model.onnx
Trained model is at: /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training

SUCCESS: ModelMaker - Training completed.

INFO:20240905-092059: model import is in progress - please see the log file for status.
configs to run: ['od-8220']
number of configs: 1

INFO:20240905-092059: parallel_run - parallel_processes:1 parallel_devices=[0]
TASKS                                                       |          |     0% 0/1| [< ]
INFO:20240905-092059: starting process on parallel_device - 0   0%|          || 0/1 [00:00<?, ?it/s]

INFO:20240905-092059: starting - od-8220
INFO:20240905-092059: model_path - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training/model.onnx
INFO:20240905-092059: model_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/work/od-8220/model/model.onnx
INFO:20240905-092059: quant_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/work/od-8220/model/model_qparams.prototxt
Downloading 1/1: /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training/model.onnx
Download done for /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training/model.onnx
Downloading 1/1: /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training/model.onnx
Download done for /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training/model.onnx
Traceback (most recent call last):
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 203, in _run_pipeline
    result = cls._run_pipeline_impl(basic_settings, pipeline_config, description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 176, in _run_pipeline_impl
    accuracy_result = accuracy_pipeline(description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 68, in __call__
    self.session.start()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/onnxrt_session.py", line 45, in start
    super().start()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 145, in start
    self.get_model()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 524, in get_model
    apply_input_optimization = self._optimize_model(model_file,
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 569, in _optimize_model
    from osrt_model_tools.onnx_tools import onnx_model_opt as onnxopt
ImportError: cannot import name 'onnx_model_opt' from 'osrt_model_tools.onnx_tools' (/home/ubuntu/.pyenv/versions/py310/lib/python3.10/site-packages/osrt_model_tools/onnx_tools/__init__.py)
cannot import name 'onnx_model_opt' from 'osrt_model_tools.onnx_tools' (/home/ubuntu/.pyenv/versions/py310/lib/python3.10/site-packages/osrt_model_tools/onnx_tools/__init__.py)
TASKS                                                       | 100%|██████████|| 1/1 [00:00<00:00,  4.75it/s]



INFO:20240905-092059: model inference is in progress - please see the log file for status.
configs to run: ['od-8220']
number of configs: 1

INFO:20240905-092059: parallel_run - parallel_processes:1 parallel_devices=[0]
TASKS                                                       |          |     0% 0/1| [< ]
INFO:20240905-092059: starting process on parallel_device - 0   0%|          || 0/1 [00:00<?, ?it/s]

INFO:20240905-092059: starting - od-8220
INFO:20240905-092059: model_path - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/training/model.onnx
INFO:20240905-092059: model_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/work/od-8220/model/model.onnx
INFO:20240905-092059: quant_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/work/od-8220/model/model_qparams.prototxt

INFO:20240905-092059: running - od-8220
INFO:20240905-092059: pipeline_config - {'task_type': 'detection', 'dataset_category': 'coco', 'calibration_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f5a64e496c0>, 'input_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f5a64e494b0>, 'preprocess': <edgeai_benchmark.preprocess.PreProcessTransforms object at 0x7f5a58539a50>, 'session': <edgeai_benchmark.sessions.onnxrt_session.ONNXRTSession object at 0x7f5a58539ab0>, 'postprocess': <edgeai_benchmark.postprocess.PostProcessTransforms object at 0x7f5a58539d50>, 'metric': {'label_offset_pred': 1}, 'model_info': {'metric_reference': {'accuracy_ap[.5:.95]%': None}, 'model_shortlist': 10}}
INFO:20240905-092059: infer  - od-8220 - this may take some time...Traceback (most recent call last):
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 203, in _run_pipeline
    result = cls._run_pipeline_impl(basic_settings, pipeline_config, description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 176, in _run_pipeline_impl
    accuracy_result = accuracy_pipeline(description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 87, in __call__
    param_result = self._run(description=description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 138, in _run
    output_list = self._infer_frames(description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 192, in _infer_frames
    is_ok = session.start_infer()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/onnxrt_session.py", line 77, in start_infer
    super().start_infer()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 171, in start_infer
    raise FileNotFoundError(error_message)
FileNotFoundError: ERROR:20240905-092059: artifacts_folder is missing, please run import (on pc) - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/work/od-8220/artifacts
ERROR:20240905-092059: artifacts_folder is missing, please run import (on pc) - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/work/od-8220/artifacts
TASKS                                                       | 100%|██████████|| 1/1 [00:00<00:00,  4.67it/s]


packaging artifacts to /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/pkg please wait...
WARNING:20240905-092100: could not package - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/work/od-8220
Traceback (most recent call last):
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/./scripts/run_modelmaker.py", line 141, in <module>
    main(config)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/./scripts/run_modelmaker.py", line 80, in main
    model_runner.run()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/edgeai_modelmaker/ai_modules/vision/runner.py", line 187, in run
    self.model_compilation.run()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/edgeai_modelmaker/ai_modules/vision/compilation/edgeai_benchmark.py", line 279, in run
    edgeai_benchmark.interfaces.package_artifacts(self.settings, self.work_dir, out_dir=self.package_dir, custom_model=True)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/interfaces/run_package.py", line 271, in package_artifacts
    with open(os.path.join(out_dir,'artifacts.yaml'), 'w') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240905-091542/yolox_s_lite/compilation/AM62A/pkg/artifacts.yaml'

Then, from my first where i used a docker for trainning, i have moved to a GPU server. On this PC i don't use docker.

Directory tree for project is :

~/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects$ tree -d
.
└── parking_ploum_aug_5k
    ├── dataset
    │   ├── annotations
    │   ├── images
    │   ├── train -> images
    │   └── val -> images
    └── run
        ├── 20240905-073523
        │   └── yolox_s_lite
        │       ├── compilation
        │       │   └── AM62A
        │       │       └── work
        │       │           └── od-8220
        │       │               └── model
        │       └── training
        ├── 20240905-084637
        │   └── yolox_s_lite
        │       ├── compilation
        │       │   └── AM62A
        │       │       └── work
        │       │           └── od-8220
        │       │               └── model
        │       └── training
        ├── 20240905-090928
        │   └── yolox_s_lite
        │       └── training
        └── 20240905-091542
            └── yolox_s_lite
                ├── compilation
                │   └── AM62A
                │       └── work
                │           └── od-8220
                │               └── model
                └── training

34 directories

Hi Mickael,

Thank you for providing the extra data. From the log it seems that you are missing the this library: "cannot import name 'onnx_model_opt' from 'osrt_model_tools.onnx_tools'". The fast way to get those is to clone the edgai-tidl-tools and start the setup script in the same virtual environment you are currently using. The link for the edgai-tidl-tools: https://github.com/TexasInstruments/edgeai-tidl-tools/tree/master. From there, follow the instructions in the readme: https://github.com/TexasInstruments/edgeai-tidl-tools/tree/master?tab=readme-ov-file#setup-on-x86_pc

When done, you can go back to modelmaker and start the training/compilation as usual.

An alternative path, if interested, you can use the Edge AI Studio: Model composer (https://dev.ti.com/modelcomposer) to train/compile the model. This online tool provide about the same capabilities as modelmaker. You can upload your dataset and start training online.

Please, let me know how it goes. 

Best regards,

Qutaiba

Hello,

Folowed your procedure with edgeai-tidl-tools but same results ...

Yes the online solution could be be a easier solution but it's not compatible with sdk 9.2 or 10

@Qutaiba Saleh:

I have the exact same issue as Mickael. However, some weeks ago the issue did not exist. Could you please provide updates for r9.2 and r9.1 for the edgeai-modelmaker.

edgeai-tensorlab/edgeai-benchmark depends on a python package from edgeai-tidl-tools and it was being used without a tag/version.

Now a tag has been added here to use a specific version:

https://github.com/TexasInstruments/edgeai-tensorlab/blob/main/edgeai-benchmark/requirements_pc.txt#L4

r8.6, r9.0, r9.1, r9.2 and main branches have ben updated with the correct fix.

Note: as of now main branch points to r9.2

Hi Manu Mathew, update requirement_pc.txt with your fix. Used r9.2.
Activate Pyenv py310
and launch

./setup_pc.sh

But same error ...

SUCCESS: ModelMaker - Training completed.

INFO:20240910-125717: model import is in progress - please see the log file for status.
configs to run: ['od-8220']
number of configs: 1

INFO:20240910-125717: parallel_run - parallel_processes:1 parallel_devices=[0]
TASKS                                                       |          |     0% 0/1| [< ]
INFO:20240910-125718: starting process on parallel_device - 0   0%|          || 0/1 [00:00<?, ?it/s]

INFO:20240910-125718: starting - od-8220
INFO:20240910-125718: model_path - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/training/model.onnx
INFO:20240910-125718: model_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/work/od-8220/model/model.onnx
INFO:20240910-125718: quant_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/work/od-8220/model/model_qparams.prototxt
Downloading 1/1: /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/training/model.onnx
Download done for /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/training/model.onnx
Downloading 1/1: /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/training/model.onnx
Download done for /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/training/model.onnx
Traceback (most recent call last):
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 203, in _run_pipeline
    result = cls._run_pipeline_impl(basic_settings, pipeline_config, description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 176, in _run_pipeline_impl
    accuracy_result = accuracy_pipeline(description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 68, in __call__
    self.session.start()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/onnxrt_session.py", line 45, in start
    super().start()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 145, in start
    self.get_model()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 524, in get_model
    apply_input_optimization = self._optimize_model(model_file,
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 569, in _optimize_model
    from osrt_model_tools.onnx_tools import onnx_model_opt as onnxopt
ImportError: cannot import name 'onnx_model_opt' from 'osrt_model_tools.onnx_tools' (/home/ubuntu/.pyenv/versions/py310/lib/python3.10/site-packages/osrt_model_tools/onnx_tools/__init__.py)
cannot import name 'onnx_model_opt' from 'osrt_model_tools.onnx_tools' (/home/ubuntu/.pyenv/versions/py310/lib/python3.10/site-packages/osrt_model_tools/onnx_tools/__init__.py)
TASKS                                                       | 100%|██████████|| 1/1 [00:00<00:00,  4.42it/s]



INFO:20240910-125718: model inference is in progress - please see the log file for status.
configs to run: ['od-8220']
number of configs: 1

INFO:20240910-125718: parallel_run - parallel_processes:1 parallel_devices=[0]
TASKS                                                       |          |     0% 0/1| [< ]
INFO:20240910-125718: starting process on parallel_device - 0   0%|          || 0/1 [00:00<?, ?it/s]

INFO:20240910-125718: starting - od-8220
INFO:20240910-125718: model_path - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/training/model.onnx
INFO:20240910-125718: model_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/work/od-8220/model/model.onnx
INFO:20240910-125718: quant_file - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/work/od-8220/model/model_qparams.prototxt

INFO:20240910-125718: running - od-8220
INFO:20240910-125718: pipeline_config - {'task_type': 'detection', 'dataset_category': 'coco', 'calibration_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f93ca8e2bf0>, 'input_dataset': <edgeai_benchmark.datasets.modelmaker_datasets.ModelMakerDetectionDataset object at 0x7f93cb35f670>, 'preprocess': <edgeai_benchmark.preprocess.PreProcessTransforms object at 0x7f92c6317a00>, 'session': <edgeai_benchmark.sessions.onnxrt_session.ONNXRTSession object at 0x7f92c6317a60>, 'postprocess': <edgeai_benchmark.postprocess.PostProcessTransforms object at 0x7f92c6317d00>, 'metric': {'label_offset_pred': 1}, 'model_info': {'metric_reference': {'accuracy_ap[.5:.95]%': None}, 'model_shortlist': 10}}
INFO:20240910-125718: infer  - od-8220 - this may take some time...Traceback (most recent call last):
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 203, in _run_pipeline
    result = cls._run_pipeline_impl(basic_settings, pipeline_config, description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 176, in _run_pipeline_impl
    accuracy_result = accuracy_pipeline(description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 87, in __call__
    param_result = self._run(description=description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 138, in _run
    output_list = self._infer_frames(description)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 192, in _infer_frames
    is_ok = session.start_infer()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/onnxrt_session.py", line 77, in start_infer
    super().start_infer()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/sessions/basert_session.py", line 171, in start_infer
    raise FileNotFoundError(error_message)
FileNotFoundError: ERROR:20240910-125718: artifacts_folder is missing, please run import (on pc) - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/work/od-8220/artifacts
ERROR:20240910-125718: artifacts_folder is missing, please run import (on pc) - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/work/od-8220/artifacts
TASKS                                                       | 100%|██████████|| 1/1 [00:00<00:00,  4.63it/s]


packaging artifacts to /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/pkg please wait...
WARNING:20240910-125718: could not package - /home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/work/od-8220
Traceback (most recent call last):
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/./scripts/run_modelmaker.py", line 141, in <module>
    main(config)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/./scripts/run_modelmaker.py", line 80, in main
    model_runner.run()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/edgeai_modelmaker/ai_modules/vision/runner.py", line 187, in run
    self.model_compilation.run()
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/edgeai_modelmaker/ai_modules/vision/compilation/edgeai_benchmark.py", line 279, in run
    edgeai_benchmark.interfaces.package_artifacts(self.settings, self.work_dir, out_dir=self.package_dir, custom_model=True)
  File "/home/ubuntu/TI/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/interfaces/run_package.py", line 271, in package_artifacts
    with open(os.path.join(out_dir,'artifacts.yaml'), 'w') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/TI/edgeai-tensorlab/edgeai-modelmaker/data/projects/parking_ploum_aug_5k/run/20240910-125317/yolox_s_lite/compilation/AM62A/pkg/artifacts.yaml'

I am not sure why it is not working for for you. 

Can you create a fresh python 3.10 environment and try this line: https://github.com/TexasInstruments/edgeai-tensorlab/blob/r9.2/edgeai-benchmark/requirements_pc.txt#L4

That is:

pip install git+https://github.com/TexasInstruments/edgeai-tidl-tools.git@09_02_09_00#subdirectory=scripts

After that, start a python console and then type the exact line that shows error in what you pasted above:

from osrt_model_tools.onnx_tools import onnx_model_opt as onnxopt

ok, result

(py310_TI) ubuntu@t2-le-45-gra11:~$ python
Python 3.10.14 (main, Sep  2 2024, 15:58:16) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from osrt_model_tools.onnx_tools import onnx_model_opt as onnxopt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.pyenv/versions/py310_TI/lib/python3.10/site-packages/osrt_model_tools/onnx_tools/onnx_model_opt.py", line 58, in <module>
    import onnx
ModuleNotFoundError: No module named 'onnx'

Hi thank you a lot !

With a new python venv and setup_pc.sh it works now !
Can you just tell me if i want to have a custom model for sdk 10.00.00.08 which branch i need to use please ?

Regards

Also can you indicate where i can change compiltation parameters please like in model composer :

Thank you !

Hello,

If i need to just compile a model already trainned.
I suppose i need to use this jupyter notebook
https://github.com/TexasInstruments/edgeai-tensorlab/blob/r9.2/edgeai-benchmark/tutorials/tutorial_detection.ipynb

Can you tell me what i need to modify to compile a model that is stored in edgeai-modelmaker folder please ?

Hi Michael,

Since you are using edgeai-modelmaker, it does both training and compilation. So you don't need to compile separately. 

(But if you are not using edgeai-modelmaker and you have trained/exported the onnx model, then you can use edgeai-benhcmark to compile the model). 

Yes i understand that. But if for example i want to recopile the model for sdk 10 ? or to change some parameters like detection top k ...

There is another e2e thread discussing model compilation: e2e.ti.com/.../sk-am62a-lp-edgeai-model-maker-custom-model-compilation