SK-AM62A-LP: Low detection accuracy using edgeai-gst-apps

Akash Dhande

Part Number: SK-AM62A-LP

Hello TI Team,

I have been trying to use my yolox-nano model trained using edgeai-modelmaker with the default configuration and a custom dataset. I compile the model with tensor_bits set to 16 bits. I have referred 2 options for performing inference using this custom model -

onnx_ep.py script from edgeai-tidl-tools/examples
using my model with edgeai-gst-apps.

I have tried two SDK versions - r10.1(with r11.0 tidl tools patched on) and r11.1. I have trained the model using the same method on both. When performing inference with onnx_ep.py I am able to see the predictions as follows -

Predictions with r10.1(with r11.0 tidl tools patched on) -

Predictions with r11.1 -

The issues arises when I want to perform inference using the gstreamer pipeline that is being used in the edgeai-gst-apps/apps_python/app_edgeai.py. Following are the images generated as output after running app_edgeai.py on the same images -

Predictions with r11.1 app_edgeai.py

I am inclined to think that both the methods use a different preprocessing for the input method. I would love to get the same/similar results using the gstreamer pipeline from app_edgeai.py instead of onnx based inference from onnx_ep.py to ensure I am running the pipeline as fast as possible as my application prioritizes the predictions being real-time.

Best

11 months ago

0 Reese Grimsley 11 months ago

TI__Mastermind 20006 points

Hello,

Input data preprocessing is a likely cause here, but it could also be postprocessing. If you are using python application, you can also get access to the data input and output from the AI model from infer_pipe.py [1]. I'd recommend printing or viewing some of that data and comparing between onnxrt_ep.py and app_edgeai.py.

The preprocessing and postprocessing specification for your model will come from a param.yaml file near your model artifacts. Could you provide that param.yaml, the model_config.py entry you use for edgeai-tidl-tools/examples/osrt_python/onnxrt_ep.py, and the printout to the terminal when you run app_edgeai.py? Please also provide your YAML config file that you use with app_edgeai.py

For app_edgeai.py output, I'm mainly interested in the gstreamer pipeline itself. There should be some preprocessing parameters used directly there.
Note that if you trained with modelmaker, it may have added a few layers to the start of the model so that it takes in uint8 and handles preprocessing for you (meaning mean subtraction and scale multiplication
Alternatively, it could be a vizualization-threshold that is preventing the outputs from being visualized. That parameter can be supplied from the config YAML used by app_edgeai.py

[1] https://github.com/TexasInstruments/edgeai-gst-apps/blob/799dcbda54f829eb7b234bf93a5669addcc1b919/apps_python/infer_pipe.py#L110

0 Akash Dhande 10 months ago in reply to Reese Grimsley

Prodigy 30 points

Hi Reese, thank you for the response. While I try to debug by accessing the data in the intermediate stages, I am attaching the files you requested in this response.

params.yaml

calibration_dataset:
  dataset_info: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/dataset.yaml
  name: modelmaker
  num_classes: null
  num_frames: 24
  path: /opt/code/edgeai-modelmaker/data/projects/project1/dataset
  shuffle: true
  split: train
  type: ModelMakerDetectionDataset
dataset_category: coco
input_dataset:
  dataset_info: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/dataset.yaml
  name: modelmaker
  num_classes: null
  num_frames: 443
  path: /opt/code/edgeai-modelmaker/data/projects/project1/dataset
  shuffle: false
  split: val
  type: ModelMakerDetectionDataset
metric:
  dataset_category: coco
  label_offset_pred: 1
  run_dir: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200
  task_name: {}
model_info:
  compact_name: yolox-nano-lite-mmdet-coco-416x416
  metric_reference:
    accuracy_ap[.5:.95]%: null
  model_shortlist: 10
  recommended: true
  shortlisted: true
postprocess:
  detection_threshold: 0.05
  formatter:
    dst_indices:
    - 4
    - 5
    name: DetectionBoxSL2BoxLS
    src_indices:
    - 5
    - 4
  ignore_index: null
  keypoint: false
  logits_bbox_to_bbox_ls: false
  normalized_detections: false
  object6dpose: false
  reshape_list:
  - - -1
    - 5
  - - -1
    - 1
  resize_with_pad: true
  shuffle_indices: null
  squeeze_axis: null
preprocess:
  add_flip_image: false
  backend: cv2
  crop: 416
  data_layout: NCHW
  interpolation: null
  pad_color:
  - 114
  - 114
  - 114
  resize: 416
  resize_with_pad:
  - true
  - corner
  reverse_channels: true
session:
  artifacts_folder: artifacts
  c7x_codegen: false
  deny_list_from_start_end_node: null
  extra_inputs: null
  input_data_layout: NCHW
  input_details:
  - name: input
    shape:
    - 1
    - 3
    - 416
    - 416
    type: tensor(float)
  input_mean:
  - 0.0
  - 0.0
  - 0.0
  input_optimization: false
  input_scale:
  - 1.0
  - 1.0
  - 1.0
  model_file: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model.onnx
  model_folder: model
  model_id: od-8200
  model_path: model/model.onnx
  model_type: null
  num_inputs: 1
  num_tidl_subgraphs: 16
  output_details:
  - name: dets
    shape:
    - 1
    - 200
    - 5
    type: tensor(float)
  - name: labels
    shape:
    - 1
    - 200
    type: tensor(int64)
  output_feature_16bit_names_list_from_start_end: null
  quant_params_proto_path: true
  run_dir: od-8200
  run_dir_tree_depth: 3
  run_suffix: null
  runtime_options:
    accuracy_level: 1
    advanced_options:activation_clipping: 1
    advanced_options:add_data_convert_ops: 3
    advanced_options:bias_calibration: 1
    advanced_options:calibration_frames: 12
    advanced_options:calibration_iterations: 12
    advanced_options:high_resolution_optimization: 0
    advanced_options:output_feature_16bit_names_list: /multi_level_conv_obj.2/Conv_output_0,
      /multi_level_conv_reg.2/Conv_output_0, /multi_level_conv_cls.2/Conv_output_0,
      /multi_level_conv_obj.1/Conv_output_0, /multi_level_conv_reg.1/Conv_output_0,
      /multi_level_conv_cls.1/Conv_output_0, /multi_level_conv_obj.0/Conv_output_0,
      /multi_level_conv_reg.0/Conv_output_0, /multi_level_conv_cls.0/Conv_output_0
    advanced_options:params_16bit_names_list: ''
    advanced_options:pre_batchnorm_fold: 1
    advanced_options:quant_params_proto_path: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model_qparams.prototxt
    advanced_options:quantization_scale_type: 4
    advanced_options:weight_clipping: 1
    artifacts_folder: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/artifacts
    debug_level: 0
    import: 'no'
    inference_mode: 0
    object_detection:confidence_threshold: 0.05
    object_detection:meta_arch_type: 6
    object_detection:meta_layers_names_list: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model.prototxt
    object_detection:top_k: 500
    platform: J7
    tensor_bits: 16
    ti_internal_nc_flag: 83886080
    tidl_tools_path: /opt/edgeai/code/edgeai-benchmark/tools/tidl_tools_package/AM62A/tidl_tools
    version: '11.1'
  session_name: onnxrt
  shape_inference: true
  supported_machines: null
  target_device: AM62A
  target_machine: pc
  tensor_bits: 8
  tidl_offload: true
  tidl_onnx_model_optimizer: false
  tidl_tools_path: /opt/edgeai/code/edgeai-benchmark/tools/tidl_tools_package/AM62A/tidl_tools
  with_onnxsim: false
  work_dir: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work
task_type: detection

model_config.py -- Only the entry being used

"od-8200_onnxrt_coco_edgeai-mmdet_yolox_nano_lite_416x416_20220214_model_onnx": create_model_config(
        task_type="detection",
        source=dict(
            model_url="http://software-dl.ti.com/jacinto7/esd/modelzoo/latest/models/vision/detection/coco/edgeai-mmdet/yolox_nano_lite_416x416_20220214_model.onnx",
            meta_arch_url="http://software-dl.ti.com/jacinto7/esd/modelzoo/latest/models/vision/detection/coco/edgeai-mmdet/yolox_nano_lite_416x416_20220214_model.prototxt",
            infer_shape=True,
        ),
        preprocess=dict(
            resize=416,
            crop=416,
            data_layout="NCHW",
            pad_color=[114, 114, 114],
            resize_with_pad=[True, "corner"],
            reverse_channels=True,
        ),
        session=dict(
            session_name="onnxrt",
            model_path=os.path.join(
                models_base_path, "yolox_nano_lite_416x416_20220214_model.onnx"
            ),
            meta_layers_names_list=os.path.join(
                models_base_path, "yolox_nano_lite_416x416_20220214_model.prototxt"
            ),
            meta_arch_type=6,
            input_mean=[0, 0, 0],
            input_scale=[1, 1, 1],
            input_optimization=True,
        ),
        postprocess=dmodel-artifactsict(
            formatter="DetectionBoxSL2BoxLS",
            resize_with_pad=True,
            keypoint=False,
            object6dpose=False,
            normalized_detections=False,
            shuffle_indices=None,
            squeeze_axis=None,
            reshape_list=[(-1, 5), (-1, 1)],
            ignore_index=None,
        ),
        extra_info=dict(
            od_type="SSD",
            framework="MMDetection",
            num_images=numImages,
            num_classes=91,
            label_offset_type="80to90",
            label_offset=1,
        ),
    ),

Config file used for app_edgeai.py

object_detection.yaml

title: "Object Detection"
log_level: 2
inputs:
    input0:
        source: /dev/video-usb-cam0
        format: jpeg
        width: 1280
        height: 720
        framerate: 30
    input1:
        source: /opt/edgeai-test-data/videos/video0_1280_768.h264
        format: h264
        width: 1280
        height: 768
        framerate: 30
        loop: True
    input2:
        source: /opt/edgeai-tidl-tools/test_data/frame_004.jpg
        width: 1280
        height: 720
        index: 0
        framerate: 1
        # loop: True
models:
    model0:
        model_path: /opt/model_zoo/ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416
        viz_threshold: 0.05
outputs:
    output0:
        sink: kmssink
        width: 1920
        height: 1080
        overlay-perf-type: graph
    output1:
        sink: /opt/edgeai-test-data/output/output_video0.mkv
        width: 1920
        height: 1080
    output2:
        sink: /opt/edgeai-test-data/output/output_frame_004.jpg
        width: 1920
        height: 1080
    output3:
        sink: remote
        width: 1920
        height: 1080
        port: 8081
        host: 127.0.0.1
        encoding: jpeg
        overlay-perf-type: graph

flows:
    flow0: [input2,model0,output2,[320,150,1280,720]]

Output after running app_edgeai.py

root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# python3 app_edgeai.py ../configs/object_detection.yaml
libtidl_onnxrt_EP loaded 0x31477e30

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /opt/edgeai-tidl-tools/test_data/frame_004.jpg                |
 | Model Name: ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416               |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
Final number of subgraphs created are : 1, - Offloaded Nodes - 271, Total Nodes - 271
APP: Init ... !!!
  1797.990532 s: MEM: Init ... !!!
  1797.990617 s: MEM: Initialized DMA HEAP (fd=5) !!!
  1797.990873 s: MEM: Init ... Done !!!
  1797.990911 s: IPC: Init ... !!!
  1798.008254 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  1798.015870 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  1798.021921 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
  1798.021973 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
  1798.021986 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
  1798.025104 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0
  1798.025305 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1
  1798.025476 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2
  1798.025618 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3
  1798.025637 s:  VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
  1798.025666 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
==========[INPUT PIPELINE(S)]==========

[PIPE-0]

multifilesrc location=/opt/edgeai-tidl-tools/test_data/frame_004.jpg index=1 ! jpegdec ! videoscale qos=True ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)416, height=(int)416;" ! tiovxdlpreproc out-pool-size=4 tensor-format=1 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0


==========[OUTPUT PIPELINE]==========

appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0 src::pool-size=2
sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>"
! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! v4l2jpegenc ! multifilesink sync=False location=/opt/edgeai-test-data/output/output_frame_004.jpg

APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
  1800.680727 s: IPC: Deinit ... !!!
  1800.681445 s: IPC: DeInit ... Done !!!
  1800.681520 s: MEM: Deinit ... !!!
  1800.681538 s: DDR_SHARED_MEM: Alloc's: 62 alloc's of 107348152 bytes
  1800.681548 s: DDR_SHARED_MEM: Free's : 62 free's  of 107348152 bytes
  1800.681556 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes
  1800.681572 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!

Looking forward to hearing your thoughts! Thank you!

0 Reese Grimsley 10 months ago in reply to Akash Dhande

TI__Mastermind 20006 points

Hello,

So one difference I see is:

param.yaml includes " input_data_layout: NCHW"

And the printed GST string includes "tiovxdlpreproc out-pool-size=4 tensor-format=1"

Analyzing with "gst-inspect-1.0 tiovxdlpreproc", I see this plugin considers tensor-format=1 to mean NHWC instead of NCHW.
- This suggests that the preprocessing for the gstreamer pipeline is laying out the tensor in memory differently than the model will expect. If you are getting correct results with the ONNX API's but not with gstreamer, this may be the root cause.

This would suggest that something is wrong with how the param.yaml is being parsed by the app_edgeai.py (and dependencies) such that is uses the wrong tensor format. I am curiousif edgeai-gst-apps optiflow and apps_cpp have the same effect. Before suggesting alterations to source code, I would try running with those tools instead.

This would be strange though, since it should be the same as the param.yaml files we see in the model zoo.

Edit: Let me rescind this -- I was looking at tensor-format instead of channel-order. Let me look more closely and provide more input

BR,
Reese

0 Reese Grimsley 10 months ago in reply to Reese Grimsley

TI__Mastermind 20006 points

I'm not otherwise seeing obvious issues. I'll make the following recommendations:

- In ONNXrt_ep.py, print the input tensor before it is passed to ONNX. What are the minimum and maximum values? What pixel values do you see for the corners of the image

- in infer_pipe.py for edgeai-gst-apps, print the tensor that is pulled from the gstreamer pipeline. Do the same analysis as above. Do the pixel values fall in the same range? Are they wildly different? Since you have static images, it should be feasible to even compare individual pixels, though they will probably not bitmatch.

- similarly in infer_pipe.py, print the output of the result after the model runs. Is there anything present? You have a very low visualization threshold, so I assume it is nearly empty or only contains null detections, but let's confirm.

I'd also suggest trying the same yaml.config with edgeai-gst-apps/apps_cpp or edgeai-gst-apps/optiflow. It would be strange to see different results here, but perhaps that is the case.

I notice that you have reverse_channels=True, so images will process as BGR instead of RGB. This is what the tensor-format on tiovxdlpreproc will do. That should not be the source of the issue, as many networks will at least make valid detections on either format, albeit not at optimal accuracy

Does your model-artifacts folder have a dataset.yaml? Can you show this?

I notice that your num_classes = null in the param.yaml. That is suspicious too... Perhaps this will cause some postprocessing to get skipped since the class ID's from the model output will always be outside the range of [0, num_classes]

BR,
Reese

0 Akash Dhande 10 months ago in reply to Reese Grimsley

Prodigy 30 points

Hi Reese,
I am able to get outputs from edgeai-gst-apps using app_edgeai.py. I am attaching all the yaml files that I used for getting the predictions.

config.yaml

task_type: detection
dataset_category: coco
calibration_dataset:
  num_classes: null
  num_frames: 24
  name: modelmaker
  path: /opt/code/edgeai-modelmaker/data/projects/project1/dataset
  split: train
  shuffle: true
  type: ModelMakerDetectionDataset
  dataset_info: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/dataset.yaml
input_dataset:
  num_classes: null
  num_frames: 443
  name: modelmaker
  path: /opt/code/edgeai-modelmaker/data/projects/project1/dataset
  split: val
  shuffle: false
  type: ModelMakerDetectionDataset
  dataset_info: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/dataset.yaml
preprocess:
  resize: 416
  crop: 416
  data_layout: NCHW
  reverse_channels: true
  backend: cv2
  interpolation: null
  add_flip_image: false
  resize_with_pad:
  - true
  - corner
  pad_color:
  - 114
  - 114
  - 114
session:
  session_name: onnxrt
  work_dir: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work
  target_machine: pc
  target_device: AM62A
  run_suffix: null
  tidl_offload: true
  input_optimization: false
  input_data_layout: NCHW
  input_mean:
  - 0.0
  - 0.0
  - 0.0
  input_scale:
  - 1.0
  - 1.0
  - 1.0
  run_dir_tree_depth: 3
  c7x_codegen: false
  runtime_options:
    platform: J7
    version: '11.1'
    tidl_tools_path: /opt/edgeai/code/edgeai-benchmark/tools/tidl_tools_package/AM62A/tidl_tools
    artifacts_folder: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/artifacts
    tensor_bits: 16
    import: 'yes'
    advanced_options:quantization_scale_type: 4
    accuracy_level: 1
    debug_level: 0
    inference_mode: 0
    advanced_options:high_resolution_optimization: 0
    advanced_options:pre_batchnorm_fold: 1
    advanced_options:calibration_frames: 12
    advanced_options:calibration_iterations: 12
    advanced_options:activation_clipping: 1
    advanced_options:weight_clipping: 1
    advanced_options:bias_calibration: 1
    advanced_options:output_feature_16bit_names_list: /multi_level_conv_obj.2/Conv_output_0,
      /multi_level_conv_reg.2/Conv_output_0, /multi_level_conv_cls.2/Conv_output_0,
      /multi_level_conv_obj.1/Conv_output_0, /multi_level_conv_reg.1/Conv_output_0,
      /multi_level_conv_cls.1/Conv_output_0, /multi_level_conv_obj.0/Conv_output_0,
      /multi_level_conv_reg.0/Conv_output_0, /multi_level_conv_cls.0/Conv_output_0
    advanced_options:params_16bit_names_list: ''
    advanced_options:add_data_convert_ops: 3
    ti_internal_nc_flag: 83886080
    object_detection:meta_arch_type: 6
    object_detection:meta_layers_names_list: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model.prototxt
    object_detection:confidence_threshold: 0.05
    object_detection:top_k: 500
    advanced_options:quant_params_proto_path: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model_qparams.prototxt
  model_path: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/training/model.onnx
  tidl_tools_path: /opt/edgeai/code/edgeai-benchmark/tools/tidl_tools_package/AM62A/tidl_tools
  run_dir: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200
  num_tidl_subgraphs: 16
  model_id: od-8200
  model_type: null
  input_details:
  - name: input
    shape:
    - 1
    - 3
    - 416
    - 416
    type: tensor(float)
  output_details:
  - name: dets
    shape:
    - 1
    - 200
    - 5
    type: tensor(float)
  - name: labels
    shape:
    - 1
    - 200
    type: tensor(int64)
  num_inputs: 1
  extra_inputs: null
  tensor_bits: 8
  quant_params_proto_path: true
  supported_machines: null
  with_onnxsim: false
  shape_inference: true
  tidl_onnx_model_optimizer: false
  deny_list_from_start_end_node: null
  output_feature_16bit_names_list_from_start_end: null
  artifacts_folder: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/artifacts
  model_folder: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model
  model_file: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model.onnx
postprocess:
  reshape_list:
  - - -1
    - 5
  - - -1
    - 1
  detection_threshold: 0.05
  formatter:
    src_indices:
    - 5
    - 4
    dst_indices:
    - 4
    - 5
    name: DetectionBoxSL2BoxLS
  resize_with_pad: true
  normalized_detections: false
  shuffle_indices: null
  squeeze_axis: null
  ignore_index: null
  logits_bbox_to_bbox_ls: false
  keypoint: false
  object6dpose: false
metric:
  label_offset_pred: 1
model_info:
  metric_reference:
    accuracy_ap[.5:.95]%: null
  model_shortlist: 10
  compact_name: yolox-nano-lite-mmdet-coco-416x416
  shortlisted: true
  recommended: true

param.yaml

calibration_dataset:
  dataset_info: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/dataset.yaml
  name: modelmaker
  num_classes: null
  num_frames: 24
  path: /opt/code/edgeai-modelmaker/data/projects/project1/dataset
  shuffle: true
  split: train
  type: ModelMakerDetectionDataset
dataset_category: coco
input_dataset:
  dataset_info: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/dataset.yaml
  name: modelmaker
  num_classes: null
  num_frames: 443
  path: /opt/code/edgeai-modelmaker/data/projects/project1/dataset
  shuffle: false
  split: val
  type: ModelMakerDetectionDataset
metric:
  dataset_category: coco
  label_offset_pred: 1
  run_dir: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200
  task_name: {}
model_info:
  compact_name: yolox-nano-lite-mmdet-coco-416x416
  metric_reference:
    accuracy_ap[.5:.95]%: null
  model_shortlist: 10
  recommended: true
  shortlisted: true
postprocess:
  detection_threshold: 0.05
  formatter:
    dst_indices:
    - 4
    - 5
    name: DetectionBoxSL2BoxLS
    src_indices:
    - 5
    - 4
  ignore_index: null
  keypoint: false
  logits_bbox_to_bbox_ls: false
  normalized_detections: false
  object6dpose: false
  reshape_list:
  - - -1
    - 5
  - - -1
    - 1
  resize_with_pad: true
  shuffle_indices: null
  squeeze_axis: null
preprocess:
  add_flip_image: false
  backend: cv2
  crop: 416
  data_layout: NCHW
  interpolation: null
  pad_color:
  - 114
  - 114
  - 114
  resize: 416
  resize_with_pad:
  - true
  - corner
  reverse_channels: true
session:
  artifacts_folder: artifacts
  c7x_codegen: false
  deny_list_from_start_end_node: null
  extra_inputs: null
  input_data_layout: NCHW
  input_details:
  - name: input
    shape:
    - 1
    - 3
    - 416
    - 416
    type: tensor(float)
  input_mean:
  - 0.0
  - 0.0
  - 0.0
  input_optimization: false
  input_scale:
  - 1.0
  - 1.0
  - 1.0
  model_file: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model.onnx
  model_folder: model
  model_id: od-8200
  model_path: model/model.onnx
  model_type: null
  num_inputs: 1
  num_tidl_subgraphs: 16
  output_details:
  - name: dets
    shape:
    - 1
    - 200
    - 5
    type: tensor(float)
  - name: labels
    shape:
    - 1
    - 200
    type: tensor(int64)
  output_feature_16bit_names_list_from_start_end: null
  quant_params_proto_path: true
  run_dir: od-8200
  run_dir_tree_depth: 3
  run_suffix: null
  runtime_options:
    accuracy_level: 1
    advanced_options:activation_clipping: 1
    advanced_options:add_data_convert_ops: 3
    advanced_options:bias_calibration: 1
    advanced_options:calibration_frames: 12
    advanced_options:calibration_iterations: 12
    advanced_options:high_resolution_optimization: 0
    advanced_options:output_feature_16bit_names_list: /multi_level_conv_obj.2/Conv_output_0,
      /multi_level_conv_reg.2/Conv_output_0, /multi_level_conv_cls.2/Conv_output_0,
      /multi_level_conv_obj.1/Conv_output_0, /multi_level_conv_reg.1/Conv_output_0,
      /multi_level_conv_cls.1/Conv_output_0, /multi_level_conv_obj.0/Conv_output_0,
      /multi_level_conv_reg.0/Conv_output_0, /multi_level_conv_cls.0/Conv_output_0
    advanced_options:params_16bit_names_list: ''
    advanced_options:pre_batchnorm_fold: 1
    advanced_options:quant_params_proto_path: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model_qparams.prototxt
    advanced_options:quantization_scale_type: 4
    advanced_options:weight_clipping: 1
    artifacts_folder: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/artifacts
    debug_level: 0
    import: 'no'
    inference_mode: 0
    object_detection:confidence_threshold: 0.05
    object_detection:meta_arch_type: 6
    object_detection:meta_layers_names_list: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work/od-8200/model/model.prototxt
    object_detection:top_k: 500
    platform: J7
    tensor_bits: 16
    ti_internal_nc_flag: 83886080
    tidl_tools_path: /opt/edgeai/code/edgeai-benchmark/tools/tidl_tools_package/AM62A/tidl_tools
    version: '11.1'
  session_name: onnxrt
  shape_inference: true
  supported_machines: null
  target_device: AM62A
  target_machine: pc
  tensor_bits: 8
  tidl_offload: true
  tidl_onnx_model_optimizer: false
  tidl_tools_path: /opt/edgeai/code/edgeai-benchmark/tools/tidl_tools_package/AM62A/tidl_tools
  with_onnxsim: false
  work_dir: /opt/code/edgeai-modelmaker/data/projects/project1/run/20250929-140708_ep500_lr0.002_b64_16b/yolox_nano_lite/compilation/work
task_type: detection

dataset.yaml

info: {}
categories:
- id: 1
  name: vehicle
  supercategory: none
color_map:
- - 0
  - 0
  - 0
- - 0
  - 0
  - 255
- - 0
  - 255
  - 0
- - 0
  - 255
  - 255
- - 255
  - 0
  - 0
- - 255
  - 0
  - 255
- - 255
  - 255
  - 0
- - 255
  - 255
  - 255

Output video from edgeai-gst-apps using app_edgeai.py -

Output video from onnxrt_ep.py -

In my experiments so far setting reverse_channels=True/False and num_channels=null has not made any significant difference. Since I am able to get outputs from the model, my focus has shifted to getting the same quality of output as the onnxrt_ep.py.

When compared with the config.yaml from the model_zoo/ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416 I can see that my config.yaml has the following values set differently -

Values from my config.yaml

session:
  input_optimization: false
  input_mean:
  - 0.0
  - 0.0
  - 0.0
  input_scale:
  - 1.0
  - 1.0
  - 1.0
  
postprocess:
  reshape_list:
  - - -1
    - 5
  - - -1
    - 1

Values from model_zoo/ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416/config.yaml

session:
  input_optimization: true
  input_mean: null
  input_scale: null
  
postprocess:
  reshape_list: -- (param not present by default)

Please note that my model is quantized using tensor_bits: 16 whereas the default `model_zoo/ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416` seems to be using tensor_bits: 8. I will be trying to perform inference by changing these different parameters and trying to get as close as possible to the outputs from onnxrt_ep.py

I am reporting these findings to hopefully arrive at my final configs which give same quality predictions as the onnxrt_ep.py sooner than later. Your insights would be appreciated.

Thank you,

0 Reese Grimsley 10 months ago in reply to Akash Dhande

TI__Mastermind 20006 points

Hi Akash,

The difference between onnxrt_ep.py and app_edgeai.py is quite clear. The visual helps. I presume this is the exact same set of model artifacts used for each.PY script, yes?

Akash Dhande said:
Values from my config.yaml

The input_optimization portion means that the first couple of layers in the model handle input preprocessing, meaning mean subtraction and scale multiplication. Looks like your model is assuming those operations are not part of the model itself (input_optimization=false), and what's more is that they are unity (the operation will not change the data). Does the model_zoo version input a cast->Add->Mul sequence of layers at the start? Are the constant values here also 0.0 and 1.0 for Add (mean) and Mul (scale), respectively? I see mean=0 and scale=1 on my side

The reshape list portion should not be an issue.

I would strongly suggest analyzing the numerical outputs of the model to make sure that the range of X,Y coordinates and confidence values are realistic. It looks like you've appropriately set the viz_threshold quite low, but I want to be sure that this is actually preprocessing instead of postprocessing

Can you pass the model artifacts and tell which SDK version you are using? I can take a look too.

BR,
Reese

0 Akash Dhande 10 months ago in reply to Reese Grimsley

Prodigy 30 points

Hi Reese,

Reese Grimsley said:
The difference between onnxrt_ep.py and app_edgeai.py is quite clear. The visual helps. I presume this is the exact same set of model artifacts used for each.PY script, yes?

Exact same params were used with onnxrt_ep.py and app_edgeai.py.

It seems like I have mean=0 and scale=1 as well. Attaching some screenshots of the layers from the model.onnx from the model_zoo that I am using -

1. Add Layer

2. Mul Layer

Upon comparing with my yolox-nano-lite model trained using edgeai-modelmaker and a custom dataset, it does not have the same Add and Mul Layers as shown below-

I want to analyze the numeric outputs as you said, however it would be of great help if you can let me know what exact stage in the pipeline should be analyzed(with reasoning to help understand the intuition if possible).

I cannot share the model-artifacts on this public forum but would love to share it with you using any other private method. I would still be happy to post the general findings of our analysis on this public forum afterwards. Please suggest a more secure and private way to share my files with you.

Thank you

0 Reese Grimsley 10 months ago in reply to Akash Dhande

TI__Mastermind 20006 points

Hi Akash,

okay, so preprocessing is confirmed to be the same. Since you've used the same input images in both cases, that it also helpful. I am leaning towards postprocessing being the issue.

You can modify the infer_pipe.py file to print the result [1]. There will probably be two output tensors here.

For onnxrt_ep.py, that output tensor comes here [2]

Akash Dhande said:
Upon comparing with my yolox-nano-lite model trained using edgeai-modelmaker and a custom dataset, it does not have the same Add and Mul Layers as shown below-

That's probably assuming you do preprocessing outside the model. As long as you have that mean and scale set the same, then I see no issue.

[1] https://github.com/TexasInstruments/edgeai-gst-apps/blob/c6b3ce5e2a6c9d46a6a66af02f858059a24c400b/apps_python/infer_pipe.py#L110

[2] https://github.com/TexasInstruments/edgeai-tidl-tools/blob/e70106c59a5bec5a4f794b81b88534279b8a2312/examples/osrt_python/ort/onnxrt_ep.py#L215

0 Reese Grimsley 10 months ago in reply to Reese Grimsley

TI__Mastermind 20006 points

Hi Akash,

I've gotten your model via one of your colleagues and have been able to reproduce the issue. Even in a few of my own test scripts, I see reasonable output tensors from your model, but within infer_pipe.py, I see zero good detections. Indeed, this isn't just an issue on your side.

I pulled the tensors in infer_pipe.py and reordered the tensor to get the image back. I notice that when I do this, the reconstruction has a ton of column artifacts -- see how choppy each of these look. My code snippet for pulling this data from infer_pipe.py is below, called within the pipeline() function

            if self.pre_proc_debug:
                self.pre_proc_debug.log(str(input_img.flatten()))
            if True: #block of added code, will produce errors on non-ONNX models
                input_img2 = input_img.copy()
                input_img2 = np.transpose(input_img2, (0,2,3,1))
                input_img2 = np.squeeze(input_img2)
                import cv2
                cv2.imwrite('/root/model-test/in-tensor-gst.png', input_img2.astype( np.uint8))

            # Inference
            start = time()
            result = self.run_time(input_img)

I've run an identical pipeline with your model and the baseline OD-8200 model under /opt/model_zoo. The only difference is that the model_zoo version takes uint8's as input and yours takes floating point. Perhaps there is a bug in the tiovxdlpreproc plugin, because all other plugins are identical.

My suggested workaround would be to modify the model such that it uses the first few layers to apply cast, mean-subtraction, and scale-multiplication as model layers (though technically, your model would only need the 'cast' from uint8-float32, since your preprocessing values are unity). Then, the model will take uint8 input (which also improves performance from reduced DDR load!) and I expect this issue will not persist.

This can be done after the .ONNX model is produced, and we have python scripts that add these layers to the start of the graph [1][2]. Please give this a try and let me know how this changes

[1] https://github.com/TexasInstruments/edgeai-tidl-tools/blob/f73a576f5610a5011b83de05c9e30081c895b752/osrt-model-tools/osrt_model_tools/onnx_tools/tidl_onnx_model_utils/onnx_model_opt.py#L65 -- the function itself

[2] https://github.com/TexasInstruments/edgeai-tidl-tools/blob/e70106c59a5bec5a4f794b81b88534279b8a2312/examples/osrt_python/common_utils.py#L293 -- where this gets called in the scripts -- it ordinarily only runs once as a model is being downloaded, but you can shortcut this logic in Python

Processors

Processors forum

SK-AM62A-LP: Low detection accuracy using edgeai-gst-apps