PROCESSOR-SDK-AM62A: Yolov8 deployment issue

Hello Machida-san,

I edited your post to put lines of code into blocks for easier parsing.

Can you supply your model artifacts tarball and the contents of  configs/yolov8.yaml here?

 The root of the problem that your seeing is that app_edgeai.py (and supporting code) is not finding the actual artifacts files. This usually means that a path was not set correctly or some files were misnamed. 

One thing I notice is that the artifacts directory is named "arfifacts", so perhaps there is a simple typo here. Often, the artifacts and model filepaths are read from the param.yaml file.

BR,
Reese

Hi Reese-san,

Thank you for your reply.
>Can you supply your model artifacts tarball and the contents of  configs/yolov8.yaml here?
Since model size is too big, so I sent this info via private message with file transfer system information.

>One thing I notice is that the artifacts directory is named "arfifacts", so perhaps there is a simple typo here. Often, >the artifacts and model filepaths are read from the param.yaml file.
I'm sorry I did not notice this typo...
I fixed it and re-try to run model. However I got following error.

---

root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/yolov8.yaml
libtidl_onnxrt_EP loaded 0xa032af0
Final number of subgraphs created are : 2, - Offloaded Nodes - 271, Total Nodes - 0
Invoke : ERROR: Unable to open network file /opt/model_zoo/yolov8_s_syncbn_fast_8xb16-100e_coco/artifacts/ �� _tidl_net.bin
Invoke : ERROR: Unable to open network file /opt/model_zoo/yolov8_s_syncbn_fast_8xb16-100e_coco/artifacts/�'
_tidl_net.bin
[ 100.872510] usb 1-1: reset high-speed USB device number 2 using xhci-hcd
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
107.444581 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
107.448576 s: VX_ZONE_INIT:Enabled
107.448621 s: VX_ZONE_ERROR:Enabled
107.448630 s: VX_ZONE_WARNING:Enabled
107.450135 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
107.450301 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
107.450453 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
107.450606 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
107.450621 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
107.451090 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
==========[INPUT PIPELINE(S)]==========

[PIPE-0]

v4l2src device=/dev/video-usb-cam0 brightness=128 contrast=32 saturation=32 ! capsfilter caps="image/jpeg, width=(int)1280, height=(int)720;" ! jpegdec ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)640, height=(int)640;" ! tiovxdlpreproc out-pool-size=4 scale-0=0.00392200006172061 scale-1=0.00392200006172061 scale-2=0.00392200006172061 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0

==========[OUTPUT PIPELINE]==========

appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0 src::pool-size=4
sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>"
! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! queue ! tiperfoverlay title=Object Detection ! kmssink sync=False max-lateness=5000000 qos=True processing-deadline=15000000 driver-name=tidss connector-id=40 plane-id=31 force-modesetting=True

+--------------------------------------------------------------------------+
| Object Detection |
+--------------------------------------------------------------------------+
+--------------------------------------------------------------------------+
| Input Src: /dev/video-usb-cam0 |
| Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco |
| Model Type: detection |
+--------------------------------------------------------------------------+
+--------------------------------------------------------------------------+[ 102.704846] kauditd_printk_skb: 1 callbacks suppressed
[ 102.704868] audit: type=1701 audit(1651172685.212:15): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=1450 comm="app_edgeai.py" exe="/usr/bin/python3.10" sig=11 res=1
[ 102.786374] audit: type=1334 audit(1651172685.296:16): prog-id=13 op=LOAD
[ 102.793319] audit: type=1334 audit(1651172685.300:17): prog-id=14 op=LOAD
[ 128.966763] audit: type=1334 audit(1651172711.476:18): prog-id=14 op=UNLOAD
[ 128.974371] audit: type=1334 audit(1651172711.476:19): prog-id=13 op=UNLOAD
Segmentation fault (core dumped)
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python#
---

I just copied network files which are included in "artifacts" folder...
I'm not sure why this error occured..

BR,

Hello,

It looks like the model artifacts are not correctly read here -- the filenames do not print correctly, so there is something wrong either in param.yaml or with the filenames themselves

Ryuuichi machida said:

Invoke : ERROR: Unable to open network file /opt/model_zoo/yolov8_s_syncbn_fast_8xb16-100e_coco/artifacts/ �� _tidl_net.bin
Invoke : ERROR: Unable to open network file /opt/model_zoo/yolov8_s_syncbn_fast_8xb16-100e_coco/artifacts/�'
_tidl_net.bin

I see the model artifacts you shared with me, and I see evidence that compilation did not complete. This is likely the cause of segmentation fault. The runtimes_visualization.svg is incomplete, and there should be a second subgraph for your network.

We should step back from the edgeai-gst-apps and running on the target, and first get this model compiled well and running with PC emulation using edgeai-tidl-tools.

For your network, I am using the follow model configuration in model_configs.py

    'yolov8_s_machida' : {
        'model_path' : os.path.join(models_base_path, 'best_coco_bbox_mAP_epoch_93.onnx'),
        'mean': [0, 0, 0],
        'scale' : [0.003921568627,0.003921568627,0.003921568627],
        'num_images' : numImages,
        'num_classes': 80,
        'model_type': 'od',
        'od_type' : 'yolo_v8',
        'meta_layers_names_list' : os.path.join(models_base_path, 'best_coco_bbox_mAP_epoch_93.prototxt'),
        'session_name' : 'onnxrt' ,
        'meta_arch_type' : 8,
        'optional_options': {
            'debug_level': 1,
        },

For me, compilation completes on a clean install, but I have challenges running the model with PC emulation. I see following error

Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/reese/1-edgeai/1-ti-tools/1-tidl-tools/9.2-tidl-tools-fresh/edgeai-tidl-tools/examples/osrt_python/ort/onnxrt_ep.py", line 240, in run_model
    imgs, output, proc_time, sub_graph_time, height, width  = infer_image(sess, input_images, config)
  File "/home/reese/1-edgeai/1-ti-tools/1-tidl-tools/9.2-tidl-tools-fresh/edgeai-tidl-tools/examples/osrt_python/ort/onnxrt_ep.py", line 129, in infer_image
    output = list(sess.run(None, {input_name: input_data}))
  File "/home/reese/1-edgeai/1-ti-tools/1-tidl-tools/9.2-tidl-tools-fresh/edgeai-tidl-tools/venv-9.2-tools-fresh/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 200, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running ReorderInput node. Name:'ReorderInput' Status Message: /onnxruntime/onnxruntime/contrib_ops/cpu/nchwc_ops.cc:17 virtual onnxruntime::common::Status onnxruntime::contrib::ReorderInput::Compute(onnxruntime::OpKernelContext*) const X_rank == 4 was false.

This often means that ONNX did not like the data format that TIDL returned for a few layers that need to run on Arm. The typical solution is to `export TIDL_RT_ONNX_VARDIM=1` in the calling linux environment, but that is not working right now. 

In this case, I think best strategy is to resolve the layers that are failing TIDL offload. That will solve above problem.

The layers not being accepted by TIDL are Maxpool layers with 5x5 size kernel -- we support max size 3x3 kernel. You can emulate 5x5 kernel with 2 maxpools of 3x3 kernel in sequence.

• Several methods for doing this, but I recommend tidl-onnx-model-optimizer within edgeai-tidl-tools. You can use by passing "-o" tag at the end of the compile command. It will make a copy of your ONNX model and make some changes to a few layers
  • I saw an issue on your model when the default rules. In the file edgeai-tidl-tools/scripts/osrt_model_tools/onnx_tools/tidl-onnx-model-optimizer/tidl_onnx_model_optimizer/ops.py, set all the "operation specific" rules to 'False" except for  convert_maxpool_to_cascaded_maxpool

I tried this and model compilation passes, as does model inference on x86 PC. Can you try the above model change and retry? 

BR,
Reese

Hello,

Thank you for your confirmation.

• Several methods for doing this, but I recommend tidl-onnx-model-optimizer within edgeai-tidl-tools. You can use by passing "-o" tag at the end of the compile command. It will make a copy of your ONNX model and make some changes to a few layers
  • I saw an issue on your model when the default rules. In the file edgeai-tidl-tools/scripts/osrt_model_tools/onnx_tools/tidl-onnx-model-optimizer/tidl_onnx_model_optimizer/ops.py, set all the "operation specific" rules to 'False" except for  convert_maxpool_to_cascaded_maxpool

=> I will try it later.

I tried this and model compilation passes, as does model inference on x86 PC. Can you try the above model change and retry? 
=> Is it possible to share your generated model via private message ?

Best Regards,

Hi,

I have one additional question.

 >You can use by passing "-o" tag at the end of the compile command
I could not find how I can perform compile for "tidl-onnx-model-optimizer".
Is there any repo which describe about above ?

Best Regards,

Hello,

Reese is out this week and won't be respond until next week. Regarding your question about ONNX model compilation, please check this FAQ: [FAQ] EDGE-AI-STUDIO: How do I use and retrain AI models from TI’s Model Zoo for my own dataset using TI Deep Learning (TIDL) on Edge AI AM6xA Processors?

Regards,

Jianzhong

Hi,

I performed compile and as Reese said, I also confirmed I did not observe error.
After that, I deployed this model on AM62A EVM with generated files which are generated on edgeai-benchmark.
However, I did not see any image on display. It seems that model work however something is wrong.

Do you have any idea why this phenomenon is observed ?

Best Regards

root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/yolov8.yaml
libtidl_onnxrt_EP loaded 0x3b498170

******** WARNING ******* : Could not open /opt/model_zoo/yolov8_s_syncbn_fast_8xb16-100e_coco/artifacts/allowedNode.txt for reading... Entire model will run on ARM without any delegation to TIDL !
Final number of subgraphs created are : 1, - Offloaded Nodes - 0, Total Nodes - 0
[   41.340035] usb 1-1: reset high-speed USB device number 2 using xhci-hcd
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
    47.978713 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
    47.982696 s:  VX_ZONE_INIT:Enabled
    47.982742 s:  VX_ZONE_ERROR:Enabled
    47.982751 s:  VX_ZONE_WARNING:Enabled
    47.984214 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
    47.984390 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
    47.984510 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
    47.984656 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
    47.984671 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
    47.985051 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
==========[INPUT PIPELINE(S)]==========

[PIPE-0]

v4l2src device=/dev/video-usb-cam0 brightness=128 contrast=32 saturation=32 ! capsfilter caps="image/jpeg, width=(int)1280, height=(int)720;" ! jpegdec ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)640, height=(int)640;" ! tiovxdlpreproc out-pool-size=4 scale-0=0.00392200006172061 scale-1=0.00392200006172061 scale-2=0.00392200006172061 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0


==========[OUTPUT PIPELINE]==========

appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0 src::pool-size=4
sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>"
! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! queue ! tiperfoverlay title=Object Detection ! kmssink sync=False max-lateness=5000000 qos=True processing-deadline=15000000 driver-name=tidss connector-id=40 plane-id=31 force-modesetting=True



 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+Exception in thread Thread-1 (pipeline):
                                     Traceback (most recent call last):
                                                                         File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
                                                                    self.run()
                                                                                File "/usr/lib/python3.10/threading.py", line 953, in run
                                                             self._target(*self._args, **self._kwargs)
                        File "/opt/edgeai-gst-apps/apps_python/infer_pipe.py", line 121, in pipeline
                        out_frame = self.post_proc(frame, result)
                                                                   File "/opt/edgeai-gst-apps/apps_python/post_process.py", line 225, in __call__
                                                                     bbox[..., self.model.formatter["dst_indices"]] = bbox_copy[
                                                IndexError: index 5 is out of bounds for axis 1 with size 1


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :      4834.22 ms   from   1    samples |
 +--------------------------------------------------------------------------+

Hi,

I saw some errors in the log:

******** WARNING ******* : Could not open /opt/model_zoo/yolov8_s_syncbn_fast_8xb16-100e_coco/artifacts/allowedNode.txt for reading... Entire model will run on ARM without any delegation to TIDL !

 +--------------------------------------------------------------------------+Exception in thread Thread-1 (pipeline):
                                     Traceback (most recent call last):
                                                                         File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
                                                                    self.run()
                                                                                File "/usr/lib/python3.10/threading.py", line 953, in run
                                                             self._target(*self._args, **self._kwargs)
                        File "/opt/edgeai-gst-apps/apps_python/infer_pipe.py", line 121, in pipeline
                        out_frame = self.post_proc(frame, result)
                                                                   File "/opt/edgeai-gst-apps/apps_python/post_process.py", line 225, in __call__
                                                                     bbox[..., self.model.formatter["dst_indices"]] = bbox_copy[
                                                IndexError: index 5 is out of bounds for axis 1 with size 1

You may want to fix these errors.

Regards,

Jianzhong

Hi,

Since I did not copy "allowedNote.txt" and "onnxrtMetadata.txt" to "artifacts" folder, so I copied them and re-run application.
Then I got following error.

---
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/yolov8.yaml
libtidl_onnxrt_EP loaded 0x980f6d0
Final number of subgraphs created are : 2, - Offloaded Nodes - 271, Total Nodes - 274
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
41.756330 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
41.760733 s: VX_ZONE_INIT:Enabled
41.760775 s: VX_ZONE_ERROR:Enabled
41.760784 s: VX_ZONE_WARNING:Enabled
41.762344 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
41.762531 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
41.762982 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
41.763095 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
41.763110 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
41.763488 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
41.789012 s: VX_ZONE_ERROR:[ownContextSendCmd:875] Command ack message returned failure cmd_status: -1
41.789053 s: VX_ZONE_ERROR:[ownNodeKernelInit:590] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
41.789064 s: VX_ZONE_ERROR:[ownNodeKernelInit:591] Please be sure the target callbacks have been registered for this core
41.789076 s: VX_ZONE_ERROR:[ownNodeKernelInit:592] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
41.789090 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:608] kernel init for node 0, kernel com.ti.tidl:1:3 ... failed !!!
41.789108 s: VX_ZONE_ERROR:[vxVerifyGraph:2159] Node kernel init failed
41.789118 s: VX_ZONE_ERROR:[vxVerifyGraph:2213] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
---

I'm refering following URLs, but it seems my issue is NOT related to them..
1. https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1227571/faq-sk-tda4vm-error-while-running-custom-compiled-model
I performed "./vx_app_arm_remote_log.out &" command, but I did not get any network error.

2. https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1296992/processor-sdk-am68a-tidl_rt_ovx-error-verify-openvx-graph-failed
I'm using below.
edgeai-tidl-tool : 09_02_09_00
edgeai-benchmark : r9.2

Do you have any idea to solve this issue ?

Best Regards,

Hi,

Previously, I did not run "tidl-onnx-model-optimizer" correctly. After generating "optimized_xxxx" model, I re-compiled model on edgeai-benchmark and edgeai-tidl-tool.
After generating files, I deploy them on AM62A EVM and run application.
As a result, I could confirm camera image on display and could got kind of following benchmark.
+--------------------------------------------------------------------------+
| Object Detection |
+--------------------------------------------------------------------------+
+--------------------------------------------------------------------------+
| Input Src: /dev/video-usb-cam0 |
| Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco |
| Model Type: detection |
+--------------------------------------------------------------------------+
| dl-inference : 81.89 ms from 83 samples |
| total time : 101.66 ms from 82 samples |
| framerate : 9.84 fps from 82 samples |
+--------------------------------------------------------------------------+
However, I'm still facing following issue.

* I continue to observe following error.
VX_ZONE_ERROR:[ownNodeKernelInit:590] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode

Could you pls give your idea to solve about above ?

Best Regards,


Log_for_E2E--20250116.txt

Hi,

Please give us a few days to look into this issue.

Regards,

Jianzhong

Hi Machida-san,

I cannot immediately tell the error from the log.

I will note that if you pass "-n" to the app_edgeai.py script, it will give more readable prints. This option disable ncurses, which makes stdout hard to read.

Ryuuichi machida said:

I performed "./vx_app_arm_remote_log.out &" command, but I did not get any network error.

We need to enable a higher verbosity through TIDL for VX to show more info. This is done with debug_level setting passed to TIDL. It must be set from edgeai_dl_inferer.py (path on SDK /usr/lib/python3.10/site-packages/edgeai_dl_inferer.py) like this: 

            if enable_tidl:
                runtime_options = { 
                    "tidl_tools_path": "null",
                    "artifacts_folder": artifacts,
                    "core_number": core_number,
---->               'debug_level': 2, #Add by RG                                                                 
                }   
                sess_options = _onnxruntime.SessionOptions()
                sess_options.log_severity_level=3
                ep_list = ["TIDLExecutionProvider", "CPUExecutionProvider"]

This is the best way to pass this option in 9.2 SDK when using edgeai-gst-apps. There are alternatives that I can share.

Once you have increased debug_level to 1 or 2, the /opt/vx_app_arm_remote_log.out should give more information. Please supply this full log when running the app_edgeai.py with -n tag.

If you have run the application many times without resetting the device, it is a good idea to do so. Applications that error out may leave HWA in unstable state if the app was killed before the core could gracefully stop. I recommend routinely rebooting when running into errors like this. 

BR,
Reese

Hello Reese-san,

Thank you for your reply.
>Once you have increased debug_level to 1 or 2, the /opt/vx_app_arm_remote_log.out should give more >information. Please supply this full log when running the app_edgeai.py with -n tag.
Attached is result when I added "debug_level : 2" command on "edgeai_dl_inferer.py". 

* $ ./app_edgeai.py ../configs/yolov8.yaml -n

>This is the best way to pass this option in 9.2 SDK when using edgeai-gst-apps. There are alternatives that I can >share.
Yes, I'm using SDK 09.02.00.05.

Best Regards,

Log_for_E2E--20250120.txt4152.vx_app_arm_remote_log.zip

Hi Reese-san,

After enabling debug_level to "2", I re-run "vx_app_arm_remote_log.out". Then I got following result.

---
root@am62axx-evm:/opt/vision_apps# ./vx_app_arm_remote_log.out &
[1] 1703
root@am62axx-evm:/opt/vision_apps# [MCU1_0] 0.043794 s: CIO: Init ... Done !!!
[MCU1_0] 0.043833 s: APP: Init ... !!!
[MCU1_0] 0.043863 s: MEM: Init ... !!!
[MCU1_0] 0.043881 s: MEM: Created heap (DDR_LOCAL_MEM, id=0, flags=0x00000004) @ af000000 of size 16777216 bytes !!!
[MCU1_0] 0.043920 s: MEM: Init ... Done !!!
[MCU1_0] 0.043946 s: IPC: Init ... !!!
[MCU1_0] 0.043962 s: IPC: 3 CPUs participating in IPC !!!
[MCU1_0] 0.044264 s: IPC: Waiting for HLOS to be ready ... !!!
[MCU1_0] 0.048614 s: #Sciserver Version: v2023.11.0.0REL.MCUSDK.MM.NN.PP.bb
[MCU1_0] 0.051387 s: ##RM_PM_HAL Version: vMM.NN.PP
[MCU1_0] 0.054334 s: ##Starting Sciserver..... PASSED
[MCU1_0] 12.441321 s: IPC: HLOS is ready !!!
[MCU1_0] 12.441404 s: IPC: Init ... Done !!!
[MCU1_0] 12.441428 s: APP: Syncing with 2 CPUs ... !!!
[MCU1_0] 12.441449 s: APP: Syncing with 2 CPUs ... Done !!!
[MCU1_0] 12.441468 s: REMOTE_SERVICE: Init ... !!!
[MCU1_0] 12.441538 s: REMOTE_SERVICE: Init ... Done !!!
[MCU1_0] 12.441561 s: FVID2: Init ... !!!
[MCU1_0] 12.441589 s: FVID2: Init ... Done !!!
[MCU1_0] 12.441605 s: VHWA: VPAC Init ... !!!
[MCU1_0] 12.441619 s: SCICLIENT: Sciclient_pmSetModuleState module=219 state=2
[MCU1_0] 12.441705 s: SCICLIENT: Sciclient_pmSetModuleState success
[MCU1_0] 12.441727 s: VHWA: LDC Init ... !!!
[MCU1_0] 12.441820 s: VHWA: LDC Init ... Done !!!
[MCU1_0] 12.441841 s: VHWA: MSC Init ... !!!
[MCU1_0] 12.442157 s: VHWA: MSC Init ... Done !!!
[MCU1_0] 12.442176 s: VHWA: VISS Init ... !!!
[MCU1_0] 12.442314 s: VHWA: VISS Init ... Done !!!
[MCU1_0] 12.442334 s: VHWA: VPAC Init ... Done !!!
[MCU1_0] 12.442359 s: VX_ZONE_INIT:Enabled
[MCU1_0] 12.442378 s: VX_ZONE_ERROR:Enabled
[MCU1_0] 12.442393 s: VX_ZONE_WARNING:Enabled
[MCU1_0] 12.443309 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MCU1-0
[MCU1_0] 12.443439 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target VPAC_LDC1
[MCU1_0] 12.443586 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target VPAC_MSC1
[MCU1_0] 12.443693 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target VPAC_MSC2
[MCU1_0] 12.443762 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target VPAC_VISS1
[MCU1_0] 12.443793 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[MCU1_0] 12.443814 s: APP: OpenVX Target kernel init ... !!!
[MCU1_0] 12.447518 s: APP: OpenVX Target kernel init ... Done !!!
[MCU1_0] 12.447545 s: VISS REMOTE SERVICE: Init ... !!!
[MCU1_0] 12.447587 s: VISS REMOTE SERVICE: Init ... Done !!!
[MCU1_0] 12.447605 s: APP: Init ... Done !!!
[MCU1_0] 12.447620 s: APP: Run ... !!!
[MCU1_0] 12.447634 s: IPC: Starting echo test ...
[MCU1_0] 12.447741 s: APP: Run ... Done !!!
[MCU1_0] 12.448331 s: IPC: Echo status: a530-0[.] r5f0-0[s] c75ss0[P]
[C7x_1 ] 12.071864 s: CIO: Init ... Done !!!
[C7x_1 ] 12.071882 s: APP: Init ... !!!
[C7x_1 ] 12.071893 s: SCICLIENT: Init ... !!!
[C7x_1 ] 12.071958 s: SCICLIENT: DMSC FW version [9.2.7--v09.02.07 (Kool Koala)]
[C7x_1 ] 12.071977 s: SCICLIENT: DMSC FW revision 0x9
[C7x_1 ] 12.071991 s: SCICLIENT: DMSC FW ABI revision 3.1
[C7x_1 ] 12.072006 s: SCICLIENT: Init ... Done !!!
[C7x_1 ] 12.072018 s: UDMA: Init ... !!!
[C7x_1 ] 12.072030 s: UDMA: Init ... Done !!!
[C7x_1 ] 12.072041 s: MEM: Init ... !!!
[C7x_1 ] 12.072054 s: MEM: Created heap (DDR_LOCAL_MEM, id=0, flags=0x00000004) @ b2000000 of size 117440512 bytes !!!
[C7x_1 ] 12.072081 s: MEM: Init ... Done !!!
[C7x_1 ] 12.072093 s: IPC: Init ... !!!
[C7x_1 ] 12.072105 s: IPC: 3 CPUs participating in IPC !!!
[C7x_1 ] 12.072310 s: IPC: Waiting for HLOS to be ready ... !!!
[C7x_1 ] 12.078244 s: IPC: HLOS is ready !!!
[C7x_1 ] 12.078314 s: IPC: Init ... Done !!!
[C7x_1 ] 12.078328 s: APP: Syncing with 2 CPUs ... !!!
[C7x_1 ] 12.441450 s: APP: Syncing with 2 CPUs ... Done !!!
[C7x_1 ] 12.441467 s: REMOTE_SERVICE: Init ... !!!
[C7x_1 ] 12.442262 s: REMOTE_SERVICE: Init ... Done !!!
[C7x_1 ] 12.442286 s: VX_ZONE_INIT:Enabled
[C7x_1 ] 12.442300 s: VX_ZONE_ERROR:Enabled
[C7x_1 ] 12.442313 s: VX_ZONE_WARNING:Enabled
[C7x_1 ] 12.443024 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1
[C7x_1 ] 12.443116 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1_PRI_2
[C7x_1 ] 12.443217 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1_PRI_3
[C7x_1 ] 12.443310 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1_PRI_4
[C7x_1 ] 12.443402 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1_PRI_5
[C7x_1 ] 12.443495 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1_PRI_6
[C7x_1 ] 12.443587 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1_PRI_7
[C7x_1 ] 12.443678 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target DSP_C7-1_PRI_8
[C7x_1 ] 12.443706 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[C7x_1 ] 12.443722 s: APP: OpenVX Target kernel init ... !!!
[C7x_1 ] 12.443928 s: APP: OpenVX Target kernel init ... Done !!!
[C7x_1 ] 12.443946 s: APP: Init ... Done !!!
[C7x_1 ] 12.443958 s: APP: Run ... !!!
[C7x_1 ] 12.443971 s: IPC: Starting echo test ...
[C7x_1 ] 12.444074 s: APP: Run ... Done !!!
[C7x_1 ] 12.448410 s: IPC: Echo status: a530-0[.] r5f0-0[P] c75ss0[s]
[C7x_1 ] 842.534263 s: VX_ZONE_ERROR:[tivxAlgiVisionCreate:335] Calling ialg.algInit failed with status = -1121
[C7x_1 ] 842.534315 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:926] tivxAlgiVisionCreate returned NULL
[C7x_1 ] 913.267666 s: VX_ZONE_ERROR:[tivxAlgiVisionCreate:335] Calling ialg.algInit failed with status = -1121
[C7x_1 ] 913.267720 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:926] tivxAlgiVisionCreate returned NULL
[C7x_1 ] 915.114007 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:910] Network version - 0x00000000, Expected version - 0x20240401
[C7x_1 ] 915.168162 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:910] Network version - 0x00000000, Expected version - 0x20240401
[C7x_1 ] 915.212917 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:910] Network version - 0x00000000, Expected version - 0x20240401
[C7x_1 ] 915.242046 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:910] Network version - 0x00000000, Expected version - 0x20240401
---

According to this, "Network version" show "0x00000000". Why this condition will happen ?

BR,

Hello,

Ah, seems like compilation failed then. This network version is filled in as a late stage of model compile process.

Can you provide log from compilation? Please also pass debug_level=2 as part of the compilation delegate options.

Can you run the model on x86 PC to emulate the C7xMMA? I expect this would fail for your current model artifacts.

I had also shared my model artifacts with you in a private message last week.

And apologies for slow responses -- I have had limited availability due to trainings and local holidays.

BR,
Reese

Hi,

At first, I run model using your "model artifacts" files which you sent via private message.
However, I saw same error as shown below.

---
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/yolov8.yaml -n
libtidl_onnxrt_EP loaded 0x39139a60
Final number of subgraphs created are : 1, - Offloaded Nodes - 277, Total Nodes - 277
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
105.881719 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
105.881858 s: VX_ZONE_INIT:Enabled
105.881873 s: VX_ZONE_ERROR:Enabled
105.881882 s: VX_ZONE_WARNING:Enabled
105.882797 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
105.883097 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
105.883455 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
105.883723 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
105.883756 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
105.884255 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
105.914990 s: VX_ZONE_ERROR:[ownContextSendCmd:875] Command ack message returned failure cmd_status: -1
105.915031 s: VX_ZONE_ERROR:[ownNodeKernelInit:590] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
105.915043 s: VX_ZONE_ERROR:[ownNodeKernelInit:591] Please be sure the target callbacks have been registered for this core
105.915055 s: VX_ZONE_ERROR:[ownNodeKernelInit:592] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
105.915070 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:608] kernel init for node 0, kernel com.ti.tidl:1:1 ... failed !!!
105.915086 s: VX_ZONE_ERROR:[vxVerifyGraph:2159] Node kernel init failed
105.915097 s: VX_ZONE_ERROR:[vxVerifyGraph:2213] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
[ 100.193010] usb 1-1: reset high-speed USB device number 2 using xhci-hcd
==========[INPUT PIPELINE(S)]==========

[PIPE-0]

v4l2src device=/dev/video-usb-cam0 brightness=128 contrast=32 saturation=32 ! capsfilter caps="image/jpeg, width=(int)1280, height=(int)720;" ! jpegdec ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)640, height=(int)640;" ! tiovxdlpreproc out-pool-size=4 scale-0=0.00392200006172061 scale-1=0.00392200006172061 scale-2=0.00392200006172061 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0

==========[OUTPUT PIPELINE]==========

appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0 src::pool-size=4
sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>"
! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! queue ! tiperfoverlay title=Object Detection ! kmssink sync=False max-lateness=5000000 qos=True processing-deadline=15000000 driver-name=tidss connector-id=40 plane-id=31 force-modesetting=True

107.762104 s: VX_ZONE_ERROR:[ownContextSendCmd:875] Command ack message returned failure cmd_status: -1
107.762161 s: VX_ZONE_ERROR:[ownNodeKernelInit:590] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
107.762189 s: VX_ZONE_ERROR:[ownNodeKernelInit:591] Please be sure the target callbacks have been registered for this core
107.762206 s: VX_ZONE_ERROR:[ownNodeKernelInit:592] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
107.762227 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:608] kernel init for node 0, kernel com.ti.tidl:1:1 ... failed !!!
107.762255 s: VX_ZONE_ERROR:[vxVerifyGraph:2159] Node kernel init failed
107.762271 s: VX_ZONE_ERROR:[vxVerifyGraph:2213] Graph verify failed
107.762464 s: VX_ZONE_ERROR:[ownGraphScheduleGraphWrapper:885] graph is not in a state required to be scheduled
107.762482 s: VX_ZONE_ERROR:[vxProcessGraph:813] schedule graph failed
107.762497 s: VX_ZONE_ERROR:[vxProcessGraph:818] wait graph failed
ERROR: Running TIDL graph ... Failed !!!
107.832261 s: VX_ZONE_ERROR:[ownContextSendCmd:875] Command ack message returned failure cmd_status: -1
107.832318 s: VX_ZONE_ERROR:[ownNodeKernelInit:590] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
107.832334 s: VX_ZONE_ERROR:[ownNodeKernelInit:591] Please be sure the target callbacks have been registered for this core
107.832348 s: VX_ZONE_ERROR:[ownNodeKernelInit:592] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
107.832366 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:608] kernel init for node 0, kernel com.ti.tidl:1:1 ... failed !!!
107.832394 s: VX_ZONE_ERROR:[vxVerifyGraph:2159] Node kernel init failed
107.832407 s: VX_ZONE_ERROR:[vxVerifyGraph:2213] Graph verify failed
107.832565 s: VX_ZONE_ERROR:[ownGraphScheduleGraphWrapper:885] graph is not in a state required to be scheduled
107.832580 s: VX_ZONE_ERROR:[vxProcessGraph:813] schedule graph failed
107.832593 s: VX_ZONE_ERROR:[vxProcessGraph:818] wait graph failed
ERROR: Running TIDL graph ... Failed !!
---

>Can you provide log from compilation? Please also pass debug_level=2 as part of the compilation delegate options.
Do you mean that you need log when I performed "run_python_example.sh" ?

>Can you run the model on x86 PC to emulate the C7xMMA?
I'm not sure how to change emulation device. Do you know any repo which is described about this ?

BR,

Hello,

My apologies, I see from the compilation logs on my side that there was error message hidden among the rest of the printout. I missed this originally in the model I shared. This fails for both your and my version of the model in the same way. The output artifacts were incomplete because the last portion of model compilation failed (seg fault after a line about Network Compiler). 

Please allow me to look into this in the backend for what might be wrong. I have seen the same behavior on 9.2.9 TIDL tools as well as 9.2.6

I have also tried this on a newer version of the SDK / tools, version 10.1 (TIDL tag 10_01_02_00, SDK version 10.1.0.5) with more success. Are you open to trying this? 

I will seek a workaround for 9.2 SDK, but I foresee 10.1 to be more straightforward.

BR,
Reese

Hello Reese-san,

Thank you for your reply.

>I have also tried this on a newer version of the SDK / tools, version 10.1 (TIDL tag 10_01_02_00, SDK version 10.1.0.5) with more success. Are you open to trying this? 
OK. I will try it. However, I'm concerned about compatibility with edgeai-benchmark.
Currently, you disclose up to "r9.2", is it OK to use this for "ver 10" ?

Best Regards,

Hello Machida-san,

Ryuuichi machida said:

Currently, you disclose up to "r9.2", is it OK to use this for "ver 10" ?

edgeai-benchmark tools were combined into a larger repo called edgeai-tensorlab shortly before the 10.0 SDK release. See here: https://github.com/TexasInstruments/edgeai-tensorlab/blob/main/edgeai-benchmark where there is a branch for r10.0

You can therefore clone the edgeai-tensorlab repo and use the same strategy from standalone edgeai-benchmark repo here in edgeai-tensorlab/edgeai-tensorlab directory.

You might try the following with tensorlab:

• Run setup scripts for tensorlab
• set your TIDL_TOOLS_PATH and LD_LIBRARY_PATH to point to edgeai-tensorlab/edgeai-benchmark/tools/AM62A/tidl_tools.
• Then, you can use the r9.2 edgeai-benchmark (your original one) to compile for SDK 10.0

There is also some compatibiltiy between 9.2 and 10.0 SDKs as described in the document here: https://github.com/TexasInstruments/edgeai-tidl-tools/blob/10_00_07_00/docs/backward_compatibility.md

• By following the directions here, you can use 10_00_00_07 release on top of 9.2 SDK. It requires some scripts run to update a few libraries, but otherwise you can use 10.0 tools with 9.2 SDK
• Let's only try this if we run into problem with the method I mentioned before with tensorlab

BR,
Reese

Hi Reese-san,

Thank you for your reply.
I will try to use AM62A Linux SDK ver 10.01.00.05, "edgeai-tidl-tools" ver 10_01_02_00" and "edgeai-benchmark" ver 10.0.2 by cloning "edgeai-tensorlab".

Best Regards, 

Hello Machida-san,

Ryuuichi machida said:

Thank you for your reply.
I will try to use AM62A Linux SDK ver 10.01.00.05, "edgeai-tidl-tools" ver 10_01_02_00" and "edgeai-benchmark" ver 10.0.2 by cloning "edgeai-tensorlab".

That sounds good. My only note here will be that you need to compile with your TIDL_TOOLS_PATH pointed to 10.1 SDK tools. Use the tools from edgeai-tidl-tools ((edgeai-tidl-tools/tidl_tools/) here, because edgeai-tensorlab does not yet have a 10.1 release. 

When setting up these repos, make sure that TIDL_TOOLS_PATH variable is not set in the Linux command line.

BR,
Reese

Hello Reese-san,

>My only note here will be that you need to compile with your TIDL_TOOLS_PATH pointed to 10.1 SDK tools. Use the tools from edgeai-tidl-tools ((edgeai-tidl-tools/tidl_tools/) here, because edgeai-tensorlab does not yet have a 10.1 release. 
Oh ! I'm trying to create new virtual machine environment separately to edgeai-tidl-tools. However from your reply, I understand that I should create both edgeai-tidl-tools and edgeai-tensorlab on same virtual machine condition. (However, compile environment should be separated by using such as venv and docker.)
Is my understanding correct ?

BR,

Hi Reese-san,

One additional thing.
I'm trying to use AM62A Linux SDK ver 10.01.00.05 as shown above, however I have problem on combination "SDK 10.01.00.05" and AM62A EVM which I own.
(Here is related thread.
(+) PROCESSOR-SDK-AM62A: Boot stack when I use "tiboot3-am62ax-gp-evm.bin" - Processors forum - Processors - TI E2E support forums)
According to my investitation, this issue is related to later SDK 10 and AM62A EVM version which I own. I also need to solve this to deploy model on EVM.
So, I may need more to confirm model...

Best Regards, 

Hello Machida-san,

Okay, let me know when that thread resolves and we can move forward here. From my own experience, I haven't had such difficulty with SDK 10.x. 

Is it true that you're using a GP EVM? Those should still be supported, but I think all of mine have been replaced with HS-FS EVMs now so haven't seen that myself. I'd also recommend trying a second SD card just in case something went broke with the SD card somehow. I have to replace mine sometimes 

BR,
Reese

Hello Machida-san,

I wanted to do a quick follow-up based on a recent learning. Supposedly tidl-tools from 9.x SDKs have shown this seg-faulting behavior on some of the later Linux Kernel versions. I was encountering this myself as well (failure to compile ANY model with 9.x tools, which explains why compilation of yolov8 was working several months ago, but does not work now. 

I tried downgrading my linux kernel to 6.5.5, and was able to compile the models. Otherwise, I'm running Ubuntu 22.04LTS with kernel 6.8.0 -- in this version, compile is failing for me. This may be the root of the problem -- what is your kernel version in the x86 PC?

BR,
Reese

Hello Reese-san,

>I tried downgrading my linux kernel to 6.5.5, and was able to compile the models. Otherwise, I'm running Ubuntu 22.04LTS with kernel 6.8.0 -- in this version, compile is failing for me. This may be the root of the problem -- what is your kernel version in the x86 PC?
I created linux envirnment on VMware. And my ubuntu kernel version was below.
* 6.8.0-52-generic

Maybe same as your condition...

By the way, previously I compiled yolox model. I used 6.5.0-17-generic at that time.

Best Regards,

Hi Machida-san,

Ryuuichi machida said:

Maybe same as your condition...

Yes, that is quite likely. 

Therefore, I think the simplest course of action is to update to 10.1 SDK. It is perhaps too inconvenient to downgrade your OS kernel for this purpose, but maybe this is not so difficult with a VM. Unfortunately, Docker containers would not be a viable solution since they share kernel with host OS.

Hi, Reese-san,

I could avoid openVX error by using following combination and run model.
>AM62A Linux SDK ver 10.01.00.05, "edgeai-tidl-tools" ver 10_01_02_00" and "edgeai-benchmark" ver 10.0.2 

- 1 -
>My only note here will be that you need to compile with your TIDL_TOOLS_PATH pointed to 10.1 SDK tools. Use the tools from edgeai-tidl-tools ((edgeai-tidl-tools/tidl_tools/) here, because edgeai-tensorlab does not yet have a 10.1 release.
You described above, but I compiled by using "tidl-tools" which is located on "edgeai-tensorlab".
I was trying to change environment value to point to "tidl-tools" directory which is located on "edgeai-tidl-tools" by modifing "run_set_env.sh"
But I got kinds of "device" error. (It seems that "tidl-tools" which is located on "edegeai-tidl-tools" is NOT located under each "device name".)
>export TIDL_TOOLS_PATH=$(pwd)/tools/${TARGET_SOC}/tidl_tools 

Therefore, I used  "tidl-tools" which is located on "edgeai-tensorlab". However custom model run correctly.
(Please see attaced "E2E--20250131.txt" about detail.)
Is there any problem above my environment for compiling ?

- 2 -
In case of above "1", as I said I could run model, but I could not see any bounding box.
So, I confirmed "confidence" parameter by modifing python script. Then I got following result.
(I'm looking "bbox" variable.)

viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]

This shows no confidence, so I changed "input scale" parameter which is defined edgeai-benchmark and edgeai-tidl-tools
After that I got bounding box, but something is strange.

viz_threshold is [-0.08125 0.50625 0.15625 0.65625 0. 1. ]
viz_threshold is [-0.16875 0.84375 0.13125 1.11875 0. 1. ]
viz_threshold is [0.26875 0.85625 0.55625 1.0312499 0. 1. ]
viz_threshold is [0.29375 0.85625 0.59375 1.11875 0. 1. ]
viz_threshold is [0.36875 0.85625 0.66875 1.11875 0. 1. ]
viz_threshold is [0.54375 0.81875 0.84375 1.11875 0. 1. ]
viz_threshold is [0.6008245 0.85625 0.90625 1.11875 0. 1. ]
viz_threshold is [0.64375 0.85625 0.93125 1.00625 0. 1. ]
viz_threshold is [0.71875 0.85625 0.96875 1.03125 0. 1. ]
viz_threshold is [0.79375 0.85625 1.09375 1.03125 0. 1. ]
viz_threshold is [0.81875 0.93125 1.11875 1.03125 0. 1. ]
viz_threshold is [0.86125696 0.81875 1.14375 1.01875 0. 1. ]

inference time became too slow. 

+--------------------------------------------------------------------------+
| Object Detection |
+--------------------------------------------------------------------------+
+--------------------------------------------------------------------------+
| Input Src: /dev/video-usb-cam0 |
| Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco |
| Model Type: detection |
+--------------------------------------------------------------------------+
| dl-inference : 3904.98 ms from 5 samples |
| total time : 3924.09 ms from 4 samples |
| framerate : 0.25 fps from 4 samples |
+--------------------------------------------------------------------------+

Parameter difference of "input scale" is below.

Previous : 0.003921568627
New : 1

I would like to see something reference for model parameter of "yolov8", but it seems there is no reference...
Do you have any configuration example for "yolov8" ?

Best Regards,

root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/yolov8.yaml
libtidl_onnxrt_EP loaded 0x16185c60
Final number of subgraphs created are : 1, - Offloaded Nodes - 277, Total Nodes - 277
APP: Init ... !!!
   138.561868 s: MEM: Init ... !!!
   138.561944 s: MEM: Initialized DMA HEAP (fd=5) !!!
   138.562138 s: MEM: Init ... Done !!!
   138.562171 s: IPC: Init ... !!!
   138.579820 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   138.584586 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   138.584753 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
   138.584770 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
   138.584782 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
   138.585998 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-0
   138.586316 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-1
   138.586581 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-2
   138.586851 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-3
   138.586892 s:  VX_ZONE_INFO: [tivxInitLocal:126] Initialization Done !!!
   138.586925 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
[  131.835146] usb 1-1: reset high-speed USB device number 2 using xhci-hcd
==========[INPUT PIPELINE(S)]==========

[PIPE-0]

v4l2src device=/dev/video-usb-cam0 brightness=128 contrast=32 saturation=32 ! capsfilter caps="image/jpeg, width=(int)1280, height=(int)720;" ! jpegdec ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)640, height=(int)640;" ! tiovxdlpreproc out-pool-size=4 scale-0=0.00392200006172061 scale-1=0.00392200006172061 scale-2=0.00392200006172061 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0


==========[OUTPUT PIPELINE]==========

appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0 src::pool-size=4
sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>"
! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! queue ! tiperfoverlay title=Object Detection ! kmssink sync=False max-lateness=5000000 qos=True processing-deadline=15000000 driver-name=tidss connector-id=40 plane-id=31 force-modesetting=True fd=38



 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        65.15 ms   from   9    samples |
 | total time                       :        86.02 ms   from   8    samples |
 | framerate                        :        11.62 fps  from   8    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        61.65 ms   from  22    samples |
 | total time                       :        80.49 ms   from  21    samples |
 | framerate                        :        12.42 fps  from  21    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        61.46 ms   from  35    samples |
 | total time                       :        79.90 ms   from  33    samples |
 | framerate                        :        12.52 fps  from  33    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        60.65 ms   from  48    samples |
 | total time                       :        79.14 ms   from  47    samples |
 | framerate                        :        12.64 fps  from  47    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        60.44 ms   from  61    samples |
 | total time                       :        78.75 ms   from  60    samples |
 | framerate                        :        12.70 fps  from  60    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        60.13 ms   from  74    samples |
 | total time                       :        78.63 ms   from  72    samples |
 | framerate                        :        12.72 fps  from  72    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        60.06 ms   from  87    samples |
 | total time                       :        78.44 ms   from  85    samples |
 | framerate                        :        12.75 fps  from  85    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.89 ms   from  100   samples |
 | total time                       :        78.25 ms   from  98    samples |
 | framerate                        :        12.78 fps  from  98    samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.73 ms   from  113   samples |
 | total time                       :        78.14 ms   from  111   samples |
 | framerate                        :        12.80 fps  from  111   samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.74 ms   from  125   samples |
 | total time                       :        78.16 ms   from  124   samples |
 | framerate                        :        12.79 fps  from  124   samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.69 ms   from  138   samples |
 | total time                       :        78.11 ms   from  137   samples |
 | framerate                        :        12.80 fps  from  137   samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.65 ms   from  151   samples |
 | total time                       :        78.03 ms   from  150   samples |
 | framerate                        :        12.81 fps  from  150   samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.55 ms   from  164   samples |
 | total time                       :        77.93 ms   from  163   samples |
 | framerate                        :        12.83 fps  from  163   samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.56 ms   from  177   samples |
 | total time                       :        77.95 ms   from  176   samples |
 | framerate                        :        12.83 fps  from  176   samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.49 ms   from  190   samples |
 | total time                       :        77.88 ms   from  189   samples |
 | framerate                        :        12.84 fps  from  189   samples |
 +--------------------------------------------------------------------------+

 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.46 ms   from  203   samples |
 | total time                       :        77.84 ms   from  202   samples |
 | framerate                        :        12.85 fps  from  202   samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.42 ms   from  216   samples |
 | total time                       :        77.83 ms   from  215   samples |
 | framerate                        :        12.85 fps  from  215   samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.44 ms   from  229   samples |
 | total time                       :        77.84 ms   from  228   samples |
 | framerate                        :        12.85 fps  from  228   samples |
 +--------------------------------------------------------------------------+


 +--------------------------------------------------------------------------+
 | Object Detection                                                         |
 +--------------------------------------------------------------------------+
 +--------------------------------------------------------------------------+
 | Input Src: /dev/video-usb-cam0                                           |
 | Model Name: yolov8_s_syncbn_fast_8xb16-100e_coco                         |
 | Model Type: detection                                                    |
 +--------------------------------------------------------------------------+
 | dl-inference                     :        59.44 ms   from  242   samples |
 | total time                       :        77.83 ms   from  241   samples |
 | framerate                        :        12.85 fps  from  241   samples |
 +--------------------------------------------------------------------------+


APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
   160.302907 s: IPC: Deinit ... !!!
   160.303925 s: IPC: DeInit ... Done !!!
   160.303985 s: MEM: Deinit ... !!!
   160.304003 s: DDR_SHARED_MEM: Alloc's: 66 alloc's of 147662304 bytes
   160.304047 s: DDR_SHARED_MEM: Free's : 66 free's  of 147662304 bytes
   160.304059 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes
   160.304075 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python#

Hello Machida-san,

1)

Ryuuichi machida said:

But I got kinds of "device" error. (It seems that "tidl-tools" which is located on "edegeai-tidl-tools" is NOT located under each "device name".)
>export TIDL_TOOLS_PATH=$(pwd)/tools/${TARGET_SOC}/tidl_tools 

Yes, edgeai-tidl-tools will create just one tidl_tools at edgeai-tidl-tools/tidl_tools. The TIDL_TOOLS_PATH variable needs to point here. The script you reference assumes a different directory structure. It is sufficient to `export` the variable to this path yourself -- no magic beyond the export command :)

I do not see a compilation log here -- the log looks like it is edgeai-gst-apps.Is this intentional?

If your network is initializing and running accelerated, then the tools you used should be fine. I believe this is the case, given that the network is nominally running in ~60 ms. This is close to my inference latency on the target (10.1 SDK)

2)

Ryuuichi machida said:

I would like to see something reference for model parameter of "yolov8", but it seems there is no reference...
Do you have any configuration example for "yolov8" ?

Yes, let me share this. We have limited online resources/examples for yolov8 due to copy-left licensing restrictions on the original MMYOLO version -- this prevents us from sharing a model file itself.

The input preprocessing (including scale and mean) will depend on the model training, where ours is based on the upstream mmyolo.

The configuration I used in edgeai-tidl-tools, SDK 10.1 (release 10_01_00_02) is the following. Note that when I use this config,

 'yolov8_s' : create_model_config(
        source=AttrDict(
            model_url='dummy',
            infer_shape=True,
        ),
        preprocess=AttrDict(
            resize=[640,640],
            crop=[640,640],
            data_layout='NCHW',
            resize_with_pad=False,
            reverse_channels=False,
        ),
        session=AttrDict(
            session_name='onnxrt' ,
            model_path=os.path.join(
                models_base_path, "yolov8_small_lite_640x640_20231117_model.onnx"
            ),
            meta_layers_names_list=os.path.join(
                models_base_path, "yolov8_small_lite_640x640_20231117_model.prototxt"
            ),            input_mean=[0,0,0],
            input_scale=[0.003921568627,0.003921568627,0.003921568627],
            input_optimization=True,
            meta_arch_type=8,
            od_type='yolo_v8',
        ),
        postprocess=AttrDict(
            formatter="DetectionBoxSL2BoxLS",
            resize_with_pad=True,
            keypoint=False,
            object6dpose=False,
            normalized_detections=False,
            shuffle_indices=None,
            squeeze_axis=None,
            reshape_list=[(-1, 5), (-1, 1)],
            ignore_index=None,
        ),
        task_type="detection",
        extra_info=AttrDict(
            od_type="YoloV5",
            framework="",
            num_images=numImages,
            num_classes=91,
            label_offset_type="80to90",
            label_offset=1,
        ),
        optional_options = {
         #'debug_level': 2,
        }
    ),

I get good detections on the TI model, but when I apply the same to yours, there are no detections. For example, I run your model with CPUExecutionProvider  by passing -d to edgeai-tidl-tools//examples/osrt_python/ort/onnxrt_ep.py. The result is no detections on the original image (on a stock image, so maybe wrong for your model)

[array([[[ 0.,  0.,  0.,  0.,  0., -1.]]], dtype=float32)]
[array([[[ 0.,  0.,  0.,  0.,  0., -1.]]], dtype=float32)]

And I get very different, but probably not correct values the same way when running with input_scale=1 and mean=0. This is independent of TIDL I cannot tell if the problem here is with the input preprocessing or the model training itself.

This is a long response; let me step back and summarize status:

• By upgrading to 10.x SDK, we resolved issue (seg fault) with model failing to compile.
  • Compilation failed before because 9.x tools have an error mode with newer linux kernels (failing on 6.8.0, working on 6.5.5)
• Now, the model runs on target
• Model results are not correct. I replicate this with your model.
  • On TI internal model with same compilation settings, detections are correct
  • When running on CPU, detections for your model are not correct even. This indicates something wrong either with training or the preprocessing

To resolve, you need to get this model working well on CPU first. Please use -d tag to disable TIDL from running model, and to use CPUExecutionProvider instead

BR,
Reese

Hi Reese-san,

For 1 :

Yes, edgeai-tidl-tools will create just one tidl_tools at edgeai-tidl-tools/tidl_tools. The TIDL_TOOLS_PATH variable needs to point here. The script you reference assumes a different directory structure. It is sufficient to `export` the variable to this path yourself -- no magic beyond the export command :)
=> Yes, Previously, I tried to compile after changing "TIDL_TOOLS_PATH" on included "run_set_env.sh". But I got error (I have not saved it so I could not see at this time..)

I do not see a compilation log here -- the log looks like it is edgeai-gst-apps.Is this intentional?
=> Yes.

Best Regards,

Hello Reese-san,

For 2 :

• By upgrading to 10.x SDK, we resolved issue (seg fault) with model failing to compile.
  • Compilation failed before because 9.x tools have an error mode with newer linux kernels (failing on 6.8.0, working on 6.5.5)
• Now, the model runs on target
• Model results are not correct. I replicate this with your model.
  • On TI internal model with same compilation settings, detections are correct
  • When running on CPU, detections for your model are not correct even. This indicates something wrong either with training or the preprocessing

To resolve, you need to get this model working well on CPU first. Please use -d tag to disable TIDL from running model, and to use CPUExecutionProvider instead

=>
I performed several change for model configuration.
Then I noticed following thing.

2-1. As you said, I performed following command to check bounding box can be observed correctly.
Case 1 : Use same setting(setting file is "E2E_ORG--20250204.txt) which I posted 1/31(File name : E2E--20250131.txt)
The result when I performed "python3 onnxrt_ep.py -d" is below.

There is no bounding box. (It seems that this is same condition as onboard(EVM) result.)

Case 2 : Change "od_type" and "framework" which is defined in "extra_info".(Setting file is "E2E_CH_OD_type--20250204.txt)
Before :
od_type="yolo_v8"
framework="MMDetection"
After :
od_type="YoloV5"
framework=""

The result when I performed "python3 onnxrt_ep.py -d" is below.


I could see bounding box.
So, I deploy this "artifacts" to SD card and performed application on EVM.
However, though I did not see any error, however I could not see bounding box on EVM.
And "bbox" variable is same as previous(2025/1/31) one..

For 2-2
At first, I initated your setting which you posted.
However, when I applied following setting, I could see error.

* In "session" variable
- input_optimization=True
- od_type='yolo_v8'

For 2-3
I suspect that "input mean" and "input scale" may cause this issue.
I'm re-checking model configuration which is posted on below URL.
https://github.com/open-mmlab/mmyolo/blob/main/configs/yolov8/yolov8_s_syncbn_fast_8xb16-500e_coco.py

According to this, it seems that "mean" is set as "0" (This is same as my and your setting.) However, I'm not sure "scale" setting.
Could you tell me how you decide this value as "0.003921568627" ?

Best Regards

"yolov8_s_syncbn_fast_8xb16-100e_coco_20250203_ORG": create_model_config(
    source=AttrDict(
    ),
    preprocess=AttrDict(
        resize=640,
        crop=640,
        data_layout="NCHW",
        resize_with_pad=True,
        pad_color=[114, 114, 114],
    ),
    session=AttrDict(
        session_name="onnxrt",
        model_path=os.path.join(
            models_base_path, "optimized_best_coco_bbox_mAP_epoch_93.onnx"
        ),
        meta_layers_names_list=os.path.join(
            models_base_path, "best_coco_bbox_mAP_epoch_93.prototxt"
        ),
        meta_arch_type=8,
        input_mean=[0, 0, 0],
        input_scale=[0.003921568627, 0.003921568627, 0.003921568627],
        input_optimization=False,
    ),
    postprocess=AttrDict(
        formatter="DetectionBoxSL2BoxLS",
        resize_with_pad=True,
        keypoint=False,
        object6dpose=False,
        normalized_detections=False,
        shuffle_indices=None,
        squeeze_axis=None,
        reshape_list=[(-1, 5), (-1, 1)],
        ignore_index=None,
    ),
    task_type="detection",
    extra_info=AttrDict(
        od_type="yolo_v8",
        framework="MMDetection",
        num_images=numImages,
        num_classes=1,
        label_offset_type="80to90",
        label_offset=1,
    ),
),

"yolov8_s_syncbn_fast_8xb16-100e_coco_20250204_CH_OD_type_1": create_model_config(
    source=AttrDict(
    ),
    preprocess=AttrDict(
        resize=640,
        crop=640,
        data_layout="NCHW",
        resize_with_pad=True,
        pad_color=[114, 114, 114],
    ),
    session=AttrDict(
        session_name="onnxrt",
        model_path=os.path.join(
            models_base_path, "optimized_best_coco_bbox_mAP_epoch_93.onnx"
        ),
        meta_layers_names_list=os.path.join(
            models_base_path, "best_coco_bbox_mAP_epoch_93.prototxt"
        ),
        meta_arch_type=8,
        input_mean=[0, 0, 0],
        input_scale=[0.003921568627, 0.003921568627, 0.003921568627],
        input_optimization=False,
    ),
    postprocess=AttrDict(
        formatter="DetectionBoxSL2BoxLS",
        resize_with_pad=True,
        keypoint=False,
        object6dpose=False,
        normalized_detections=False,
        shuffle_indices=None,
        squeeze_axis=None,
        reshape_list=[(-1, 5), (-1, 1)],
        ignore_index=None,
    ),
    task_type="detection",
    extra_info=AttrDict(
        od_type="YoloV5",
        framework="",
        num_images=numImages,
        num_classes=1,
        label_offset_type="80to90",
        label_offset=1,
    ),
),

Hello Machida-san,

Let me start here:

Ryuuichi machida said:

Could you tell me how you decide this value as "0.003921568627" ?

Yes, this is 1/255. Therefore, with a mean of 0 and this scale, it is normalizing the image (uint8) to have all values on interval [0,1).

See below from the link you suggested, the mean is 0 and scale is 255. I believe that convention for preprocessing in these training frameworks is that 

processed_input = (input - mean) / std

• In our script, we are multiplying the scale value instead of dividing by std. It is the same concept, though.

https://github.com/open-mmlab/mmyolo/blob/8c4d9dc503dc8e327bec8147e8dc97124052f693/configs/yolov8/yolov8_s_syncbn_fast_8xb16-500e_coco.py#L100 

• They are also using mean 0 and std=255. Therefore, equivalent scale would be 1/255=0.0039215...

----------------------------------------------

Thanks for the test image, this was actually very helpful. With that, mean=0 and scale=1/255, I'm able to get a decent bounding box with -d (CPU) and with C7xMMA emulation (no arg). The detection box is slightly different, but nominally covers the same region you show in the image. My config was very similar to yours, and resulted in good emulation results.

'yolov8_s_machida' : create_model_config(
        source=AttrDict(
            model_url='dummy',  
            infer_shape=True,
        ),
        preprocess=AttrDict(
            resize=[640,640],
            crop=[640,640],
            data_layout='NCHW',
            resize_with_pad=False,
            reverse_channels=False,
        ),
        session=AttrDict(
            session_name='onnxrt' ,
            model_path=os.path.join(
                models_base_path, "best_coco_bbox_mAP_epoch_93.onnx"
            ),
            meta_layers_names_list=os.path.join(
                models_base_path, "best_coco_bbox_mAP_epoch_93.prototxt"
            ),            
            input_mean=[0,0,0],
            input_scale=[1/255,1/255,1/255],
            input_optimization=False,
            meta_arch_type=8,
        ),
        postprocess=AttrDict(
            formatter="DetectionBoxSL2BoxLS",
            resize_with_pad=True,
            keypoint=False,
            object6dpose=False,
            normalized_detections=False,
            shuffle_indices=None,
            squeeze_axis=None,
            reshape_list=[(-1, 5), (-1, 1)],
            ignore_index=None,
        ),
        task_type="detection",
        extra_info=AttrDict(
            od_type='YoloV5', #od_type="YoloV5" or yolo_v8
            framework="", #or ''
            num_images=numImages,
            num_classes=1,
            label_offset_type="80to90",
            label_offset=1,
        ),
    }

From here, I can also validate that accuracy on target is matching emulation. I copied edgeai-tidl-tools to the target and ran the same scripts (runtime near 64ms, of which ~50ms is actual C7xMMA offload time). The output was identical to x86 PC emulation. Using these same scripts on target ensures that the preprocessing is handled the same way.

For your testing, can you try running this on fixed images? May be worthwhile to print the actual outputs as well so you can investigate the values themselves. In edgeai-gst-apps (and other SW tools), there is typically a confidence threshold. For example, viz_threshold in configs/object_detection.yaml. Try setting this to a low value like 0.1. For your test image, I see a confidence of 0.31, which would have been lower than the typical default of 0.6

BR,
Reese

Hello Reese-san,

Thank you for your confirmation.

For your testing, can you try running this on fixed images? May be worthwhile to print the actual outputs as well so you can investigate the values themselves. In edgeai-gst-apps (and other SW tools), there is typically a confidence threshold. For example, viz_threshold in configs/object_detection.yaml. Try setting this to a low value like 0.1. For your test image, I see a confidence of 0.31, which would have been lower than the typical default of 0.6
=> I copied image to SD card and change "yaml" file to refer static this image.
And also change viz_threshold to "0.1". However, I could not confirm bounding box on EVM.

Is described confidence of "0.31" shown on EVM ?
Here is result under above condition (Use static image and change viz_threshold "0.1")
----
root@am62axx-evm:/opt/edgeai-gst-apps# ./app_edgeai.py ../configs/yolov8.yaml
-sh: ./app_edgeai.py: No such file or directory
root@am62axx-evm:/opt/edgeai-gst-apps# cd apps_python/
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python#
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python#
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/yolov8.yaml
libtidl_onnxrt_EP loaded 0x1fe38c0
Final number of subgraphs created are : 1, - Offloaded Nodes - 277, Total Nodes - 277
APP: Init ... !!!
350.164317 s: MEM: Init ... !!!
350.164403 s: MEM: Initialized DMA HEAP (fd=5) !!!
350.164597 s: MEM: Init ... Done !!!
350.164629 s: IPC: Init ... !!!
350.181850 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
350.186493 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
350.186657 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
350.186674 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
350.186687 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
350.187834 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-0
350.188168 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-1
350.188503 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-2
350.188756 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-3
350.188793 s: VX_ZONE_INFO: [tivxInitLocal:126] Initialization Done !!!
350.188836 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
==========[INPUT PIPELINE(S)]==========

[PIPE-0]

multifilesrc location=/opt/edgeai-test-data/images/cat1.jpg loop=True ! jpegdec ! videoscale qos=True ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)640, height=(int)640;" ! tiovxdlpreproc out-pool-size=4 scale-0=0.00392200006172061 scale-1=0.00392200006172061 scale-2=0.00392200006172061 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0

==========[OUTPUT PIPELINE]==========

appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0 src::pool-size=4
sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>"
! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! queue ! tiperfoverlay title=Object Detection ! kmssink sync=False max-lateness=5000000 qos=True processing-deadline=15000000 driver-name=tidss connector-id=40 plane-id=31 force-modesetting=True fd=35

viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
----

Best Regards, 

Hello Reese-san,

Is it possible to share files(dataset.yaml, config.yaml, param.yaml, and run.log) which you generated on "edgeai-tensorlab" ?

BR,

Hello Reese-san,

As trial, I performed below.

My only note here will be that you need to compile with your TIDL_TOOLS_PATH pointed to 10.1 SDK tools. Use the tools from edgeai-tidl-tools ((edgeai-tidl-tools/tidl_tools/) here, because edgeai-tensorlab does not yet have a 10.1 release. 

When setting up these repos, make sure that TIDL_TOOLS_PATH variable is not set in the Linux command line.
=> At first, I just pointed "TIDL_TOOLS_PATH" which is included in "edgeai-tidl-tools" on  "run_set_env.sh".
Then, I got following error. (This error is previous one which I said in this thread.)

---
(benchmark) machida@machida-VirtualBox:~/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark$ ./run_custom_pc.sh AM62A
TARGET_SOC: AM62A
TARGET_MACHINE: pc
DEBUG MODE: false @ machida-VirtualBox:5678
TIDL_TOOLS_PATH=/home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tidl-tools/tidl_tools
LD_LIBRARY_PATH=/home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tidl-tools/tidl_tools
PYTHONPATH=:
===================================================================
work_dir = ./work_dirs/modelartifacts/AM62A/8bits
packaged_dir = ./work_dirs/modelpackage/AM62A/8bits
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Traceback (most recent call last):
File "/home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/./scripts/benchmark_custom.py", line 216, in <module>
interfaces.run_accuracy(settings, work_dir, pipeline_configs)
File "/home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/edgeai_benchmark/interfaces/run_accuracy.py", line 43, in run_accuracy
assert settings.target_device in os.environ['TIDL_TOOLS_PATH'], \
AssertionError: found target_device in settings: AM62A but it does not seem to match tidl_tools path: /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tidl-tools/tidl_tools
---

So, I copied "tidl-tools" which is included in "edgeai-tidl-tools" to under "edgeai-benchmark/tools/AM62A/FM_edgeai_tensorlab"
("FM_edgeai_tensorlab" is created by myself.)
Then, I could compile correctly (Log is "E2E_Tensorlab--20250206.txt)

I deployed these(config,dataset,param,result.yaml and run.log) to SD card.
However, result under static image was same as previous one...

----
root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/yolov8.yaml
[ 43.451666] usb 1-1: reset high-speed USB device number 2 using xhci-hcd
libtidl_onnxrt_EP loaded 0xc30df20
Final number of subgraphs created are : 1, - Offloaded Nodes - 277, Total Nodes - 277
APP: Init ... !!!
54.848424 s: MEM: Init ... !!!
54.848501 s: MEM: Initialized DMA HEAP (fd=5) !!!
54.848722 s: MEM: Init ... Done !!!
54.848756 s: IPC: Init ... !!!
54.865651 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
54.874389 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
54.878415 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
54.878464 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
54.878474 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
54.880809 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-0
54.880996 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-1
54.881113 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-2
54.881241 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-3
54.881256 s: VX_ZONE_INFO: [tivxInitLocal:126] Initialization Done !!!
54.881278 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
==========[INPUT PIPELINE(S)]==========

[PIPE-0]

multifilesrc location=/opt/edgeai-test-data/images/cat1.jpg loop=True ! jpegdec ! videoscale qos=True ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)640, height=(int)640;" ! tiovxdlpreproc out-pool-size=4 scale-0=0.00392200006172061 scale-1=0.00392200006172061 scale-2=0.00392200006172061 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0

==========[OUTPUT PIPELINE]==========

appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0 src::pool-size=4
sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>"
! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! queue ! tiperfoverlay title=Object Detection ! kmssink sync=False max-lateness=5000000 qos=True processing-deadline=15000000 driver-name=tidss connector-id=40 plane-id=31 force-modesetting=True fd=35

viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
viz_threshold is [-0.0015625 -0.0015625 -0.0015625 -0.0015625 -1. 0. ]
---

I would like to compare your validation environment...

Best Regards,

(benchmark) machida@machida-VirtualBox:~/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark$ ./run_custom_pc.sh AM62A
TARGET_SOC:     AM62A
TARGET_MACHINE: pc
DEBUG MODE:     false @ machida-VirtualBox:5678
TIDL_TOOLS_PATH=/home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/tools/AM62A/FM_edgeai_tensorlab/tidl_tools
LD_LIBRARY_PATH=/home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/tools/AM62A/FM_edgeai_tensorlab/tidl_tools
PYTHONPATH=:
===================================================================
work_dir = ./work_dirs/modelartifacts/AM62A/8bits
packaged_dir = ./work_dirs/modelpackage/AM62A/8bits
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
configs to run: ['imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx']
number of configs: 1

INFO:20250206-094727: parallel_run - parallel_processes:1 parallel_devices=[0]
TASKS                                                       |          |     0% 0/1| [< ]
INFO:20250206-094727: starting process on parallel_device - 0   0%|          || 0/1 [00:00<?, ?it/s]

INFO:20250206-094727: starting - imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx
INFO:20250206-094727: model_path - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-modelzoo/models/vision/detection/DGX/yolov8_s_syncbn_fast_8xb16-100e_coco/optimized_best_coco_bbox_mAP_epoch_93.onnx
INFO:20250206-094727: model_file - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/work_dirs/modelartifacts/AM62A/8bits/imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx/model/optimized_best_coco_bbox_mAP_epoch_93.onnx
INFO:20250206-094727: quant_file - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/work_dirs/modelartifacts/AM62A/8bits/imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx/model/optimized_best_coco_bbox_mAP_epoch_93_qparams.prototxt
Downloading 1/1: /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-modelzoo/models/vision/detection/DGX/yolov8_s_syncbn_fast_8xb16-100e_coco/optimized_best_coco_bbox_mAP_epoch_93.onnx
Download done for /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-modelzoo/models/vision/detection/DGX/yolov8_s_syncbn_fast_8xb16-100e_coco/optimized_best_coco_bbox_mAP_epoch_93.onnx
Downloading 1/1: /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-modelzoo/models/vision/detection/DGX/yolov8_s_syncbn_fast_8xb16-100e_coco/optimized_best_coco_bbox_mAP_epoch_93.onnx
Download done for /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-modelzoo/models/vision/detection/DGX/yolov8_s_syncbn_fast_8xb16-100e_coco/optimized_best_coco_bbox_mAP_epoch_93.onnx

INFO:20250206-094728: running - imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx
INFO:20250206-094728: pipeline_config - {'task_type': 'detection', 'calibration_dataset': <edgeai_benchmark.datasets.coco_det.COCODetection object at 0x7c19baee31f0>, 'input_dataset': <edgeai_benchmark.datasets.coco_det.COCODetection object at 0x7c19cf620250>, 'preprocess': <edgeai_benchmark.preprocess.PreProcessTransforms object at 0x7c19ba8dd7e0>, 'session': <edgeai_benchmark.sessions.onnxrt_session.ONNXRTSession object at 0x7c19ba8dd8a0>, 'postprocess': <edgeai_benchmark.postprocess.PostProcessTransforms object at 0x7c19ba8de710>, 'metric': {'label_offset_pred': {0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9, 9: 10, 10: 11, 11: 13, 12: 14, 13: 15, 14: 16, 15: 17, 16: 18, 17: 19, 18: 20, 19: 21, 20: 22, 21: 23, 22: 24, 23: 25, 24: 27, 25: 28, 26: 31, 27: 32, 28: 33, 29: 34, 30: 35, 31: 36, 32: 37, 33: 38, 34: 39, 35: 40, 36: 41, 37: 42, 38: 43, 39: 44, 40: 46, 41: 47, 42: 48, 43: 49, 44: 50, 45: 51, 46: 52, 47: 53, 48: 54, 49: 55, 50: 56, 51: 57, 52: 58, 53: 59, 54: 60, 55: 61, 56: 62, 57: 63, 58: 64, 59: 65, 60: 67, 61: 70, 62: 72, 63: 73, 64: 74, 65: 75, 66: 76, 67: 77, 68: 78, 69: 79, 70: 80, 71: 81, 72: 82, 73: 84, 74: 85, 75: 86, 76: 87, 77: 88, 78: 89, 79: 90, 80: 91}}, 'model_info': {'metric_reference': {'accuracy_ap[.5:.95]%': 42.4}, 'model_shortlist': 70}}
INFO:20250206-094729: import  - imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx - this may take some time.../home/machida/.pyenv/versions/benchmark/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:115: UserWarning: Specified provider 'TIDLCompilationProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
*************** EP Error ***************
EP Error Unknown Provider Type: TIDLCompilationProvider when using ['TIDLCompilationProvider', 'CPUExecutionProvider']
Falling back to ['CPUExecutionProvider'] and retrying.
****************************************
================================ import model =============

INFO:20250206-094731: import completed  - imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx - 2 sec
INFO:20250206-094731: infer  - imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx - this may take some time.../home/machida/.pyenv/versions/benchmark/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:115: UserWarning: Specified provider 'TIDLExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
*************** EP Error ***************
EP Error Unknown Provider Type: TIDLExecutionProvider when using ['TIDLExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CPUExecutionProvider'] and retrying.
****************************************
infer : imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt|  64%|######3   || 74/116 [00:53<00:29,  1.41it/s]
INFO:20250206-094827: parallel_run - num_total_tasks:1 len(queued_tasks):0 len(process_dict):1 len(result_list):0
infer : imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt| 100%|##########|| 116/116 [01:22<00:00,  1.40it/s]

INFO:20250206-094853: infer completed  - imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx - 83 secLoading and preparing results...
DONE (t=0.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.06s).
Accumulating evaluation results...
DONE (t=0.01s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.020
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.080
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.020
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.027
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.030
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.030
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.030


SUCCESS:20250206-094854: benchmark results - {'infer_path': 'imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx', 'accuracy_ap[.5:.95]%': 1.986918, 'accuracy_ap50%': 7.986799, 'num_subgraphs': 0, 'infer_time_core_ms': 0.0, 'infer_time_subgraph_ms': 0.0, 'ddr_transfer_mb': 0.0, 'perfsim_time_ms': 0.0, 'perfsim_ddr_transfer_mb': 0.0, 'perfsim_gmacs': 0.0}
TASKS                                                       | 100%|██████████|| 1/1 [01:26<00:00, 86.53s/it]
TASKS                                                       | 100%|██████████|| 1/1 [01:26<00:00, 86.42s/it]

packaging artifacts to ./work_dirs/modelpackage/AM62A/8bits please wait...
SUCCESS:20250206-094856: finished packaging - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/work_dirs/modelartifacts/AM62A/8bits/imagedet-custom_cat_3_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx
SUCCESS:20250206-094858: finished packaging - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/work_dirs/modelartifacts/AM62A/8bits/imagedet-custom_cat_CH_param_1_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx
SUCCESS:20250206-094900: finished packaging - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/work_dirs/modelartifacts/AM62A/8bits/imagedet-custom_cat_CH_param_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx
SUCCESS:20250206-094902: finished packaging - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/work_dirs/modelartifacts/AM62A/8bits/imagedet-custom_cat_ORG_20250203_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx
SUCCESS:20250206-094904: finished packaging - /home/machida/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark/work_dirs/modelartifacts/AM62A/8bits/imagedet-custom_cat_ORG_20250206_CH_TIDLTOOLS_onnxrt_DGX_yolov8_s_syncbn_fast_8xb16-100e_coco_optimized_best_coco_bbox_mAP_epoch_93_onnx
-------------------------------------------------------------------
settings: {'include_files': None, 'pipeline_type': 'accuracy', 'num_frames': 5000, 'calibration_frames': 5, 'calibration_iterations': 5, 'configs_path': './configs', 'models_path': '../edgeai-modelzoo/models', 'modelartifacts_path': './work_dirs/modelartifacts/', 'modelpackage_path': './work_dirs/modelpackage/', 'datasets_path': './dependencies/datasets', 'target_device': None, 'target_machine': 'pc', 'run_suffix': None, 'parallel_devices': None, 'parallel_processes': 1, 'tensor_bits': 8, 'runtime_options': None, 'run_import': True, 'run_inference': True, 'run_missing': True, 'detection_threshold': 0.3, 'detection_top_k': 200, 'detection_nms_threshold': None, 'detection_keep_top_k': None, 'save_output': False, 'num_output_frames': 50, 'model_selection': None, 'model_shortlist': None, 'model_exclusion': None, 'task_selection': None, 'runtime_selection': None, 'session_type_dict': {'onnx': 'onnxrt'}, 'dataset_type_dict': {'imagenet': 'imagenetv2c'}, 'dataset_selection': None, 'dataset_loading': True, 'config_range': None, 'enable_logging': True, 'verbose': False, 'capture_log': False, 'additional_models': False, 'experimental_models': False, 'rewrite_results': False, 'with_udp': True, 'flip_test': False, 'model_transformation_dict': None, 'report_perfsim': False, 'tidl_offload': True, 'input_optimization': None, 'run_dir_tree_depth': None, 'target_device_preset': True, 'fast_calibration_factor': None, 'param_template_file': None, 'skip_pattern': '_package', 'settings_file': 'settings_import_on_pc.yaml', 'basic_keys': ['include_files', 'pipeline_type', 'num_frames', 'calibration_frames', 'calibration_iterations', 'configs_path', 'models_path', 'modelartifacts_path', 'modelpackage_path', 'datasets_path', 'target_device', 'target_machine', 'run_suffix', 'parallel_devices', 'parallel_processes', 'tensor_bits', 'runtime_options', 'run_import', 'run_inference', 'run_missing', 'detection_threshold', 'detection_top_k', 'detection_nms_threshold', 'detection_keep_top_k', 'save_output', 'num_output_frames', 'model_selection', 'model_shortlist', 'model_exclusion', 'task_selection', 'runtime_selection', 'session_type_dict', 'dataset_type_dict', 'dataset_selection', 'dataset_loading', 'config_range', 'enable_logging', 'verbose', 'capture_log', 'additional_models', 'experimental_models', 'rewrite_results', 'with_udp', 'flip_test', 'model_transformation_dict', 'report_perfsim', 'tidl_offload', 'input_optimization', 'run_dir_tree_depth', 'target_device_preset', 'fast_calibration_factor', 'param_template_file', 'skip_pattern', 'settings_file'], 'dataset_cache': None}
results found for 5 models
Report generated at ./work_dirs/modelartifacts/
-------------------------------------------------------------------
===================================================================
(benchmark) machida@machida-VirtualBox:~/ti-processor-sdk-linux-edgeai-am62a-evm-10_01_00_05/edgeai-tensorlab/edgeai-benchmark$

Hello Machida-san,

I am also having a hard time running both your artifacts and mine with edgeai-gst-apps -- it is not just you. For both your model and mine, I receive an array of only [-1,-1,-1,-1,0,-1] (x200 times), even for static test images. It's not the same output as yours, but it is similar.

• As a check, please run with debug_level=2 for the onnxruntime and run /opt/vx_app_arm_remote_log.out in the background. Are there any warnings or errors?

I am not sure why this is, because I can run the image with edgeai-tidl-tools on the target

Ryuuichi machida said:

Is described confidence of "0.31" shown on EVM ?

Yes, exactly. I can run on EVM by copying edgeai-tidl-tools to the EVM, and running from there. I receive expected output results that are identical to PC-based emulation of C7x.

When I try to run the same model with edgeai-gst-apps, I receive only empty detections, which are [-1,-1,-1,-1,0,-1]. The same is also true with another testing application I've written -- I see only empty detections. This is consistent between your model and the TI one

I think this is still related to input image preprocessing. Here's why:

• I ran a quick and ugly test, where I used the 'pickle' library to save the image that's used with edgeai-tidl-tools. This makes sure the data is 100% consistent.
• I then modified a few other testing scripts that, when using an input image, gave empty detections.
• When I replaced the input in those scripts to use the un-pickled file, I get the exact same results as edgeai-tidl-tools. The same is true for edgeai-gst-apps

This means there is not something wrong with the model itself or how the model is invoked. Rather, the issue is part of image preprocessing. I note that I used 10.1 SDK for these tests.

Said another way, I don't think your model nor TIDL are the issue. Preprocessing for the image is more likely the cause of this. I am still using mean=0 and scale=1/255 here

Without spending more time on this, I'm not sure the issue, but I'm confident is with image preparation/preprocessing. I have limited available time right now due to an upcoming work trip, so please expect delays in my response for the next week or two.

BR,
Reese

Hello Reese-san,

Thank you for your reply.

I am fairly confident that it is the confidence_threshold in your prototxt. I have noted that no output for your model running with TIDL has had confidence > 0.3. This is consistent with the prototxt. I would suggest reducing the threshold for both NMS and confidence -- for confidence, try 0.1.
=> My prototxt setting for NMS and confidence was below.

---
detection_output_param {
num_classes: 1
share_location: true
background_label_id: -1
nms_param {
nms_threshold: 0.45
top_k: 200
}
code_type: CODE_TYPE_DIST2BBOX
keep_top_k: 200
confidence_threshold: 0.3
}
---

It seems that setting is same as "det_box_overlay" of "common_utils.py".

I noticed one thing. When I used image which is used on inference of "edgeai-tidl-tools", I got bbox. However, though I used several images, but I got bbox only following image. So, as you said, it seems that model works correctly. 

 

This doesn't entirely explain why your model is not giving the same output through edgeai-gst-apps. My current thought is that your model is just barely sensitive enough that a slight change to preprocessing has caused the confidence value to drop below 0.3. If your model was trained on a small dataset and/or for few epochs, this could explain the sensitivity to small variations in the input.
=> Yes, I used only approx 150 images as dataset, so, I understand this is small. I could see bbox for only one image, but I could see bbox, so this issue may be caused by accuracy...

Best Regards,

Hi Reese-san,

Thank you for your support.
I really appreciate your dedicated response. When I have another issue, I will open new thread.

Best Regards,