SK-AM62A-LP: TIDL Inference on AM62A

Sajan Saji

Part Number: SK-AM62A-LP

Tool/software:

Hello expert,

Previous days, I am working to find the below text overlay while running a pipeline.

Here is the pipeline I used.

gst-launch-1.0 -e \
v4l2src device=/dev/video-imx219-cam1 io-mode=5 ! \
queue leaky=2 ! video/x-bayer, width=3280, height=2464, framerate=15/1, format=rggb10 ! \
tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/7140.dcc_viss.bin format-msb=9 \
sink_0::dcc-2a-file=/opt/imaging/imx219/linear/7140.dcc_2a.bin sink_0::device=/dev/v4l-imx219-subdev1 ! \
video/x-raw, format=NV12 ! \
tiovxmultiscaler src_1::pool-size=4 name=split_01 src_0::roi-startx=205 src_0::roi-starty=154 src_0::roi-width=2870 src_0::roi-height=2156 target=0 \
split_01. ! queue ! video/x-raw, format=NV12, width=718, height=540, framerate=15/1 ! tiovxmultiscaler target=1 ! \
video/x-raw, width=224, height=224 ! tiovxdlpreproc model=/opt/model_zoo/mobileNetV2  out-pool-size=4 ! \
application/x-tensor-tiovx ! tidlinferer target=1  model=/opt/model_zoo/mobileNetV2 ! post_0.tensor \
split_01. ! queue ! video/x-raw, format=NV12, width=3280, height=2464, framerate=15/1 ! post_0.sink \
tidlpostproc name=post_0 model=/opt/model_zoo/mobileNetV2 top-N=1 display-model=false ! \
kmssink

1. Tried to locate it, in gsttidlpostproc.cpp and gsttidlinferer.cpp present in /opt/edgeai-gst-plugins/ext/ti . Unfortunately nothing about drawing this text overlay and contrast color box.
In apps_python it is present in post_process.py file. I am trying to create a python app for my use case. Where can I find this text overlay?

2. Where can I find the logic of recognizing class_name. post_process.py, #L156 to #L167 . I mean I need to do some tuning in the class_name that is detected by the model.

Please help me to find this 2 sections in code.

Warm Regards,
Sajan

1 month ago

0 Reese Grimsley 1 month ago

TI__Genius 15156 points

Hi Sajan,

Some of the code you're looking for is probably within /opt/edgeai-dl-inferer [0], which is a dependency of that gsttidlpostproc.cpp

I think you'll find the source code that is relevant for both your questions in edgeai-dl-inferer.

In case your curious about the dependencies here, please see the edgeai-app-stack repo [1], which has a good graphic to explain this

[0] https://git.ti.com/cgit/edgeai/edgeai-dl-inferer/tree/post_process/src/ti_post_process_image_classification.cpp

[1] https://github.com/TexasInstruments/edgeai-app-stack

BR,
Reese

0 Sajan Saji 1 month ago in reply to Reese Grimsley

Intellectual 490 points

Hello Reese,

Can you please answer me. How can I do these modification in apps_python very simply. Can I do it in the program? Used the optiflow pipeline to create the app.

Can I increase the resolution to inferer:
split_01. ! queue ! video/x-raw, format=NV12, width=718, height=540, framerate=15/1 ! tiovxmultiscaler target=1 ! \
video/x-raw, width=224, height=224 ! tiovxdlpreproc model=/opt/model_zoo/mobileNetV2 out-pool-size=4 !
Did these affects the accuracy of model

Also I can't find the text "Recognized classes (Top %d): in the above links
https://github.com/TexasInstruments/edgeai-app-stack/blob/main/edgeai-dl-inferer 404 not found

Warm Regards,
Sajan

0 Reese Grimsley 1 month ago in reply to Sajan Saji

TI__Genius 15156 points

Hi Sajan,

You can modify apps_python files directly to change this post-processing code -- your original message indicates you have already found this code for the python-based postprocessing.

Optiflow is only using gstreamer plugins, so using this framework for you application entails editing source code consumed by that plugin, recompiling, and reinstalling. There are scripts under edgeai-gst-apps/scripts to build/install each of the components within the edgeai-app-stack.

Changing the postprocessing for an optiflow-based application means you must alter the GST plugins.

Sajan Saji said:
Can I increase the resolution to inferer:
split_01. ! queue ! video/x-raw, format=NV12, width=718, height=540, framerate=15/1 ! tiovxmultiscaler target=1 ! \
video/x-raw, width=224, height=224 ! tiovxdlpreproc model=/opt/model_zoo/mobileNetV2 out-pool-size=4 !
Did these affects the accuracy of model

No, the model dimensions are static. You would need to change the model resolution (independent of TI tooling), and then recompile the model. This is usually straight-forward for classification models since the structure is often resolution-agnostic. ONNX models are easiest for this, whereas TFLITE are generally harder to modify.

Sajan Saji said:
Also I can't find the text "Recognized classes (Top %d): in the above links
https://github.com/TexasInstruments/edgeai-app-stack/blob/main/edgeai-dl-inferer 404 not found

edgeai-dl-inferer is a separate repository that GITHUB cannot search. It is under our own GIT infrastructure.

You can also try searching in your SDK under /opt/edgeai-dl-inferer. In SDK 10.1, I find this at line 184 in ti_post_process_image_classification.cpp
- https://git.ti.com/cgit/edgeai/edgeai-dl-inferer/tree/post_process/src/ti_post_process_image_classification.cpp?h=10.01.00.06#n184

0 Sajan Saji 1 month ago in reply to Reese Grimsley

Intellectual 490 points

Reese Grimsley said:
There are scripts under edgeai-gst-apps/scripts to build/install each of the components within the edgeai-app-stack

Then is it possible to create app like apps_python. So I think it is easy to modify the code without changing plugins. Is it means apps_python post_process.py code override the edgeai-dl-inferer

Processors

Processors forum

SK-AM62A-LP: TIDL Inference on AM62A