SK-AM62: How to test / demo the AM62 AI?

Hello Rio,

Ti Deep Learning (TIDL) refers to the software packages that program hardware accelerators for Deep Learning. AM62X does not include this hardware, although the AM62A does. Therefore, there is not TIDL support on AM62X.

There is support for open source runtimes Tensorflow-Lite and ONNX for running neural network inference on the CPU. That includes CPP and Python bindings - all AM6x processors include this in the default linux images, but not all devices have scripts available in the default to test this.

Can you tell me more about what you're trying to show?

Best,
Reese

Hi Reese:

Due to we have show some customers the AM62 has the 1~2 TOPS for AI, and customer knows the AI can be done by pure ARM cores.

So, do we have the plan to merge the basic ARM AI demo in the native pre-build image?

Because we don't have the AM62A to prove the AI for now.

And some of the customers only needs the lowest performance of pure ARM code AI.

Please consider the pure ARM code AI demo on the AM62.

Thanks.

BR Rio

Hi Rio,

The edgeai-gst-apps repo hosts some demos that you should be able to run on the AM62X using arm-only. There is a setup script and an init script that needs to run first, which requires internet connection. This will download some models to the device and a few packages that are not part of the SDK. I do not think there are currently plans to include this in the default image, but it is present on AM62A's default SDK iamge

https://github.com/TexasInstruments/edgeai-gst-apps

This will construct a full gstreamer pipeline to process input images, videos, or camera feed, run deep learning, and show output on the display or write to output file. The terminal will show the inference speed (per image) and total FPS of the entire gstreamer pipeline.

Best,
Reese

Hi Resse:

Thanks for these, 

I learned these from you thru the other E2E I posted for this kind of query.

But, it will be benefit for Local FAE and BU if you can "default" have the Pure ARM core AI running on all the AM62 family.

This will make us / customer easy to show / evaluate the AI demo. (On Android + Linux Both.)

Please consider to merge in this way.

BR Rio

Hi Resse:

I just have free time for testing this.

https://github.com/TexasInstruments/edgeai-gst-apps

#1. I have copied this onto the SD card, inserted the card onto EVM.

#2. But, from the git, there is no detail document for guiding me how to do.

      Do you have the document for this?

BR Rio

Hi Resse:

I need to make this working before 4/12, please guide me how to make this working.

THanks.

BR Rio

BTW, im wondering if this is actually working.

plz see the below errors:

CONTRIBUTING LICENSE README.md apps_cpp apps_python configs docker download_models.sh download_test_data.sh init_script.sh scripts setup_script.sh tests
root@am62xx-evm:/opt/edgeai-gst-apps-main# ./init_script.sh
bash: /opt/edgeai-gst-apps/scripts/setup_cameras.sh: No such file or directory
root@am62xx-evm:/opt/edgeai-gst-apps-main# ./download_models.sh
ERROR: not supported
root@am62xx-evm:/opt/edgeai-gst-apps-main# ./download_test_data.sh
Connecting to software-dl.ti.com (23.11.170.192:443)
Connecting to downloads.ti.com (23.76.85.123:443)
wget: server returned error: HTTP/1.1 404 Not Found
tar: edgeai-test-data.tar.gz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
Connecting to software-dl.ti.com (23.11.170.192:443)
Connecting to downloads.ti.com (23.76.85.123:443)
wget: server returned error: HTTP/1.1 404 Not Found
tar: -oob-demo-assets.tar.gz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
rm: invalid option -- 'o'
Try 'rm --help' for more information.
mv: invalid option -- 'o'
Try 'mv --help' for more information.
ln: failed to create symbolic link '/videos/': No such file or directory

Hello Rio,

Seems there are some problems, yes. It looks like recent changes for adding more device to edgeai-gst-apps affected AM62X, since it is an Arm-only target and compiled models are not valid for it. I would recommend setting the SOC am62a so that it downloads some models.

export SOC=am62a

The script to start with is "setup_script.sh" instead of "init_script.sh". The setup_script runs once, and the init_script is meant to run in a new terminal session before running any of the other python examples.

The first line (setup_cameras.sh) with issue is because you have the directory at /opt/edgeai-gst-apps-main instead of /opt/edgeai-gst-apps. Assuming the absolute path like this should be changed in the init_script - I will raise with the team (as well as a readme). I would recommend for now that you  rename your directory to match /opt/edgeai-gst-apps.

Edge AI SDK Documentation (Running an app; Configuring Applications) is relevant for how to run the examples. This applies after running the setup_script.sh (once forever) and init_script.sh (once per terminal). The workflow is the same between. Recommended to have USB camera and HDMI monitor connected. Selection of input source and output sink depends on the config file used.

I would recommend running apps_python/app_edgeai.py with configuration configs/image_classification.yaml. Please modify the SOC env variable back to SOC=am62 before running these examples.

I will validate that examples work after data/models are downloaded on 8.5 release for my AM62X and get back to you within the next day.

Best,
Reese

Hi Resse:

Do you have the step by step document for making this?

BR Rio

BTW, I cannot use the CSI camera for this demo?

Customers are using the CSI camera already...

BR Rio

CSI camera does work on AM62x. There is a 15-pin flat-pin cable (FPC) similar to raspberry Pi and TDA4VM SK. Only cameras with onboard ISP will work, and they also require a driver to work. View the page here: https://dev.ti.com/tirex/explore/node?node=A__AIwN3jgf3NkdPtMYdoNKKw__linux_academy_am62x__XaWts8R__LATEST. This requires a device tree overlay be applied as part of the boot (boot partition, uEnv.txt - name_overlays variable). The main camera we recommend for this purpose is OV5640 since this has upstreamed linux support and a precompiled DTBO file.

Recent versions of the tools I linked on github are giving me issues with AM62X. I had validated these several months before, but recent changes to add more Jacinto and Sitara MPU Edge AI devices have affected Arm-only targets. Issue is raised with the development team.

Nonetheless, I can confirm pipelines with AI and gstreamer on live camera input do work. There is not a published guide on this, but I can share example code in an FAQ for better visibility for arm-only devices in the short term.

Best,
Reese

Setup_scripts.sh gives me this error:

In file included from /opt/edgeai-dl-inferer/dl_inferer/src/ti_dl_inferer.cpp:71:
/opt/edgeai-dl-inferer/dl_inferer/include/ti_dlr_inferer.h:70:10: fatal error: dlr.h: No such file or directory
70 | #include <dlr.h>
| ^~~~~~~
compilation terminated.
make[2]: *** [dl_inferer/CMakeFiles/edgeai_dl_inferer.dir/build.make:63: dl_inferer/CMakeFiles/edgeai_dl_inferer.dir/src/ti_dl_inferer.cpp.o] Error 1
make[2]: Leaving directory '/opt/edgeai-dl-inferer/build'
make[1]: *** [CMakeFiles/Makefile2:199: dl_inferer/CMakeFiles/edgeai_dl_inferer.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 12%] Building CXX object post_process/CMakeFiles/edgeai_post_process.dir/src/ti_post_process_config.cpp.o
[ 16%] Building CXX object post_process/CMakeFiles/edgeai_post_process.dir/src/ti_fonts.cpp.o
[ 20%] Building CXX object post_process/CMakeFiles/edgeai_post_process.dir/src/ti_post_process_utils.cpp.o
[ 24%] Building CXX object post_process/CMakeFiles/edgeai_post_process.dir/src/ti_post_process_image_classification.cpp.o
[ 28%] Building CXX object post_process/CMakeFiles/edgeai_post_process.dir/src/ti_post_process_object_detection.cpp.o
[ 32%] Building CXX object post_process/CMakeFiles/edgeai_post_process.dir/src/ti_post_process_semantic_segmentation.cpp.o
[ 36%] Building CXX object post_process/CMakeFiles/edgeai_post_process.dir/src/ti_post_process_human_pose_estimation.cpp.o
[ 40%] Linking CXX static library ../../lib/Release/libedgeai_post_process.a
make[2]: Leaving directory '/opt/edgeai-dl-inferer/build'
[ 40%] Built target edgeai_post_process
make[1]: Leaving directory '/opt/edgeai-dl-inferer/build'
make: *** [Makefile:130: all] Error 2
Setup FAILED! : Make sure you have active network connection
Setup FAILED! : Make sure you have active network connection

Hi Reese:

Sorry, SDK8.6 needs to use the HF-FS EVM to test.

I'm using the SDK8.5 now.

So, SDK8.5 is not able to run the stuff you mentioned on the above?

BR Rio

I build this package in the ubuntu 18.04, it gives me no errors.

But if I build this package on the 62 SK EVM, it gives me the errors (as the previous log posted above.

In the previous conversation.

There are 2 sentences:

#1.

since it is an Arm-only target and compiled models are not valid for it. I would recommend setting the SOC am62a so that it downloads some models.

#2.

I would recommend running apps_python/app_edgeai.py with configuration configs/image_classification.yaml. Please modify the SOC env variable back to SOC=am62 before running these examples.

My questions:

Q1. may you clearly list the steps in detail sequence?

Q2.I need to setup those scripts on the ubuntu or the SKEVM? Im very confused.

Q3. May you share your working image of Demo AI with CSI? (I just made my CSI working last night.)

BR Rio

Hello Rio,

Lots of comments and questions floating around in this thread. I'll hit as many of them as I can.

- edgeai-tidl-tools: There has been a regression on ARM-only evaluation for devices like AM62X. This is not currently working, but I am in contact with owners of this software to ensure it gets fixed. I would not expect this to work for the time being.

- edgeai-gst-apps: Similar to above. There is infrastructure in place for AM62X but it is not fully functional on the device.

- SDK versioning: 8.6 is missing necessary dependencies for openCV and Tensorflow--lite. ONNX runtime is present. Use 8.5 images instead

- Models: Compiled models always include a general file like MODEL-NAME.onnx  or MODEL-NAME.tflite. Those files on their own are sufficient to run a model on CPU. Models generally have to be downloaded to the device if the device does not have an Edge AI SDK build (AM62X does not). The edgeai-tidl-tools follows a workflow in which you compile the models on PC and transfer to target. For AM62X, you still need to download and transfer to target (no compilation required). I recommended attempting to download AM62A so you could easily get a few models. Unfortunately, the download_models.sh script has a dependency not met on Am62X linux image. Instead, download directly from model-zoo

- Setup: edgeai-tidl-tools requires setup on the ubuntu host. The necessary dependencies should already be on AM62X

- Build failures: This build should not happen on an arm-only target, as far as I know. It's failing because DLR/TVM is not available on the device - this is not a supported runtime on arm-only devices for TI processors currently. DL-inferer is a plugin meant for devices with AI accelerators. In short, do not run the setup script for edgeai-tidl-tools on the target

Summarizing, typical tools that I would recommend are not in a functional state. However, 8.5 still has the necessary dependencies to run AI applications.

In the TAR archive I'm attaching, there are two models and a python3 script "gstreamer-ai-pipeline-arm.py". This will construct a gstreamer pipeline to run a model on live input and label the image based on the recognition. I've written this to work for image classification on imagenet and object detection on COCO-trained models. There is a small help dialog to select the model type and training set. 

 https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/gst_2D00_ai_2D00_pipeline_2D00_armonly.tar.xz

If you add a new model, it is important that a param.yaml file is provided (these are generated by tidl-tools when compiling a model, and include info about preprocessing, postprocessing, etc. A simpler version for AI inference with tflite is the cls_tfl_arm.py script. 

If you need to use a CSI camera, alter the function that is creating the Gstreamer string, such that it uses the correct input format, device, etc.

This will function identically (well, slower on AM62X) on PC and target device, provided the dependencies of OpenCV, TFLite, ONNX runtime, and gstreamer are present.

Best,
Reese

Hi Reese:

So, SDK8.5 / SDK8.6 are all cracked for Edge AI camera demo?

My purpose is simple:

My EVM is GP E3.

Please give me the demo image for proving the EdgeAI camera working well on AM62 for the customers.

Thanks.

BR Rio

Hi Rio,

8.5 is functional for Edge AI camera demo. The gst-ai-pipeline-armonly.tar.gz above is verified for AM62X-SK E2 EVM on 8.5.

-Reese

Hi Reese:

May we know what's the demo setup detail steps?

The CSI caemra is not activated.

BR Rio

root@am62xx-evm:~/vision-edgeai-app# ls
cls_tfl_arm.py coco_classes.txt gst-ai-pipeline-armonly.tar.xz gstreamer-ai-pipeline-arm.py imagenet_classes.txt model run_am62x_gst-ai-arm.sh setup_cameras.sh
root@am62xx-evm:~/vision-edgeai-app# ./setup_cameras.sh
Unable to setup formats: Device or resource busy (16)
CSI Camera 0 detected
device = /dev/video0
name = ov5640
format = [fmt:UYVY8_1X16/1280x720@1/30]
subdev_id = /dev/v4l-subdev2
isp_required = no