AM62A7: Stream Switching Crash Issue on AM62A (wave5 Decoder)

Hi Sajan,

What SDK version are you using to validate this? Can you share the application and or gstreamer pipeline that you are running to switching the streams?

Best Regards

Suren

Hello Suren,

Suren Porwar said:

What SDK version are you using to validate this?

SDK 10.01.

Gstreamer pipeline:

pipeline_str = (
            f"udpsrc port={config['port']} address=127.0.0.1 "
            "caps=\"application/x-rtp,media=video,encoding-name=H264,payload=96\" ! "
            "rtph264depay ! h264parse config-interval=-1 ! "
            "v4l2h264dec capture-io-mode=dmabuf ! "
            "kmssink sync=false force-modesetting=true"
        )

BR,
Sajan

Hi Sajan,

If you can migrate to using latest SDK and follow the procedure provided below to upgrade the Gstreamer version to 1.24.13, you would not have the issue of memory leaks when switching between video streams.

Please find below the Migration to Gstreamer 1.24.13 and Latest kernel steps using Yocto Scarthgap.

The Folder contains two  things:

A WIC mage that can be tested with AM62P EVM.

Yocto build files latest ZIP file. 

Link:tidrive.ext.ti.com/.../de3f3067-fc1a-439a-844b-4a4ac9bf9310

Access code:cS1gyv2_

Steps to replicate the build on your end with these changes on AM62P EVM assuming you are using the oe layer config to setup the environment using 

processor-sdk-scarthgap-chromium-11.01.16.13-config.txt

1.

Unzip the yocto-build-files-latest.zip file. You should see three folders (oe-core, arago, ti) each containing the patches to be applied.

2.

Copy respective patches in sources/oe-core, sources/meta-arago and sources/meta-ti directory of your yocto setup.

3.

cd into sources/oe-core and run git am 0001-move-oe-core-to-1.24.13.patch
cd into sources/meta-arago and run git am 0001-meta-arago-patches-to-migrate-to-1.24.13.patch
cd into source/meta-ti and run git am 0001-meta-ti-meta-ti-bsp-update-to-latest-kernel.patch

4.

remove qt6 and chromium from config to speed up the time and remove the qt6 dependency error
- this will be in the build/conf/bblayers.conf

5.

add lines to bottom of local.conf to remove additional dependencies that require qt6
IMAGE_INSTALL:remove = "ti-apps-launcher qmltermwidget"
DEMOS:remove = "ti-apps-launcher"

6.

update file in meta-ti-sdk layer to change PV for the gstreamer file
- found in sources/meta-tisdk/meta-ti-foundational/recipes-multimedia/gstreamer/

mv  gstreamer1.0-plugins-good_1.22.%.bbappend gstreamer1.0-plugins-good_1.24.%.bbappend

7.  Go to build directory and issue the build.

build> . conf/setenv

build> MACHINE=am62pxx-evm bitbake -k tisdk-default-image

Let me know how it goes.

Best Regards

Suren

Hi Suren,

Thank you for sharing this — very helpful post. I am using an AM62A EVM and wanted to check if these same patches and steps are applicable for AM62A as well, or if there are any AM62A-specific changes needed.

Also, I am unable to access the TI Drive link you shared. I tried logging in via ENT.TI.COM but both login options are failing on my end. Could you check if the link/access permissions are still valid, or share the files via an alternate method?

Additionally, could you clarify why the kernel update (the meta-ti-bsp patch) is also necessary alongside the GStreamer upgrade? Is the memory leak partly a kernel/driver-side issue?

I am currently on SDK 10.01 and haven't yet tested my application on the Scarthgap SDK. Would you recommend first validating the application on Scarthgap before applying these GStreamer patches, or should both changes be done together?

Thanks again for your help.

Best Regards,
Sajan

Hi Sajan,

Since the Gstreamer version hasn't changed from 10.1 to 11.1 SDK and codec IP is same for AM62A/AM62 this shoudl work.

If you can take just the gstreamer related patches and don't want to upgrade the kernel, then it should work even on AM62A.

Let me know how it goes.

Best Regards,

Suren

Hello Suren,

Could you please help me to access the patch files mentioned you

Best Regards,
Sajan

Hi Sajan,

My previous post had the information to download:

Suren Porwar said:

A WIC mage that can be tested with AM62P EVM.

Yocto build files latest ZIP file. 

Link:tidrive.ext.ti.com/.../de3f3067-fc1a-439a-844b-4a4ac9bf9310

Access code:cS1gyv2_

Zip file should have the patches.

Best Regards

Suren

Hello Suren,

I tried to access that, unfortunately login not working for me.

Best Regards,
Sajan

Hi Sajan,

Check if you are able to access this:

https://tidrive.ext.ti.com/u/hRhdTc0YXQqXGF1f/de3f3067-fc1a-439a-844b-4a4ac9bf9310?l

jc2hH9`C

Best Regards

Suren

Hello Suren,

You mentioned using processor-sdk-scarthgap-chromium-11.01.16.13-config.txt as the base — could you also confirm whether I should go with processor-sdk-analytics-11.01.07.05 or 11.02.00 instead of that? And can I apply the given patches on that?

Also, could you please verify that the following logs are not a programmatic error (such as improper resource cleanup, memory not being freed, etc.) and are just known/expected warnings from the decoder/encoder?

v4l2h264dec0: Too old frames, bug in decoder -- please file a bug
vdec 30210000.video-codec: wave5_vpu_enc_finish_encode: encoded buffer (1) was not in ready queue 0.
vdec 30210000.video-codec: wave5_vpu_enc_finish_encode: no source buffer with index: 1 found

Thanks!

Best Regards,
Sajan

Hi Sajan,

Are you using AM62A or AM62P for validation? If using AM62P, then scarthgap-chromimum-11.01.16.13-config.txt as base would work. 

For AM62A - you need to use the 

Sajan Saji said:

processor-sdk-analytics-11.01.07.05

All the above warnings/errors have been fixed in the latest SDK releases.  

Best Regards

Suren

Dear Suren,

Yes I am using AM62A.
Tried to patch using the provided patches in processor-sdk-analytics-11.01.07.05.

sources/oe-core$ git am 0001-move-oe-core-to-1.24.13.patch 
Applying: move oe-core to 1.24.13
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base/CVE-2025-47806.patch: does not exist in index
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base/CVE-2025-47807.patch: does not exist in index
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base/CVE-2025-47808.patch: does not exist in index
error: patch failed: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.22.12.bb:10
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.22.12.bb: patch does not apply
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good/CVE-2025-47183-001.patch: does not exist in index
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good/CVE-2025-47183-002.patch: does not exist in index
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good/CVE-2025-47219.patch: does not exist in index
error: patch failed: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.22.12.bb:4
error: meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.22.12.bb: patch does not apply
Patch failed at 0001 move oe-core to 1.24.13
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

sources/meta-ti$ git am 0001-meta-ti-meta-ti-bsp-update-to-latest-kernel.patch 
Applying: meta-ti: meta-ti-bsp: update to latest kernel
error: patch failed: meta-ti-bsp/recipes-kernel/linux/linux-ti-staging_6.12.bb:22
error: meta-ti-bsp/recipes-kernel/linux/linux-ti-staging_6.12.bb: patch does not apply
Patch failed at 0001 meta-ti: meta-ti-bsp: update to latest kernel
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

sources/meta-arago$ git am 0001-meta-arago-patches-to-migrate-to-1.24.13.patch 
Applying: meta-arago patches to migrate to 1.24.13
.git/rebase-apply/patch:165: trailing whitespace.
 
.git/rebase-apply/patch:169: trailing whitespace.
 
.git/rebase-apply/patch:178: trailing whitespace.
 
.git/rebase-apply/patch:187: trailing whitespace.
 
.git/rebase-apply/patch:195: trailing whitespace.
 
warning: squelched 36 whitespace errors
warning: 41 lines add whitespace errors.

I tried manually patch every file and doing rebuild .Could you please suggest the fix?

Warm Regards,
Sajan

Hi Sajan

I am looking into this to replicate it on AM62A and will get back to you by mid next week. What are you using AM62A for? I thought you guys were working on AM62P, so shared those patches with you, which resolves few issues pertaining to V4L2 framework with gstreamer and fixes few memory leaks.

Best Regards

Suren

Hi Suren,

Thanks for looking into this.

I initially started this discussion specifically for AM62A, as mentioned in my original query "AM62A7: Stream Switching Crash Issue on AM62A (wave5 Decoder)".

Currently, I am working on AM62A, where I’m running two image classification pipelines simultaneously, streaming them via UDP, and trying to switch between the streams on the same board.

Terminal log of the tried build:

$ bitbake tisdk-edgeai-image
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
NOTE: Started PRServer with DBfile: /home/tisdk/tisdk/build/cache/prserv.sqlite3, Address: 127.0.0.1:40107, PID: 487060
Loading cache: 100% |##################################################################################################################################################################| Time: 0:00:06
Loaded 11148 entries from dependency cache.
WARNING: No recipes in default available for:
  /home/tisdk/tisdk/sources/meta-tisdk/meta-ti-foundational/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.22.%.bbappend
No recipes in k3r5 available for:
  /home/tisdk/tisdk/sources/meta-tisdk/meta-ti-foundational/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.22.%.bbappend
NOTE: Resolving any missing task queue dependencies
NOTE: Resolving any missing task queue dependencies
NOTE: Resolving any missing task queue dependencies
NOTE: Resolving any missing task queue dependencies
NOTE: Resolving any missing task queue dependencies
NOTE: Resolving any missing task queue dependencies

Build Configuration (mc:default):
BB_VERSION           = "2.8.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-22.04"
TARGET_SYS           = "aarch64-oe-linux"
MACHINE              = "am62axx-evm"
DISTRO               = "arago"
DISTRO_VERSION       = "2025.01"
TUNE_FEATURES        = "aarch64"
TARGET_FPU           = ""
meta-ti-foundational = "HEAD:3265fc78d1dddd1642c112a2f3f42507bfa176d9"
meta-arago-distro    
meta-arago-extras    
meta-arago-test      = "HEAD:c7435ac1d8cd45735d5371b0e88da8305b412a86"
meta-qt6             = "HEAD:2b4e60e319348ae2d53072b901645f30b4cc42bf"
meta-virtualization  = "HEAD:a5449c0c50aa07d02186f548fe6bb6c1ce8823dc"
meta-networking      
meta-python          
meta-oe              
meta-gnome           
meta-filesystems     
meta-multimedia      = "HEAD:e8fd97d86af86cdcc5a6eb3f301cbaf6a2084943"
meta-lts-mixins      = "HEAD:a8046d5ec53b1856169ac795aa87cb0d5db84c04"
meta-ti-extras       
meta-beagle          
meta-ti-bsp          = "HEAD:7b5114a4c7dbabcf520c9e3d2b235e5a77dc653b"
meta-arm             
meta-arm-toolchain   = "HEAD:8e0f8af90fefb03f08cd2228cde7a89902a6b37c"
meta-clang           = "HEAD:aef850f7fa53121c74b244b7ae40d31fb9809ccf"
meta                 = "HEAD:3550da705f9b25e08610f35bf1360ca7b360afd6"
meta-edgeai          = "HEAD:2f3ea75ca6733035d5a60fc2a3fc7d15af9e9851"


Build Configuration:
BB_VERSION           = "2.8.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-22.04"
TARGET_SYS           = "arm-oe-eabi"
MACHINE              = "am62axx-evm-k3r5"
DISTRO               = "arago"
DISTRO_VERSION       = "2025.01"
TUNE_FEATURES        = "arm armv7a vfp thumb callconvention-hard"
TARGET_FPU           = "hard"
meta-ti-foundational = "HEAD:3265fc78d1dddd1642c112a2f3f42507bfa176d9"
meta-arago-distro    
meta-arago-extras    
meta-arago-test      = "HEAD:c7435ac1d8cd45735d5371b0e88da8305b412a86"
meta-qt6             = "HEAD:2b4e60e319348ae2d53072b901645f30b4cc42bf"
meta-virtualization  = "HEAD:a5449c0c50aa07d02186f548fe6bb6c1ce8823dc"
meta-networking      
meta-python          
meta-oe              
meta-gnome           
meta-filesystems     
meta-multimedia      = "HEAD:e8fd97d86af86cdcc5a6eb3f301cbaf6a2084943"
meta-lts-mixins      = "HEAD:a8046d5ec53b1856169ac795aa87cb0d5db84c04"
meta-ti-extras       
meta-beagle          
meta-ti-bsp          = "HEAD:7b5114a4c7dbabcf520c9e3d2b235e5a77dc653b"
meta-arm             
meta-arm-toolchain   = "HEAD:8e0f8af90fefb03f08cd2228cde7a89902a6b37c"
meta-clang           = "HEAD:aef850f7fa53121c74b244b7ae40d31fb9809ccf"
meta                 = "HEAD:3550da705f9b25e08610f35bf1360ca7b360afd6"
meta-edgeai          = "HEAD:2f3ea75ca6733035d5a60fc2a3fc7d15af9e9851"

Sstate summary: Wanted 113 Local 108 Mirrors 0 Missed 5 Current 6886 (95% match, 99% complete)########################################################################                 | ETA:  0:00:01
Initialising tasks: 100% |#############################################################################################################################################################| Time: 0:00:17
NOTE: Executing Tasks
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
WARNING: Duplicate inclusion for /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/include/branding-core.inc in /home/tisdk/tisdk/sources/meta-arago/meta-arago-distro/conf/distro/arago.conf
ERROR: tisdk-edgeai-image-1.0-r0_edgeai_8 do_rootfs: Unable to install packages. Command '/home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/opkg.conf -t /home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/temp/ipktemp/ -o /home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/rootfs  --force_postinstall    --add-ignore-recommends busybox-syslog --add-ignore-recommends coreutils --add-ignore-recommends coreutils-dev --add-ignore-recommends gdbserver-c6x-dev --add-ignore-recommends libulm-dev install build-essential docker dtc edgeai-apps-utils-source edgeai-dl-inferer-source edgeai-dl-inferer-staticdev edgeai-gst-apps-dev edgeai-gst-apps-source edgeai-gst-plugins-dev edgeai-gst-plugins-source edgeai-init edgeai-studio-agent edgeai-test-data edgeai-tidl-models edgeai-tiovx-apps-dev edgeai-tiovx-apps-source edgeai-tiovx-kernels-dev edgeai-tiovx-kernels-source edgeai-tiovx-modules-dev edgeai-tiovx-modules-source git linux-libc-headers-dev opkg packagegroup-arago-base packagegroup-arago-console packagegroup-arago-gst-sdk-target packagegroup-arago-opencl packagegroup-arago-tisdk-addons packagegroup-arago-tisdk-addons-extra packagegroup-arago-tisdk-connectivity packagegroup-arago-tisdk-crypto packagegroup-arago-tisdk-graphics packagegroup-arago-tisdk-gtk packagegroup-arago-tisdk-multimedia packagegroup-arago-tisdk-sysrepo packagegroup-arago-vulkan packagegroup-core-boot packagegroup-core-tools-debug packagegroup-edgeai-tisdk-addons psplash resize-rootfs run-postinsts ti-gpio-cpp-dev ti-gpio-cpp-source ti-gpio-py ti-gpio-py-source ti-test ti-test-extras ti-tidl-dev ti-tidl-osrt-dev ti-tidl-osrt-staticdev ti-vision-apps-dev' returned 1:
 * Solver encountered 1 problem(s):
 * Problem 1/1:
 *   - package edgeai-init-1.0-r0_edgeai_0.0.aarch64 requires edgeai-gui-app, but none of the providers can be installed
 *   - conflicting requests
 *   - nothing provides gstreamer1.0-plugins-good-qml6 needed by edgeai-gui-app-1.0.0-r0_edgeai_1.0.aarch64
 * 
 * Solution 1:
 *   - do not ask to install a package providing edgeai-init

 * opkg_finalize_intercepts: Failed to open dir /home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/temp/ipktemp//opkg-VhEuDj/opkg-intercept-PMOJbu: No such file or directory.
 * rm_r: Failed to open dir /home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/temp/ipktemp//opkg-VhEuDj/opkg-intercept-PMOJbu: No such file or directory.
Disfavor package: busybox-syslog
Disfavor package: coreutils-dev
Disfavor package: coreutils


ERROR: Logfile of failure stored in: /home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/temp/log.do_rootfs.487730
ERROR: Task (/home/tisdk/tisdk/sources/meta-edgeai/recipes-core/images/tisdk-edgeai-image.bb:do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 15092 tasks of which 15084 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 4 seconds
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 4 seconds

Summary: 1 task failed:
  /home/tisdk/tisdk/sources/meta-edgeai/recipes-core/images/tisdk-edgeai-image.bb:do_rootfs
    log: /home/tisdk/tisdk/build/arago-tmp-default-glibc/work/am62axx_evm-oe-linux/tisdk-edgeai-image/1.0/temp/log.do_rootfs.487730
Summary: There were 9 WARNING messages.
Summary: There was 1 ERROR message, returning a non-zero exit code.

Thanks again for your support, and I’ll wait for your findings.

Best regards,
Sajan

Dear Suren,

Best regards,
Sajan

Hello Suren,

Could you please share the patch.

Warm Regards,
Sajan

Hello Suren,

Could you please provide the patch.

Warm Regards,
Sajan

Hi Sajan,

On AM62A, Can you take the wic Image and retest your application and let us know if it resolves the issue:

https://tidrive.ext.ti.com/u/zCbsiYQm7MPeJuyb/d6939a39-8f98-4131-a214-28705a2c4ebd?l

4mu3r!Mr

Let me know how it goes with this Image.

Best Regards

Suren

Dear Suren,

I tried my application on the wic seems some error log:

root@am62axx-evm:/opt# ./sample_app/app_edgeai.py config/custom_config.yaml -n
libtidl_onnxrt_EP loaded 0x12ca1420 
Final number of subgraphs created are : 1, - Offloaded Nodes - 124, Total Nodes - 124 
APP: Init ... !!!
   526.059427 s: APP_LOG: ERROR: Unable to map memory @ 0xa1000000 of size 262144 bytes !!!
APP: ERROR: Log writer init failed !!!
APP_LOG: ERROR: Unable to map memory @ 0xa2000000 of size 3288576 bytes !!!
APP: ERROR: File IO writer init failed !!!
APP: Init ... Done !!!
Traceback (most recent call last):
  File "/opt/./sample_app/app_edgeai.py", line 39, in <module>
    main(sys.argv)
  File "/opt/./sample_app/app_edgeai.py", line 16, in main
    demo = EdgeAIDemo(config)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/sample_app/edge_ai_class.py", line 94, in __init__
    model_obj.create_runtime()
  File "/usr/lib/python3.12/site-packages/edgeai_dl_inferer.py", line 315, in create_runtime
    self.run_time = RunTime(self.artifacts,
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/edgeai_dl_inferer.py", line 170, in __init__
    self.interpreter = _onnxruntime.InferenceSession(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 387, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 439, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Create state function failed. Return value:-1

Same issue also for the edgeai-gst-apps.
Could you please help me to fix

Warm Regards,
Sajan

Hi Sajan

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/libgstvideo4linux2.so

Please copy this to your rootfs (sudo cp libgstvideo4linux2.so /media/<username>/rootfs/usr/lib/gstreamer-1.0/libgstvideo4linux2.so)

and give it a try.

Hope this helps

Best Regards

Suren

Hi Suren,

Suren Porwar said:

e2e.ti.com/.../libgstvideo4linux2.so

Please copy this to your rootfs (sudo cp libgstvideo4linux2.so /media/<username>/rootfs/usr/lib/gstreamer-1.0/libgstvideo4linux2.so)

I tried this:

root@am62axx-evm:/opt# ls -la /usr/lib/gstreamer-1.0/libgstvideo*
-rwxr-xr-x 1 root root 398968 Apr 24 04:43 /usr/lib/gstreamer-1.0/libgstvideo4linux2.so
-rwxr-xr-x 1 root root  68352 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideobox.so
-rwxr-xr-x 1 root root  68728 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideoconvertscale.so
-rwxr-xr-x 1 root root  68704 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideocrop.so
-rwxr-xr-x 1 root root  68944 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideofilter.so
-rwxr-xr-x 1 root root  67896 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideofiltersbad.so
-rwxr-xr-x 1 root root  68504 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideoframe_audiolevel.so
-rwxr-xr-x 1 root root 134784 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideomixer.so
-rwxr-xr-x 1 root root 334408 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideoparsersbad.so
-rwxr-xr-x 1 root root  68512 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideorate.so
-rwxr-xr-x 1 root root  67712 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideosignal.so
-rwxr-xr-x 1 root root  68656 Mar  9  2018 /usr/lib/gstreamer-1.0/libgstvideotestsrc.so

Unfortunately, still errors exists.

root@am62axx-evm:/opt# ./sample_app/app_edgeai.py -n
libtidl_onnxrt_EP loaded 0x1a025d30 
Final number of subgraphs created are : 1, - Offloaded Nodes - 283, Total Nodes - 283 
APP: Init ... !!!
   552.519995 s: APP_LOG: ERROR: Unable to map memory @ 0xa1000000 of size 262144 bytes !!!
APP: ERROR: Log writer init failed !!!
APP_LOG: ERROR: Unable to map memory @ 0xa2000000 of size 3288576 bytes !!!
APP: ERROR: File IO writer init failed !!!
APP: Init ... Done !!!
Traceback (most recent call last):
  File "/opt/./sample_app/app_edgeai.py", line 38, in <module>
    main(sys.argv)
  File "/opt/./sample_app/app_edgeai.py", line 16, in main
    demo = EdgeAIDemo(config)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/sample_app/edge_ai_class.py", line 85, in __init__
    model_obj.create_runtime()
  File "/usr/lib/python3.12/site-packages/edgeai_dl_inferer.py", line 315, in create_runtime
    self.run_time = RunTime(self.artifacts,
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/edgeai_dl_inferer.py", line 170, in __init__
    self.interpreter = _onnxruntime.InferenceSession(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 387, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 439, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Create state function failed. Return value:-1

Hello Suren,

Could you please help for the same.

Regards,
Sajan

Hello Sajan,

Did you add the ti/k3-am62a7-sk-edgeai.dtbo in the name_overlays in uEnv.txt and see if the error that you are seeing vanishes. 

Refer: https://software-dl.ti.com/processor-sdk-linux/esd/AM62AX/11_01_07_05/exports/docs/devices/AM62AX/edgeai/faq.html#why-does-error-unable-to-map-memory-0xa2000000-appear-after-applying-a-custom-dtbo-using-name-overlays-from-sdk-11-1-with-edgeai

Best Regards

Suren

Hello Suren,

Did the suggested solution and it is working. I am trying to recompile the model for sdk v11.

I will update you after that about video switching on kmssink.

Thanks.

Best Regards,
Sajan

Hello Suren,

I tried testing the switching of two RTSP video streams through kmssink. While running the same application on SDK 10, I observed a gradual increase in RAM usage over time, which eventually led to an out-of-memory condition and application failure.

After removing the video-saving logic and its related features from both applications, the RAM usage remained stable and both applications continued running without any noticeable memory growth.

From my initial observations, the video switching behaviour appears stable on SDK 11, and the terminal errors that were previously seen are no longer occurring. However, during testing on SDK 11, the following kernel crash/error was observed (log attached).

Based on this behavior, could the RAM consumption issue be related to the video-saving implementation, buffer handling, or a possible memory leak within the pipeline? Please share your thoughts on what might be causing the gradual memory increase and eventual crash.

Log:

[  852.524614] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000160
[  852.533449] Mem abort info:
[  852.536284]   ESR = 0x0000000096000006
[  852.540073]   EC = 0x25: DABT (current EL), IL = 32 bits
[  852.545409]   SET = 0, FnV = 0
[  852.548485]   EA = 0, S1PTW = 0
[  852.551678]   FSC = 0x06: level 2 translation fault
[  852.556619] Data abort info:
[  852.559507]   ISV = 0, ISS = 0x00000006, ISS2 = 0x00000000
[  852.565039]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[  852.570137]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[  852.575466] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000888da6000
[  852.581925] [0000000000000160] pgd=08000008860a7003, p4d=08000008860a7003, pud=0800000886f11003, pmd=0000000000000000
[  852.592569] Internal error: Oops: 0000000096000006 [#1] PREEMPT SMP
[  852.598839] Modules linked in: wave5 imx219 v4l2_cci ds90ub953 rpmsg_ctrl snd_soc_hdmi_codec rpmsg_char e5010_jpeg_enc snd_soc_simple_card snd_soc_simple_card_utils crct10dif_ce v4l2_jpeg displa]
[  852.672472] CPU: 3 UID: 0 PID: 1567 Comm: vpu_irq_thread Tainted: G           O       6.12.35-ti-00946-g72f48d59b8f0 #1
[  852.683250] Tainted: [O]=OOT_MODULE
[  852.686734] Hardware name: Texas Instruments AM62A7 SK (DT)
[  852.692298] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  852.699256] pc : wave5_vdi_write_register+0x0/0x14 [wave5]
[  852.704761] lr : wave5_vpu_clear_interrupt+0x34/0x48 [wave5]
[  852.710426] sp : ffff800083ff3d80
[  852.713732] x29: ffff800083ff3d80 x28: ffff000812c24000 x27: 0000000000005150
[  852.720861] x26: 0000000000000100 x25: 0000000000000000 x24: ffff000812cbfd10
[  852.727997] x23: 0000000000000000 x22: 0000000000000100 x21: 0000000000000000
[  852.735132] x20: 0000000000000100 x19: ffff000812c24000 x18: 0000000000000001
[  852.742267] x17: 0000000000000000 x16: 0000000000000000 x15: 010761a223d989a0
[  852.749399] x14: 029d887c69ac319e x13: ffff0008012a5300 x12: 0000000000000001
[  852.756534] x11: ffff0008012a5300 x10: ffff00080efeb200 x9 : ffff00087f8272c0
[  852.763666] x8 : ffff0008012a5300 x7 : ffff000801bc0710 x6 : 0000000000000002
[  852.770802] x5 : 0000000000000400 x4 : 0000000000000902 x3 : ffff00080efeb180
[  852.777935] x2 : 0000000000000000 x1 : 0000000000000030 x0 : 0000000000000000
[  852.785068] Call trace:
[  852.787513]  wave5_vdi_write_register+0x0/0x14 [wave5]
[  852.792666]  wave5_vpu_handle_irq+0xdc/0x1a8 [wave5]
[  852.797639]  wave5_vpu_irq_work_fn+0x3c/0x4c [wave5]
[  852.802613]  kthread_worker_fn+0xc4/0x174
[  852.806629]  kthread+0x110/0x114
[  852.809851]  ret_from_fork+0x10/0x20
[  852.813431] Code: 9104a042 91120021 95b486de 17fffff4 (f940b000) 
[  852.819517] ---[ end trace 0000000000000000 ]---

Warm Regards,
Sajan

Hi Sajan,

Can you help us replicate this issue on 11.1 SDK? As we are not seeing this issue on our end.

Best Regards

Suren

Hello Suren,

I tried to paste the code but code insertion is not working. I tried two applications at same time, both sink are rtsp, switch the stream by a simple python script. Got the issue after running it for more than 30 minutes.

Warm Regards,
Sajan

Hi Sajan,

Can we sync up on Thursday morning(Dallas time) to see what might be happening on your setup?

Best Regards

Suren

Hello Suren,

Yes, Sure.

Warm Regards,
Sajan

Sent you an email with details.

Best Regards

Suren

Hi Sajan

I currently ran the below pipeline with IMX219 connected to my AM62A with Gstreamer upgraded to 1.24 as per the patch that I had provided earlier. I am able to start/stop the pipeline every 2 seconds and don't see any issue.

I have ran an application like one below:

/* imx219-switch-fakesink.c
 *
 * GStreamer pipeline: v4l2src -> tiovxisp -> v4l2h264enc -> fakesink
 * Cycles: record 2s, pause 2s, repeat.
 *
 * Build:
 *   gcc $(pkg-config --cflags --libs gstreamer-1.0) \
 *       -o imx219-switch-fakesink imx219-switch-fakesink.c
 */

#include <gst/gst.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>

#define DEVICE "/dev/video-imx219-cam0"

#define PIPELINE_STR \
    "v4l2src device=/dev/video-imx219-cam0 io-mode=dmabuf-import ! " \
    "video/x-bayer,width=1920,height=1080,format=rggb ! " \
    "tiovxmemalloc pool-size=2 ! queue max-size-bytes=0 ! " \
    "tiovxisp sensor-name=\"SENSOR_SONY_IMX219_RPI\" "\
    "dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin " \
    "sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin sink_0::device=/dev/v4l-imx219-subdev0 ! " \
    "video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! tee name=t " \
    "t. ! queue ! kmssink driver-name=tidss sync=false " \
    "t. ! queue ! v4l2h264enc ! fakesink "

static GstElement *pipeline = NULL;
static GstBus     *bus      = NULL;

static volatile int keep_running = 1;
static volatile int is_running   = 0;

/* ------------------------------------------------------------------ */

/* Poll the bus for error / EOS messages (non-blocking). */
static void check_messages(void)
{
    if (!bus)
        return;

    GstMessage *msg;
    while ((msg = gst_bus_pop_filtered(bus, GST_MESSAGE_ERROR | GST_MESSAGE_EOS))) {
        switch (GST_MESSAGE_TYPE(msg)) {
        case GST_MESSAGE_ERROR: {
            GError *err   = NULL;
            gchar  *debug = NULL;
            gst_message_parse_error(msg, &err, &debug);
            fprintf(stderr, "Error: %s, %s\n",
                    err->message, debug ? debug : "none");
            g_error_free(err);
            g_free(debug);
            keep_running = 0;
            break;
        }
        case GST_MESSAGE_EOS:
            printf("End of stream reached\n");
            break;
        default:
            break;
        }
        gst_message_unref(msg);
    }
}

/* Sleep for `ms` milliseconds in 100 ms increments, checking messages
 * and honouring the keep_running flag. */
static void sleep_ms(int ms)
{
    for (int i = 0; i < ms / 100 && keep_running; i++) {
        usleep(100000);
        check_messages();
    }
}

/* ------------------------------------------------------------------ */

static void create_pipeline(void)
{
    GError *error = NULL;

    pipeline = gst_parse_launch(PIPELINE_STR, &error);
    if (!pipeline || error) {
        fprintf(stderr, "Failed to create pipeline: %s\n",
                error ? error->message : "unknown error");
        if (error)
            g_error_free(error);
        exit(1);
    }

    bus = gst_element_get_bus(pipeline);
}

static void start_recording(void)
{
    if (pipeline) {
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        pipeline = NULL;
    }
    if (bus) {
        gst_object_unref(bus);
        bus = NULL;
    }

    create_pipeline();
    gst_element_set_state(pipeline, GST_STATE_PLAYING);
    printf("Recording started\n");
    is_running = 1;
}

static void stop_recording(void)
{
    if (pipeline) {
        gst_element_send_event(pipeline, gst_event_new_eos());
        usleep(500000); /* 0.5 s – let EOS propagate */
        gst_element_set_state(pipeline, GST_STATE_NULL);
        printf("Recording stopped\n");
    }
    is_running = 0;
}

static void cleanup(void)
{
    if (pipeline) {
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        pipeline = NULL;
    }
    if (bus) {
        gst_object_unref(bus);
        bus = NULL;
    }
    printf("Cleanup complete\n");
}

/* ------------------------------------------------------------------ */

static void signal_handler(int sig)
{
    (void)sig;
    printf("\nInterrupted by user. Stopping...\n");
    keep_running = 0;
}

int main(int argc, char *argv[])
{
    gst_init(&argc, &argv);
    signal(SIGINT, signal_handler);

    while (keep_running) {
        printf("\n--- Starting new recording cycle ---\n");

        start_recording();
        sleep_ms(2000);
        stop_recording();

        printf("Pausing for 2 seconds\n");
        sleep_ms(2000);
    }

    if (is_running)
        stop_recording();

    cleanup();
    return 0;
}

Log:

root@am62axx-evm:/home# gst-launch-1.0 -v \
> v4l2src io-mode=dmabuf-import device=/dev/video-imx219-cam0 ! video/x-bayer,width=1920,height=1080,format=rggb ! \
> tiovxmemalloc pool-size=2 ! queue max-size-bytes=0 ! \
> tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin \
> sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin sink_0::device=/dev/v4l-imx219-subdev0 ! \
> video/x-raw,format=NV12,width=1920, height=1080, framerate=30/1 ! tee name=t \
> t. ! queue ! kmssink driver-name=tidss sync=false \
> t. ! queue ! v4l2h264enc ! fakesink
APP: Init ... !!!
  2047.487850 s: MEM: Init ... !!!
  2047.487906 s: MEM: Initialized DMA HEAP (fd=8) !!!
  2047.488089 s: MEM: Init ... Done !!!
  2047.488104 s: IPC: Init ... !!!
  2047.505551 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  2047.513725 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  2047.517790 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
  2047.517844 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
  2047.517857 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
  2047.520179 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0
  2047.520371 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1
  2047.520500 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2
  2047.520612 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3
  2047.520628 s:  VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
  2047.520640 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 1920
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 1200
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstTIOVXMemAlloc:tiovxmemalloc0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstTIOVXMemAlloc:tiovxmemalloc0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXMisoPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstTee:t.GstTeePad:src_0: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstTee:t.GstTeePad:src_1: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstTee:t.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
Redistribute latency...
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
Redistribute latency...
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:12.423291695
Setting pipeline to NULL ...
Freeing pipeline ...
root@am62axx-evm:/home# vi imx219-switch-dispenc.c
root@am62axx-evm:/home# gcc -o test imx219-switch-dispenc.c $(pkg-config --cflags --libs gstreamer-1.0)
root@am62axx-evm:/home# ./test

--- Starting new recording cycle ---
APP: Init ... !!!
  2137.237879 s: MEM: Init ... !!!
  2137.237948 s: MEM: Initialized DMA HEAP (fd=8) !!!
  2137.238099 s: MEM: Init ... Done !!!
  2137.238112 s: IPC: Init ... !!!
  2137.255794 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  2137.259882 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  2137.260032 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
  2137.260048 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
  2137.260059 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
  2137.260769 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0
  2137.261055 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1
  2137.261305 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2
  2137.261553 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3
  2137.261588 s:  VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
  2137.261600 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
Recording started
Recording stopped
Pausing for 2 seconds

--- Starting new recording cycle ---
Recording started
Recording stopped
Pausing for 2 seconds

--- Starting new recording cycle ---
Recording started
Recording stopped
Pausing for 2 seconds

--- Starting new recording cycle ---
Recording started
Recording stopped
Pausing for 2 seconds

--- Starting new recording cycle ---
Recording started
Recording stopped
Pausing for 2 seconds

--- Starting new recording cycle ---
Recording started
Recording stopped
Pausing for 2 seconds

--- Starting new recording cycle ---
Recording started
Recording stopped
Pausing for 2 seconds

--- Starting new recording cycle ---
Recording started
Recording stopped

Can you try and run the same application on your end and provide your observations so we can setup a debug call later this week.

Best Regards

Suren

Hello Suren,

Could you please try switching the streams by the keystroke or mouse event. Also I used 2 cameras on rx0 and rx1. The sink used is udpsink then accessed the videos from the ports.

Best Regards,
Sajan

Hi Sajan,

So you are encoding each of the camera capture and streaming it on Network. You keep switching between these cameras every 2secs? Were you able to run the application that I provided?

Can you share your application/sample example pipeline on how you are switching?

Best Regards

Suren

Hi Suren,

Suren Porwar said:

So you are encoding each of the camera capture and streaming it on Network

Yes. Also I can run the application you provided and it is working.

Suren Porwar said:

Can you share your application/sample example pipeline on how you are switching?

I am sharing as Google drive link because of I can't insert code in this chat.
https://docs.google.com/document/d/1blvsq_zr3Qj33_jXYlz2Um9BMTMi-P01DoySqMUTPL4/edit?usp=sharing

Keep the CL and OD applications output to remote.
For Example:

outputs:
    output0:
        sink: remote
        host: 127.0.0.1
        port: 8081
        encoding: h264
        width: 1280
        height: 720
        bitrate: 4000000
        gop-size: 10


Warm Regards,
Sajan

Hi Sajan,

A Modified version of your code that fixes a few issues with your code:

• ✓ Added debouncing to prevent rapid stream switching
• ✓ Added GStreamer bus error handling
• ✓ Drain all input events (not just one)
• ✓ Reset modifier state on errors
• ✓ Better error logging throughout

#!/usr/bin/env python3
import os
import sys
import gi
import time
import struct
import select
import threading
import termios
import tty
import signal
import ctypes

gi.require_version('Gst', '1.0')
from gi.repository import Gst, GLib

Gst.init(None)

_libc       = ctypes.CDLL("libc.so.6", use_errno=True)
IN_CREATE   = 0x100
IN_DELETE   = 0x200
IN_NONBLOCK = 0x800
_INEVT_HDR  = struct.Struct('iIII')

def _inotify_init():
    fd = _libc.inotify_init1(IN_NONBLOCK)
    if fd < 0:
        raise OSError(ctypes.get_errno(), "inotify_init1 failed")
    return fd

def _inotify_add_watch(ifd, path, mask):
    wd = _libc.inotify_add_watch(ifd, path.encode(), mask)
    if wd < 0:
        raise OSError(ctypes.get_errno(), f"inotify_add_watch on {path}")
    return wd

def _drain_inotify(ifd):
    names = set()
    try:
        while True:
            raw = os.read(ifd, 4096)
            off = 0
            while off + _INEVT_HDR.size <= len(raw):
                wd, mask, cookie, length = _INEVT_HDR.unpack_from(raw, off)
                off += _INEVT_HDR.size
                if length:
                    name = raw[off:off + length].rstrip(b'\x00').decode(errors='replace')
                    names.add(name)
                    off += length
    except BlockingIOError:
        pass
    return names

def is_tty():
    return os.isatty(sys.stdin.fileno())

def hide_cursor():
    if is_tty():
        sys.stdout.write("\033[?25l")
        sys.stdout.flush()

def show_cursor():
    if is_tty():
        sys.stdout.write("\033[?25h")
        sys.stdout.flush()

def disable_echo():
    if not is_tty():
        return None
    fd  = sys.stdin.fileno()
    old = termios.tcgetattr(fd)
    tty.setcbreak(fd)
    s   = termios.tcgetattr(fd)
    s[3] &= ~termios.ECHO
    termios.tcsetattr(fd, termios.TCSADRAIN, s)
    return old

def restore_terminal(old):
    if old is None:
        return
    termios.tcsetattr(sys.stdin.fileno(), termios.TCSADRAIN, old)

class VideoSwitcher:
    STREAMS = {
        '1': {'port': 8081, 'name': 'stream1'},
        '2': {'port': 8082, 'name': 'stream2'},
    }
    TRIGGER_MODIFIER_KEYS = {56, 100}
    TRIGGER_ACTION_KEY    = 15
    TRIGGER_MOUSE_CODES   = {272}
    INPUT_DIR             = '/dev/input'
    _EVT = struct.Struct('llHHI')

    def __init__(self):  # ✓ FIXED
        self.current_pipeline = None
        self.current_key      = '1'
        self.loop             = GLib.MainLoop()
        self._alt_held        = False
        self._switching       = False  # ✓ Prevent rapid toggling

    def _pipeline_str(self, key):
        cfg = self.STREAMS[key]
        return (
            f"udpsrc port={cfg['port']} address=127.0.0.1 "
            "caps=\"application/x-rtp,media=video,encoding-name=H264,payload=96\" ! "
            "rtph264depay ! h264parse config-interval=-1 ! "
            "v4l2h264dec capture-io-mode=dmabuf ! "
            "kmssink sync=false force-modesetting=true"
        )

    def start_stream(self, key):
        if self._switching:  # ✓ Debounce
            return
        self._switching = True
        
        try:
            if self.current_pipeline:
                print(f"\rStopping {self.STREAMS[self.current_key]['name']}...")
                self.current_pipeline.set_state(Gst.State.NULL)
                self.current_pipeline = None
                time.sleep(0.2)
            
            self.current_key = key
            cfg = self.STREAMS[key]
            print(f"\rStarting {cfg['name']} on port {cfg['port']}...")
            
            p = Gst.parse_launch(self._pipeline_str(key))
            if not p:
                print("\rFailed to create pipeline")
                return
            
            # ✓ Add bus watch for errors
            bus = p.get_bus()
            bus.add_signal_watch()
            bus.connect("message::error", self._on_error)
            
            self.current_pipeline = p
            ret = self.current_pipeline.set_state(Gst.State.PLAYING)
            if ret == Gst.StateChangeReturn.FAILURE:
                print("\rFailed to start pipeline")
                self.current_pipeline = None
                
        except Exception as e:
            print(f"\rPipeline error: {e}")
        finally:
            self._switching = False

    def _on_error(self, bus, message):
        err, debug = message.parse_error()
        print(f"\rGStreamer Error: {err.message}")
        self._alt_held = False  # ✓ Reset state on error

    def toggle(self):
        self.start_stream('2' if self.current_key == '1' else '1')

    def listen_all_inputs(self):
        epoll = select.epoll()
        fds   = {}
        
        ifd = _inotify_init()
        _inotify_add_watch(ifd, self.INPUT_DIR, IN_CREATE | IN_DELETE)
        epoll.register(ifd, select.EPOLLIN)

        def open_device(path):
            if path in fds.values():
                return
            if not os.path.basename(path).startswith('event'):
                return
            try:
                fd = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
                epoll.register(fd, select.EPOLLIN | select.EPOLLHUP | select.EPOLLERR)
                fds[fd] = path
                print(f"\r[+] {path}")
            except OSError as e:  # ✓ Log error
                print(f"\r[!] Cannot open {path}: {e}")

        def close_fd(fd):
            path = fds.pop(fd, 'unknown')
            print(f"\r[-] {path}")
            self._alt_held = False
            try:
                epoll.unregister(fd)
            except Exception as e:  # ✓ Log unregister failures
                print(f"\r[!] epoll.unregister failed: {e}")
            try:
                os.close(fd)
            except:
                pass

        for name in sorted(os.listdir(self.INPUT_DIR)):
            open_device(os.path.join(self.INPUT_DIR, name))
        
        print("Input listener ready (inotify + epoll)")
        
        while True:
            try:
                events = epoll.poll(-1)
            except Exception as e:
                print(f"\r[ERROR] epoll: {e}")
                time.sleep(1)
                continue

            for fd, ev in events:
                if fd == ifd:
                    names = _drain_inotify(ifd)
                    for name in names:
                        path = os.path.join(self.INPUT_DIR, name)
                        if os.path.exists(path):
                            open_device(path)
                        else:
                            for ofd, opath in list(fds.items()):
                                if opath == path:
                                    close_fd(ofd)
                    continue

                if ev & (select.EPOLLHUP | select.EPOLLERR):
                    close_fd(fd)
                    continue

                if ev & select.EPOLLIN:
                    try:
                        # ✓ Drain all available events
                        while True:
                            raw = os.read(fd, self._EVT.size)
                            if len(raw) < self._EVT.size:
                                break
                            _, _, etype, code, value = self._EVT.unpack(raw)
                            
                            if etype == 1:
                                if code in self.TRIGGER_MODIFIER_KEYS:
                                    self._alt_held = (value == 1)
                                elif value == 1:
                                    if code == self.TRIGGER_ACTION_KEY and self._alt_held:
                                        print(f"\r[DEBUG] Alt+Tab detected")
                                        GLib.idle_add(self.toggle)
                                    elif code in self.TRIGGER_MOUSE_CODES:
                                        print(f"\r[DEBUG] mouse trigger code={code}")
                                        GLib.idle_add(self.toggle)
                    except BlockingIOError:  # ✓ Expected when drained
                        pass
                    except OSError:
                        close_fd(fd)

    def run(self):
        hide_cursor()
        old_tty = disable_echo()
        signal.signal(signal.SIGINT, lambda *_: GLib.idle_add(self.loop.quit))
        threading.Thread(target=self.listen_all_inputs, daemon=True).start()
        self.start_stream('1')
        try:
            self.loop.run()
        finally:
            if self.current_pipeline:
                self.current_pipeline.set_state(Gst.State.NULL)
            restore_terminal(old_tty)
            show_cursor()

if __name__ == '__main__':  # ✓ FIXED
    VideoSwitcher().run()

Let me know how it goes.

Best Regards

Suren