AM62A7: AM62A SDK 10.05 - H.264 Dual Stream Decode+Encode Issue

Hi Yeoncheon Yi,

Can you share the logs you see when the decoder hangs/stops processing? 

Does the issue occur only with 1920x1080@30fps streams. Can you try reducing the resolution to 1280x720 on one of them and try once, while I reproduce this issue on my side.

There are no known issues on decoder side and we should be able to decode multiple decode streams without any artifacts/performance issues.

You are using 10.01.00.05 SDK released software to test, correct?

Best Regards,

Suren

I am currently testing video decoding using GStreamer on the TI SDK (version 10.01.00.05), and I'm encountering an issue where the decoder pipeline stops unexpectedly after some time.

Issue Summary

• Setup: Two decoding pipelines are running simultaneously.

• Resolution: One is 1920x1080, and the other is either 640x360 or 640x480.

• Problem: One of the pipelines stops unexpectedly, usually within a few minutes, and never longer than an hour. The other pipeline continues running.

• After the issue: Restarting the failed pipeline does not restore normal operation.

• On Ctrl+C: Sometimes, when stopping with Ctrl+C, the GStreamer process does not terminate cleanly and remains as a zombie process.

• GStreamer logs: When increasing the GStreamer debug log level, video playback does not function properly, and no meaningful logs are shown.

• Kernel logs: After the pipeline stalls, the following kernel messages appear:

Because of these logs, I suspect the issue is related to the decoder (wave5 VPU).

Questions

• Can you explain what the message result not ready: 0x800 and could not get output info actually mean?

• Is this a known issue with the decoder or firmware?

• Are there any recommended workarounds or fixes to avoid this behavior?

Thank you in advance for your help.

Hello,

We are currently testing with the official TI evaluation board purchased directly from the TI website, using the software image provided at:

 https://www.ti.com/tool/PROCESSOR-SDK-AM62A

The version in use is 10.01.00.05, and we flashed the SD card image as provided without modification.

Test Setup

• Input streams:

  • Port 5000: H.264 encoded stream at 1920x1080 @ 30fps

  • Port 6000: H.264 encoded stream at 640x360

• Both streams are received using GStreamer and then re-transmitted to remote endpoints using separate pipelines.

• Initially, both streams are received, decoded, and transmitted correctly. The remote receivers using GStreamer can also display the streams without any issue.

Problem

After a random period—sometimes as short as 2–3 minutes, and at most within an hour—one of the two pipelines stops transmitting. The failure is not consistent; either pipeline can stop randomly.

While both are running normally, each GStreamer pipeline consumes about 5% CPU as seen in top. However, once one of them stops, its CPU usage drops to 0%, indicating it's no longer active.

After the Failure

If we press Ctrl-C, one of the pipelines exits and prints the following logs:

[   43.753540] audit: type=1334 audit(1753926992.689:21): prog-id=20 op=LOAD
[   43.901166] audit: type=1334 audit(1753926992.837:22): prog-id=20 op=UNLOAD
[  682.139715] vdec 30210000.video-codec: wave5_vpu_firmware_command_queue_error_check: result not ready: 0x800
[  682.149678] vdec 30210000.video-codec: wave5_vpu_firmware_command_queue_error_check: result not ready: 0x800
[  682.159515] vdec 30210000.video-codec: wave5_vpu_dec_finish_decode: could not get output info.
[  682.159658] vdec 30210000.video-codec: wave5_vpu_firmware_command_queue_error_check: result not ready: 0x800

The other pipeline does not exit at all, and remains in a zombie-like state.

Restart Behavior

If we attempt to restart the failed pipeline using the following command:

gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp, media=video, clockrate=90000, encoding-name=H264, payload=96" ! rtph264depay ! h264parse ! queue ! v4l2h264dec capture-io-mode=4 ! queue ! v4l2h264enc output-io-mode=5 extra-controls="controls,h264_i_frame_period=60,video_gop_size=60" ! h264parse config-interval=1 ! rtph264pay pt=96 ! udpsink host=192.168.123.180 port=7000

—there is no error, no log message, and no video is transmitted. The pipeline simply appears stuck and unresponsive.

Summary

• The issue appears to be related to the vdec (wave5 VPU) decoder block.

• The error result not ready: 0x800 and could not get output info appear only after the stream stops.

• We suspect a decoder hang or firmware queue deadlock but would appreciate any insights.

Questions

• Is this a known issue with the wave5 decoder in this SDK version?

• Are there any workarounds or patches available?

• How can we recover the decoder pipeline without a full system reboot?

Thank you for your assistance.

Hi Yi YeonCheon,

Please have this tested with the upcoming SDK release. It is scheduled to be released in 2 weeks. That has the fixes for decoder. 

Best Regards,

Suren

Hi Suren,

Thank you for the update.

I’ve already tested with the latest SDK version 11.01.07.05, which appears to have been recently released. Unfortunately, the same decoder issue still persists with this version.

Could you please confirm if the fix you mentioned will be included in an upcoming SDK release scheduled within the next two weeks? I’d like to ensure that this release will indeed address the decoder issue.

Best regards,
Yi YeonCheon

Hi Yi YeonCheon,

I ran the command on my host PC:

gst-launch-1.0 -v videotestsrc ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! x264enc ! queue ! rtph264pay ! tee name=t t. ! queue ! udpsink host=128.247.75.79 port=5000 t. ! queue ! udpsink host=128.247.75.79 port=6000

Target EVM :

gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtpjitterbuffer latency=50 ! rtph264depay ! h264parse ! v4l2h264dec ! kmssink driver-name=tidss sync=false

gst-launch-1.0 -v udpsrc port=6000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtpjitterbuffer latency=50 ! rtph264depay ! h264parse ! v4l2h264dec ! kmssink driver-name=tidss sync=false

For more than an hour or so,  I haven't observed the hang or stuck issue. Although sometimes the second stream appears on my display with some latency.

I will try and run on 11.1 SDK that has already been released. You should not see those warnings any more as the commit is already part of 11.1 SDK

https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/commit/drivers/media/platform/chips-media/wave5?h=ti-linux-6.12.y&id=eb39ef7d05eb995c941a74cb627f3a72b7458f61

Best Regards,

Suren

Dear Suren

It seems that your test was conducted by splitting the same video stream into two branches using tee, and in that case, the issue does not occur.
However, to reproduce the problem, you need to test with two completely different video streams, each sent to a separate port.

While I’m not certain of the root cause, I also confirmed that when using the same stream on both ports, everything worked fine. But when two entirely different streams were used, the issue occurred almost immediately.

Also, please make sure to add output-io-mode=5 to the encoder settings.
This ensures that the test fully utilizes the hardware encoder, which is essential for accurate validation.

Would you be able to try it that way?

BRs

Yeoncheon Yi

n my case, I tested with decoding followed by re-encoding, so I had to add output-io-mode=5.
But since you're using kmssink directly after decoding, you probably don't need output-io-mode in your setup.

gst-launch-1.0 -v \
udpsrc port=5000 buffer-size=524288 caps="application/x-rtp, media=video, clockrate=90000, encoding-name=H264, payload=96" ! \
rtph264depay ! h264parse ! queue ! \
v4l2h264dec capture-io-mode=4 ! \
queue max-size-buffers=4 leaky=1 ! v4l2h264enc output-io-mode=5 extra-controls="controls,h264_i_frame_period=30,video_gop_size=30" ! \
h264parse config-interval=1 ! rtph264pay pt=96 ! \
udpsink host=192.168.123.180 port=7000

gst-launch-1.0 -v \
udpsrc port=6000 buffer-size=524288 caps="application/x-rtp, media=video, clockrate=90000, encoding-name=H264, payload=96" ! \
rtph264depay ! h264parse ! queue ! \
v4l2h264dec capture-io-mode=4 ! \
queue max-size-buffers=4 leaky=1 ! v4l2h264enc output-io-mode=5 extra-controls="controls,h264_i_frame_period=30,video_gop_size=30" ! \
h264parse config-interval=1 ! rtph264pay pt=96 ! \
udpsink host=192.168.123.180 port=8000

I'v been test above pipeline.

Thanks

Hi Yi YeonCheon,

I verified on 11.1 SDK software and didn't see the hang or stuck issue on decoder. Also upon stopping the pipelines, I no longer see those warnings.

Attached are my scripts that I run from my host PC and on target EVM.

Host PC: 

./send.sh (modify the host IP address in the script to point to your target EVM).

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/send.sh

./receive.sh

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/receive.sh

Target EVM:

./multistream.sh <host PC IP addr>

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/multistream.sh

I didn't see any issues for about an hour run.

Best Regards,

Suren

Dear Suren

First of all, thank you for your test and support.
I have tested using the script you provided.

When using test patterns or video with moving balls, no issue occurred.
Even when I created RTP streams from MP4 files and sent two different streams over different UDP ports, the decoding worked fine without any problem.

However, our actual use case involves receiving real-time video from cameras and re-transmitting it. So we tested with real camera inputs.
We configured two cameras to output H.264, 1080p, 30fps via UDP. When testing with a single camera input, there was no problem. But once the second camera stream was added, both decoders stopped working within a few minutes.

Unlike MP4 or test pattern inputs, we suspect that various factors from the live camera input could be affecting the decoder.
The issue is not limited to a specific camera model — we observed the same problem across different camera brands, all configured identically (H.264/1080p/30fps).

If possible, could you try testing with actual camera input as well?
For our test, we used IP cameras via RTSP, then re-encoded and sent the streams to the AM62A device via UDP.

Thank you again for your support.

Best regards,

Yeoncheon 

Hi Yeoncheon,

I have tried with 2 camera inputs as well and didn't see any issue.

Here is my setup.

One AM62A board (Board A) (which has 4 cameras connected via FPD Link  all 1080p @30ps), I capture the camera input and encode using H.264 encoder and stream it via UDP to another AM62A board (Board B).

On Board B (AM62A), I run the multistream.sh script (Which is decoding the incoming UDP streams from Board A) and encode using H.264 encoder and stream it my host PC via UDP.

On my Host PC. I run the receive.sh script that I attached yesterday and I don't see any crashes after running it for more than 3 hours.

I assume something is wrong with your setup or network which might be causing the issues.

Also, if there is a decoder crash/ decoder stopped working, Gstreamer log would have captured it.

Best Regards,

Suren

Hello Suren.

They're experiencing decoder errors and video freezing when handling live camera streaming on our AM62A setup.

From their observations, potential causes include network delay, packet loss over UDP, buffering issues, and timing/synchronization errors between streams. These issues appear only with live camera inputs and not with test patterns or MP4-based streams.

Could you please provide guidance or recommend best practices to improve the stability of multi-camera live streaming in this environment?

We would appreciate any suggestions on network handling, buffering parameters, and error recovery settings for the decoder.

I menat If one channel’s stream is delayed due to network traffic and the decoder times out, is there a recovery mechanism implemented?

It seems that the issue cannot be reproduced in your environment at the moment. Would it be possible to test by artificially triggering the issue by adjusting the timeout value or buffering size?

Regards,

Jack

Hi Jack,

In my previous run, I am using camera inputs and it ran without any issues in an overnight run.

Can you check your network bandwidth and understand why you are having issues. Even if the network bandwidth is low, I would have assumed you would have a lot of frame drops but decoder wouldn't crash. Did the gstreamer logs specific to video decoder point to any issues in your run?

Best Regards,

Suren

Hi Suren

I have spent considerable time trying to play two streams simultaneously on SDK 11.x, but the result has been unsuccessful. There are no network issues or misconfigurations. I tested with commercial cameras, connected directly to a network switch and then to the EVM, so there is no packet loss.

The camera resolutions are 1920×1080 and 640×360. The lower-resolution stream consistently stops.

I tried many approaches to resolve this, and ultimately confirmed that streaming works correctly on SDK 9.02. However, SDK 9.02 has a critical issue: when decoding and re-encoding 1920×1080, the top 8 pixels of the output video appear corrupted. This issue does not occur on 10.xx or 11.xx.

If there is a patch to address this problem on SDK 9.02, please let me know.

Thanks

BRs

Yeoncheon Yi

Hi,

Since you are not seeing the issues on 10.x or 11.x, can you consume the Wave5 driver from 10.x or 11.x?

Best Regards,

Suren

Dear Suren

The 10.xx and 11.xx versions are not compatible with the 9.xx version because the kernel versions differ, making the wave driver incompatible, and the wave firmware versions are also different, so they cannot be used.

Thanks

BRs

Yeoncheon Yi

Hi Yeoncheon,

Please share the exact pipeline you are using? 

Best Regards,

Suren

Hi Suren

This is our pipeline.

First Stream : 1920x1080(30fps)
gst-launch-1.0 -e \
rtspsrc location=rtsp://admin:quopin1234@192.168.123.202/Streaming/Channels/101 latency=200 protocols=tcp ! queue ! \
rtph264depay ! h264parse config-interval=-1 ! \
v4l2h264dec capture-io-mode=4 ! \
v4l2h264enc output-io-mode=5 \
extra-controls='controls,video_bitrate_mode=0,video_bitrate=2000000,vbv_buffer_size=3000,frame_level_rate_control_enable=1,video_gop_size=30,h264_mb_level_rate_control=1' ! \
h264parse config-interval=1 ! rtph264pay pt=96 config-interval=1 ! \
udpsink host=192.168.123.180 port=8000 sync=false

Second Stream : 640x360(30fps)

gst-launch-1.0 -e \
rtspsrc location=rtsp://admin:quopin1234@192.168.123.202/Streaming/Channels/102 latency=200 protocols=tcp ! queue ! \
rtph264depay ! h264parse config-interval=-1 ! \
v4l2h264dec capture-io-mode=4 ! \
v4l2h264enc output-io-mode=5 \
extra-controls='controls,video_bitrate_mode=0,video_bitrate=2000000,vbv_buffer_size=3000,frame_level_rate_control_enable=1,video_gop_size=30,h264_mb_level_rate_control=1' ! \
h264parse config-interval=1 ! rtph264pay pt=96 config-interval=1 ! \
udpsink host=192.168.123.180 port=8000 sync=false

“If necessary, we can open an RTSP server so that you can access the two streams. If you can specify the exact time, I can set up an environment where you can test the streams for about 12 hours. Would that be possible?”

Thanks 

BRs

YeoncheonYi

Dear Suren

I have made it accessible for you.
I would greatly appreciate it if you could test using the link below:

Frist one :

rtsp://admin:quopin1234@115.94.118.69/Streaming/Channels/101

Second one :

rtsp://admin:quopin1234@115.94.118.69/Streaming/Channels/102

Please test it on version 11.xx

look forward to positive results. Thank you very much

BRs Yeoncheon Yi