TDA4VM-Q1: tda4vm: encode: when run encode function, block at appGstDeqAppSrc()

My SDK is 8.4 linux

Hi Tahm, 

I will take a look at this and try to replicate. Do you have a raw stream file I can test with?

Thanks,
Sarabesh S.

Hi, Sarabesh, 

Thanks for your reply.

Do you mean the file I use to test ？ it was a picture, the format is UYVY 720p

 test_1280x720.uyvy.txt

Regards,

Tahm

Hi, SaraBesh,

Hi, Sarabesh,

Now I can save the data as an H.264 file by modifying the GStreamer command line. Thank you very much for your attention. If I want to send the encoded data out via the RTP protocol, how should I modify the command? Alternatively, can I access the encoded data in the code?

PS:
   Is the error in the picture normal?

BRs.

Tahm

Hello Tahm, 

If you write to a file, is the output h264 file valid when you open it? Are you trying to stream the encoded data onto your host machine through RTP? 

Thanks,
Sarabesh S.

Hi, Sarabesh,

The H.264 file saved by GStreamer opens correctly. The file is attached below. I haven't tried sending the encoded data via RTP, and I'm not sure how to modify the code to make GStreamer send the encoded data via RTP.

output_video_0.h264.txt

BRs,

Tahm

Hi Tahm, 

When I analyzed the stream with one of our tools, I see some unexpected corrupted frames in the first 4 frames. Could you provide me the input .yuv file to test?

xie jc said:

I haven't tried sending the encoded data via RTP, and I'm not sure how to modify the code to make GStreamer send the encoded data via RTP.

I have an FAQ page that goes over streaming encoded data via RTP through GStreamer from a camera input. You can use this as reference (FAQ). Based on this I believe the GStreamer pipeline would look something like this:

gst-launch-1.0 filesrc location=/bbb_1080p30_short.yuv ! rawvideoparse width=1920 height=1080 format=nv12 framerate=30/1 colorimetry=bt709 ! video/x-raw,format=NV12 ! videoconvert ! v4l2h264enc extra-controls="enc,video_gop_size=5" ! rtph264pay ! udpsink host=<yourip> port=<sdp port specified>

Keep in mind, to have this play on the host machine, you need to create a .sdp file. The FAQ goes over instructions to do this.

Thank You,
Sarabesh S.

Hi, Sarabesh,

this is the input file I attached before.  I can not upload the file with suffix *.yuyv, so I add .txt in the file name. 

BRs，

Tahm

you can check this file

xie jc said:

Hi, Sarabesh, 

Thanks for your reply.

Do you mean the file I use to test ？ it was a picture, the format is UYVY 720p

 test_1280x720.uyvy.txt

Regards,

Tahm

Hi Tahm, 

So after looking into it, our software driver does not have support for encoding YUV422 (HERE). Did you you some type of converter to successfully encode the output_video_0.h264 file you shared?

GStreamer File Source RTP Streaming:

Additionally, if you are trying to RTP stream to your host machine from a filesrc input, the following pipeline should work:

• gst-launch-1.0 filesrc location=<input_file_path> ! rawvideoparse width=1920 height=1080 format=nv12 framerate=30/1 colorimetry=bt709 ! video/x-raw,format=NV12 ! videoconvert ! v4l2h264enc extra-controls="enc,video_gop_size=5" ! rtph264pay ! udpsink host=<yourip> port=<sdp port specified>

However, the host machine will not pick up the header information in the initial packets sent so you will need to enable IDR frames for every frame to ensure header information will get picked up when RTP streaming. This can be done with the following changes to the Linux installation directory board-support/linux-5.10.162+gitAUTOINC+76b3e88d56-g76b3e88d56/drivers/media/platform/vxe-vxd/encoder/vxe_v4l2.c: 

• In the function vxe_s_ext_ctrls under the case V4L2_CID_MPEG_VIDEO_GOP_SIZE setting ctx->vparams.idr_period = 1; and under the case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD setting ctx->rc.intra_freq = 1; and ctx->vparams.intra_cnt = 1; will set every frame to be an IDR frame.

Furthermore, as I mentioned in the previous response, instructions for the setup and commands (i.e. creating .sdp file and ffmpeg command) to RTP stream onto your host machine are linked HERE in the second section. 

Regards,
Sarabesh S.

Hi, Sarabesh,

Sorry for the delayed response. Previously, I was able to perform encoding on the EVM board, but when running on our own board, there were issues. After several days of troubleshooting, I found that it was caused by previous optimization during trimming.

Sarabesh Srinivasan said:

So after looking into it, our software driver does not have support for encoding YUV422 (HERE). Did you you some type of converter to successfully encode the output_video_0.h264 file you shared?

my video chain is  reay yuv422 picture -> ldc convert to nv12 -> msc -> encode

Sarabesh Srinivasan said:

GStreamer File Source RTP Streaming:

Additionally, if you are trying to RTP stream to your host machine from a filesrc input, the following pipeline should work:

• gst-launch-1.0 filesrc location=<input_file_path> ! rawvideoparse width=1920 height=1080 format=nv12 framerate=30/1 colorimetry=bt709 ! video/x-raw,format=NV12 ! videoconvert ! v4l2h264enc extra-controls="enc,video_gop_size=5" ! rtph264pay ! udpsink host=<yourip> port=<sdp port specified>

However, the host machine will not pick up the header information in the initial packets sent so you will need to enable IDR frames for every frame to ensure header information will get picked up when RTP streaming. This can be done with the following changes to the Linux installation directory board-support/linux-5.10.162+gitAUTOINC+76b3e88d56-g76b3e88d56/drivers/media/platform/vxe-vxd/encoder/vxe_v4l2.c: 

• In the function vxe_s_ext_ctrls under the case V4L2_CID_MPEG_VIDEO_GOP_SIZE setting ctx->vparams.idr_period = 1; and under the case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD setting ctx->rc.intra_freq = 1; and ctx->vparams.intra_cnt = 1; will set every frame to be an IDR frame.

Do you mean that after the modification, every frame encoded as h.264 is an IDR frame? Or are you saying that each frame contains one IDR frame?

BRs，

Tahm

Hi, Sarabesh,

The file i dump using tcpdump is:

test-1.pcap.txt

BRs,

Tahm

xie jc said:

In the function vxe_s_ext_ctrls under the case V4L2_CID_MPEG_VIDEO_GOP_SIZE setting ctx->vparams.idr_period = 1; and under the case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD setting ctx->rc.intra_freq = 1; and ctx->vparams.intra_cnt = 1; will set every frame to be an IDR frame.

Hi, Sarabesh, after I set all this variables to 30, i dump the file:

test-2.pcap.txt

The parameter "extra-controls="enc,video_gop_size=5"" within the gstreamer command seems to be ineffective. Is this normal?

BRs,

Regards.

Hi Xie,

Please note that Sarabesh is not available this week, so please expect a delay in response on this thread.

Sarabesh will get back to you on this thread next week.

regards

Suman

Hi, Suman,

Thank you！

Sure, no issues.

Hi Tahm, 

xie jc said:

Do you mean that after the modification, every frame encoded as h.264 is an IDR frame? Or are you saying that each frame contains one IDR frame?

Enabling this will encode every frame as an IDR frame. This will ensure header information for the stream is picked up through the RTP streaming. 

xie jc said:

The parameter "extra-controls="enc,video_gop_size=5"" within the gstreamer command seems to be ineffective. Is this normal?

Instead of the parameter extra-controls="enc,video_gop_size=5" could you try it with extra-controls='controls,video_gop_size=30,h264_profile=0' 

Regards,
Sarabesh S.