TDA4VM: Something wrong with gstreamer at receiver side with higher resolution

Hi Michael,

Can you enable more verbose debug logs by appending GST_DEBUG=4 to your gst-launch command, similar to: GST_DEBUG=4 gst-launch-1.0 videotestsrc ! autovideosink, and then share the logs?

I'm assuming something like the "playbin" plugin is being used from the trace.logs you have shared. Plugins that automatically does caps negotiation in the background is hard to predict what the behavior would be for a given platform. My initial suggestion would be to explicitly define which plugins are used instead of using a bin. For example, something like https://software-dl.ti.com/jacinto7/esd/edgeai-sdk-j721e/latest/exports/docs/data_flows.html#rtsp-source

Regards,

Takuma

Hi Takuma,

I use v4l2sink instead of playbin.

Here are some of my reservations from GST_DEBUG=4 log:

I. The debug logs from test case 1. and 2. are pretty much the same at receiver side. 

Test case 1. gst cmd: GST_DEBUG=4 gst-launch-1.0 -e udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG,payload=26 ! \
                                   rtpjpegdepay ! jpegdec ! queue ! videoconvert ! 'video/x-raw, format=NV12' ! autovideosink

disp_1080x1344_d4_pc.log

Test case 2. gst cmd: GST_DEBUG=4 gst-launch-1.0 -e udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG,payload=26 ! \
                                   rtpjpegdepay ! jpegdec ! queue ! videoconvert ! 'video/x-raw, format=NV12' ! v4l2sink device=/dev/video2   

disp_1080x1344_d4.log

II. There are some header warings at resolution of 1080x1344 on teste case 1. and 2. but not at resolution 1080x672.

disp_1080x672_d4.log

Thanks 

Michael

Hi Michael,

I see in the logs for the v4l2sink the following:

0:00:00.185461690  1408     0x2680d2d0 INFO                    v4l2 gstv4l2object.c:3025:gst_v4l2_object_setup_pool:<v4l2sink0:sink> accessing buffers via mode 2
0:00:00.185667485  1408     0x2680d2d0 INFO          v4l2bufferpool gstv4l2bufferpool.c:568:gst_v4l2_buffer_pool_set_config:<v4l2sink0:pool:sink> increasing minimum buffers to 2
0:00:00.185678875  1408     0x2680d2d0 INFO          v4l2bufferpool gstv4l2bufferpool.c:581:gst_v4l2_buffer_pool_set_config:<v4l2sink0:pool:sink> reducing maximum buffers to 32
0:00:00.185742520  1408     0x2680d2d0 INFO          v4l2bufferpool gstv4l2bufferpool.c:592:gst_v4l2_buffer_pool_set_config:<v4l2sink0:pool:sink> can't allocate, setting maximum to minimum
0:00:00.185769750  1408     0x2680d2d0 INFO                v4l2sink gstv4l2sink.c:520:gst_v4l2sink_set_caps:<v4l2sink0> outputting buffers via mode 2
0:00:00.185859535  1408     0x2680d2d0 INFO          v4l2bufferpool gstv4l2bufferpool.c:592:gst_v4l2_buffer_pool_set_config:<v4l2sink0:pool:sink> can't allocate, setting maximum to minimum
0:00:00.196587475  1408     0x2680d320 INFO            videodecoder gstvideodecoder.c:3184:gst_video_decoder_clip_and_push_buf:<jpegdec0> First buffer since flush took 0:00:00.140481455 to produce
0:00:00.196722205  1408     0x2680d2d0 WARN          v4l2bufferpool gstv4l2bufferpool.c:820:gst_v4l2_buffer_pool_start:<v4l2sink0:pool:sink> Uncertain or not enough buffers, enabling copy threshold

I have also tested out the GStreamer pipeline using the following, both running on the same TDA4VM board:

• gst-launch-1.0 videotestsrc ! videoconvert ! video/x-raw, format=NV12, width=1080, height=1344 ! jpegenc ! jpegparse ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000
• gst-launch-1.0 -e udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! queue ! videoconvert ! 'video/x-raw, format=NV12' ! autovideosink

Visually, these pipelines seem to show no issues. So my guess is that there is some issue with using v4l2sink. If lower resolutions are working with v4l2sink, then it may be a limitation on the v4l2 device side or memory.

These are instructions for a camera, but you can check if the v4l2 device supports certain formats by running:

v4l2-ctl -d /dev/video2 --all

And as another quick experiment, you can try to see if changing the io-mode and other buffer related properties of v4l2sink will resolve the issue: https://gstreamer.freedesktop.org/documentation/video4linux2/v4l2sink.html?gi-language=c

Regards,

Takuma

Hi Takuma,

1. Yes, it is as you mention. This issue might be caused by v4l2sink, because the gtreamer pipeline can work properly by replacing v4l2sink with multifilesink.

2. "/dev/video2" is a v4l2loopback device. You can set abitrary resolutions and formats as you wish.

    https://github.com/umlaeute/v4l2loopback

3. The weired thing is that it works normally at resolution of 1080x1344 if video source comes from PC USB webcam instead of TDA4 fpd-link YUV camera. And the same issue, incomplete pipeline, happens at receiver side till 2040x2040.

    I. PC USB webcam (camera resolution: 1080x1344) ---> TDA4

        It works.

    II. TDA4 (camera resolution: 1080x1344) ---> TDA4

        Incomplete gstreamer pipeline at display side

    III. PC USB webcam (camera resolution: 2040x2040) ---> TDA4

        Incomplete gstreamer pipeline at display side

4. I will try to set different properties of v4l2sink like io-mode.

Thanks for help. 

Regards,

Michael

Hi Jay,

Jay Sun said:

1. Yes, it is as you mention. This issue might be caused by v4l2sink, because the gtreamer pipeline can work properly by replacing v4l2sink with multifilesink

Is the pipeline working with multifilesink for both TDA4 fpd-link YUV camera and USB camera? If it is not working for FPD-Link camera, then we may still have some issues on the camera side as well.

Regards,

Takuma

Hi Takuma, 

Both TDA4 fpd-link YUV camera and USB camera work with multifilesink at resolution of 1080x1344. 

The situation did not change by setting different io-mode of v4l2sink. It even gave extra error like "v4l2sink0: Failed to allocated required memory.", but this also happened at lower resolution.

Thanks 

Michael

Hi Michael, 

I am not really familiar with v4l2sink, as we do not use this with any of our demos, so it will take some time on our end to debug this issue. However, could you experiment with fps and resolution using the method documented on the v4l2sink page you linked:https://github.com/umlaeute/v4l2loopback#changing-the-runtime-behaviour

Some experiments you can try are:

• Setting FPS to a very low value, such as 1 FPS - to test if this is a bandwidth issue
• Setting resolution manually to match resolution of camera - I see a warning on the v4l2sink page saying "Please note that GStreamer-style caps (e.g. video/x-raw,format=UYVY,width=640,height=480) or no longer supported!", so it may be that we need to set the resolution manually before running the GStreamer pipeline
• Setting resolution manually to a smaller resolution, and larger resolution than the camera - to test whether the pipeline will break depending on our manually set resolution

Also, since v4l2sink is an open source plugin, you could try searching online to see if there is somebody else having a related issue.

Regards,

Takuma

Hi Takuma, 

• Setting FPS to a very low value, such as 1 FPS - to test if this is a bandwidth issue
  • It is probably not a bandwidth issue since Gstreamer even cannot be executed successfully at 1 FPS.
• Setting resolution manually to match resolution of camera - I see a warning on the v4l2sink page saying "Please note that GStreamer-style caps (e.g. video/x-raw,format=UYVY,width=640,height=480) or no longer supported!", so it may be that we need to set the resolution manually before running the GStreamer pipeline
  • As I understood, this means that the format can be set with v4l2loopback restricted style, v4l2loopback-ctl set-caps /dev/video0 "UYVY:640x480", instead of Gstreamer style. Furthermore,  those parameters are set with ioctl.
• Setting resolution manually to a smaller resolution, and larger resolution than the camera - to test whether the pipeline will break depending on our manually set resolution
  • The smaller resolution than the camera(1280x720) was tested and it worked.

Since CSI is supported at Linux SDK v8.4, I am trying to implement those application at Linux side. And v4l2sink would no longer be needed. 

If you have a minute, please take a look at my new thread.

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1162037/tda4vm-some-errors-when-using-imx390-from-otobrite-on-linux?tisearch=e2e-sitesearch&keymatch=%20user%3A487798#

Thanks

Michael