AM572x GStreamer v4l2src field capture

Dan Searles

I've looked at the Processor Training:Multimedia page (http : //processors.wiki.ti.com/index.php/Processor_Training:_Multimedia#Gstreamer_Pipelines_for_Multimedia)

but do not see a suitable gstreamer pipeline for my case.

Previous to gstreamer, we used VIP to capture Top/Bottom fields (240/288 lines) from an NTSC/PAL video device, and queued those fields/buffers into a VPE to get full frames (480/576 lines). And in addition to NTSC/PAL, we also have 1080i and 720i HD video devices with similar field to frame processing needed.

Under gstreamer, I am trying to use v4l2src, but do not see a way to properly describe the above pipeline. I believe the difficulty is with trying to describe the fact that the v4l2src is supplying only top/bottom fields and or that VPE should be combining them into frames.

The closest I can get with the capture is:

gst-launch-1.0 -v v4l2src device=/dev/video2 io-mode=4 ! kmssink or

gst-launch-1.0 -v v4l2src device=/dev/video2 io-mode=4 ! vpe ! kmssink

but these show 576 lines on the display, where only the top 240 (NTSC source) are from the video capture - the bottom is all green.

I have also tried adding various forms of 'video/x-raw, format=YUY2, width=720, height=576' to the pipeline but if height is not 480 or 576, I get:

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
../../../../gstreamer-1.8.3/libs/gst/base/gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)

My version of gstreamer is

gst-launch-1.0 version 1.8.3
GStreamer 1.8.3

Linux 4.4.84

Is that the correct version for current TI support?

over 7 years ago

0 Margarita Gashova over 7 years ago

TI__Guru* 80190 points

Hello,

Dan Searles said:
../../../../gstreamer-1.8.3/libs/gst/base/gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)

This error means that the capsfilter is wrong or missing between v4l2src and the next element.

Please check the multimedia wiki again.

gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=1000 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720' ! vpe num-input-buffers=8 ! queue ! kmssink

You could add framerate=(fraction)30/1 for example also.
In the capsfilter this interlace-mode could be set. You could check this link for more information:
https://gstreamer.freedesktop.org/documentation/design/interlaced-video.html




BR

Margarita

0 Dan Searles over 7 years ago in reply to Margarita Gashova

Intellectual 610 points

Hello,

Your response and the wiki gives no examples of use of 'interlace-mode'. The possible settings appear to be 'progressive', 'interleaved' and 'mixed'.

I've tried many permutations of the pipeline and none have worked so far.

Do you have any example of VIP capturing/producing half frame height top/bottom fields, and VPE combining into full frames as previously described, using gstreamer?

0 Margarita Gashova over 7 years ago in reply to Dan Searles

TI__Guru* 80190 points

Hello,

The example above was to check how the caps filter could be set.

However you could try something like:
gst-launch-1.0 -v v4l2src device=/dev/video1 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)720, height=(int)288, framerate=(fraction)30/1' \
! vpe num-input-buffers=8 ! 'video/x-raw, format=(string)NV12, width=(int)720, height=(int)576, framerate=(fraction)30/1' ! queue ! waylandsink

You may also try capturevpedisplay demo.

BR
Margarita

0 Dan Searles over 7 years ago in reply to Margarita Gashova

Intellectual 610 points

Gstreamer does not support the ALTERNATE mode of v4l2, so it is not possible to use gstreamer (without modification) for the described pipeline.
I have been able to get the vpe stage to scale up the fields, but that approach has top/bottom field jitter.

0 Margarita Gashova over 7 years ago in reply to Dan Searles

TI__Guru* 80190 points

Hello,

Actually v4l2src element is not a TI element.
However you could check these threads:
e2e.ti.com/.../618455
e2e.ti.com/.../660340
e2e.ti.com/.../2465762

BR
Margarita

0 Dan Searles over 7 years ago in reply to Margarita Gashova

Intellectual 610 points

Gstreamer 'v4l2src' does not support passing top/bottom field information. It may be possible to patch gstreamer to support this, but at this time it doesn't work.

I have checked the other threads you mention, 618455 is basically the same issue I am reporting, and it does not appear to have actually been resolved. 660340 does not mention gstreamer. 666961 was closed as a duplicate of 660340.

0 Margarita Gashova over 7 years ago in reply to Dan Searles

TI__Guru* 80190 points

Hello,

Dan Searles said:
Gstreamer 'v4l2src' does not support passing top/bottom field information. It may be possible to patch gstreamer to support this, but at this time it doesn't work.

As I said gstreamer v4l2src element is not a TI element.

Dan Searles said:
I have checked the other threads you mention, 618455 is basically the same issue I am reporting, and it does not appear to have actually been resolved. 660340 does not mention gstreamer. 666961 was closed as a duplicate of 660340.

Please check with capturevpedisplay demo do you observe the same problem.

You could try with our latest PSDK release also:

https://e2e.ti.com/support/arm/sitara_arm/f/791/t/677971

BR
Margarita

0 Dan Searles over 7 years ago in reply to Margarita Gashova

Intellectual 610 points

Hello

Problem is not with capturevpedisplay. With capturevpedisplay (or our application) there is better control over the buffers and how they are passed from VIP to VPE.

Issue is with using gstreamer, specifically for the case where video source produces buffers with just one field (top or bottom) at a time - common for NTSC or PAL capture devices, but also possible for HD devices capturing 'i' format video (1080i, 720i, etc). capturevpedisplay handles these easily, but gstreamer appears to lack necessary support. However, even gstreamer uses TI vip.c code via its v4l2 source, and it may be possible that TI vip.c could work around gstreamer limitation... I don't know for sure. It may be also necessary to modify gstreamer code, which is outside the scope of TI support.

Note that some capture devices may produce buffers containing a top and bottom field. That is not the case for this issue. I believe gstreamer has caps/support for this case.

Processors

Processors forum

AM572x GStreamer v4l2src field capture