• State
  • Locked Locked
  • Replies 8 replies
  • Subscribers 101 subscribers
  • Views 988 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Gstreamer Pipeline on custom 816x "Could not negotiate format"

Christian Barth
Intellectual 290 points
Other Parts Discussed in Thread: TVP7002

Hi,


I have problems to capture an video with gstreamer on a v4l2-source.

The pipeline is working, because i can capture raw-images with the v4l2-api. But this is not very performant.

Convert the videotestsource with gstreamer to h264 is working too. So there is no problem with gstreamer principally.

Our HW-Pipline: 8168_[VIP0] <-- FPGA <-- ADV7611 (HDMI_IN)

- I have no TVP7002 and comment the subdevice calls in the v4l2 kernel sources. This works with capturing raw images.

- To configure the FPGA we use our own SW. Start/Stop Capturing, Convert Colorspace, etc.

My simple pipeline:  GST_DEBUG=3 gst-launch -v v4l2src device=/dev/video0 ! fakesink

Errorlog:

8726.gstreamer_error_001.txt 
GST_DEBUG=3 gst-launch -v v4l2src device=/dev/video0 ! fakesink
0:00:00.000581074  1193    0x16050 INFO                GST_INIT gst.c:605:init_pre: Initializing GStreamer Core Library version 0.10.32
0:00:00.000824851  1193    0x16050 INFO                GST_INIT gst.c:606:init_pre: Using library installed in /usr/lib
0:00:00.000914814  1193    0x16050 INFO                GST_INIT gst.c:616:init_pre: Linux basicCON4121 2.6.37+ #2 Tue Apr 15 08:50:35 CEST 2014 armv7l
0:00:00.001488703  1193    0x16050 INFO                GST_INIT gstquery.c:108:_gst_query_initialize: init queries
0:00:00.006331259  1193    0x16050 INFO                GST_INIT gstmessage.c:73:_gst_message_initialize: init messages
0:00:00.007553296  1193    0x16050 INFO      GST_PLUGIN_LOADING gstplugin.c:350:_gst_plugin_initialize: registering 0 static plugins
0:00:00.008369481  1193    0x16050 INFO      GST_PLUGIN_LOADING gstplugin.c:255:gst_plugin_register_static: registered static plugin "staticelements"
0:00:00.008492037  1193    0x16050 INFO      GST_PLUGIN_LOADING gstplugin.c:257:gst_plugin_register_static: added static plugin "staticelements", result: 1
0:00:00.013792185  1193    0x16050 INFO            GST_REGISTRY gstregistry.c:1599:ensure_current_registry: reading registry cache: /home/root/.gstreamer-0.10/registry.arm.bin
0:00:00.173844445  1193    0x16050 INFO            GST_REGISTRY gstregistrybinary.c:614:gst_registry_binary_read_cache: loaded /home/root/.gstreamer-0.10/registry.arm.bin in 0.1599                                               13 seconds
0:00:00.174191112  1193    0x16050 INFO            GST_REGISTRY gstregistry.c:1469:scan_and_update_registry: Validating plugins from registry cache: /home/root/.gstreamer-0.10/regi                                               stry.arm.bin
0:00:00.269032519  1193    0x16050 INFO            GST_REGISTRY gstregistry.c:1561:scan_and_update_registry: Registry cache has not changed
0:00:00.269145000  1193    0x16050 INFO            GST_REGISTRY gstregistry.c:1628:ensure_current_registry: registry reading and updating done, result = 1
0:00:00.269196185  1193    0x16050 INFO                GST_INIT gst.c:792:init_post: GLib runtime version: 2.24.1
0:00:00.269247074  1193    0x16050 INFO                GST_INIT gst.c:794:init_post: GLib headers version: 2.24.1
0:00:00.269467111  1193    0x16050 INFO            GST_PIPELINE gstparse.c:299:gst_parse_launch_full: parsing pipeline description 'v4l2src device=/dev/video0 ! fakesink '
0:00:00.419349296  1193    0x16050 INFO      GST_PLUGIN_LOADING gstplugin.c:859:gst_plugin_load_file: plugin "/usr/lib/gstreamer-0.10/libgstvideo4linux2.so" loaded
0:00:00.419473926  1193    0x16050 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:371:gst_element_factory_create: creating element "v4l2src"
0:00:00.423449186  1193    0x16050 INFO        GST_ELEMENT_PADS gstelement.c:716:gst_element_add_pad:<GstBaseSrc@0x13c060> adding pad 'src'
0:00:00.488938519  1193    0x16050 INFO      GST_PLUGIN_LOADING gstplugin.c:859:gst_plugin_load_file: plugin "/usr/lib/gstreamer-0.10/libgstcoreelements.so" loaded
0:00:00.489063815  1193    0x16050 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:371:gst_element_factory_create: creating element "fakesink"
0:00:00.490085185  1193    0x16050 INFO        GST_ELEMENT_PADS gstelement.c:716:gst_element_add_pad:<GstBaseSink@0x142098> adding pad 'sink'
0:00:00.490357445  1193    0x16050 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:371:gst_element_factory_create: creating element "pipeline"
0:00:00.490869556  1193    0x16050 INFO            GST_PIPELINE ./grammar.y:568:gst_parse_perform_link: linking v4l2src0:(any) to fakesink0:(any) (0/0) with caps "(nil)"
0:00:00.490976185  1193    0x16050 INFO        GST_ELEMENT_PADS gstutils.c:1568:gst_element_link_pads_full: trying to link element v4l2src0:(any) to element fakesink0:(any)
0:00:00.491059778  1193    0x16050 INFO                GST_PADS gstutils.c:1028:gst_pad_check_link: trying to link v4l2src0:src and fakesink0:sink
0:00:00.491866333  1193    0x16050 INFO                GST_PADS gstutils.c:1468:prepare_link_maybe_ghosting: v4l2src0 and fakesink0 in same bin, no need for ghost pads
0:00:00.492400149  1193    0x16050 INFO                GST_PADS gstpad.c:1941:gst_pad_link_prepare: trying to link v4l2src0:src and fakesink0:sink
0:00:00.493133334  1193    0x16050 INFO                GST_PADS gstpad.c:2124:gst_pad_link_full: linked v4l2src0:src and fakesink0:sink, successful
Setting pipeline to PAUSED ...
0:00:00.493800815  1193    0x16050 INFO              GST_STATES gstelement.c:2341:gst_element_continue_state:<fakesink0> completed state change to READY
0:00:00.493877704  1193    0x16050 INFO              GST_STATES gstelement.c:2354:gst_element_continue_state:<fakesink0> posting state-changed NULL to READY
0:00:00.494065519  1193    0x16050 INFO              GST_STATES gstbin.c:2492:gst_bin_change_state_func:<pipeline0> child 'fakesink0' changed state to 2(READY) successfully
0:00:00.494508075  1193    0x16050 INFO                    v4l2 v4l2_calls.c:480:gst_v4l2_open:<v4l2src0> Opened device 'TI81xx_catalogue' (/dev/video0) successfully
0:00:00.494716519  1193    0x16050 INFO              GST_STATES gstelement.c:2341:gst_element_continue_state:<v4l2src0> completed state change to READY
0:00:00.494783556  1193    0x16050 INFO              GST_STATES gstelement.c:2354:gst_element_continue_state:<v4l2src0> posting state-changed NULL to READY
0:00:00.494877519  1193    0x16050 INFO              GST_STATES gstbin.c:2492:gst_bin_change_state_func:<pipeline0> child 'v4l2src0' changed state to 2(READY) successfully
0:00:00.494986963  1193    0x16050 INFO              GST_STATES gstelement.c:2314:gst_element_continue_state:<pipeline0> committing state from NULL to READY, pending PAUSED, next P                                               AUSED
0:00:00.496433334  1193    0x16050 INFO              GST_STATES gstelement.c:2323:gst_element_continue_state:<pipeline0> continue state change READY to PAUSED, final PAUSED
0:00:00.496753186  1193    0x16050 INFO              GST_STATES gstbin.c:2498:gst_bin_change_state_func:<pipeline0> child 'fakesink0' is changing state asynchronously to PAUSED
0:00:00.496954778  1193    0x16050 INFO                    v4l2 gstv4l2object.c:1036:gst_v4l2_object_fill_format_list:<v4l2src0> got 4 format(s):
0:00:00.497015482  1193    0x16050 INFO                    v4l2 gstv4l2object.c:1042:gst_v4l2_object_fill_format_list:<v4l2src0>   NV12
0:00:00.497062778  1193    0x16050 INFO                    v4l2 gstv4l2object.c:1042:gst_v4l2_object_fill_format_list:<v4l2src0>   YUYV
0:00:00.497106445  1193    0x16050 INFO                    v4l2 gstv4l2object.c:1042:gst_v4l2_object_fill_format_list:<v4l2src0>   RGB3
0:00:00.497149371  1193    0x16050 INFO                    v4l2 gstv4l2object.c:1042:gst_v4l2_object_fill_format_list:<v4l2src0>   NV16
0:00:00.497555148  1193    0x16050 INFO                 v4l2src gstv4l2src.c:611:gst_v4l2src_get_caps:<v4l2src0> probed caps: 0xac4a0

 Mode set is 0
0:00:00.497920482  1193    0x16050 INFO                    v4l2 gstv4l2object.c:1525:gst_v4l2_object_get_caps_info:<v4l2src0> rowstride: 32768
0:00:00.497995334  1193    0x16050 WARN                 basesrc gstbasesrc.c:2778:gst_base_src_start:<v4l2src0> error: Could not negotiate format
0:00:00.498045667  1193    0x16050 WARN                 basesrc gstbasesrc.c:2778:gst_base_src_start:<v4l2src0> error: Check your filtered caps, if any
0:00:00.498162111  1193    0x16050 INFO        GST_ERROR_SYSTEM gstelement.c:1865:gst_element_message_full:<v4l2src0> posting message: Could not negotiate format
0:00:00.498274630  1193    0x16050 INFO        GST_ERROR_SYSTEM gstelement.c:1888:gst_element_message_full:<v4l2src0> posted error message: Could not negotiate format
0:00:00.498341815  1193    0x16050 WARN                 basesrc gstbasesrc.c:2974:gst_base_src_activate_push:<v4l2src0> Failed to start in push mode
0:00:00.498389741  1193    0x16050 INFO                GST_PADS gstpad.c:983:gst_pad_activate_push:<v4l2src0:src> failed to activate in push mode
0:00:00.498445445  1193    0x16050 WARN                GST_PADS gstpad.c:734:gst_pad_set_active:<v4l2src0:src> Failed to activate pad
0:00:00.498511519  1193    0x16050 INFO              GST_STATES gstelement.c:2658:gst_element_change_state:<v4l2src0> have FAILURE change_state return
0:00:00.498566519  1193    0x16050 INFO              GST_STATES gstelement.c:2235:gst_element_abort_state:<v4l2src0> aborting state from READY to PAUSED
0:00:00.498621111  1193    0x16050 INFO              GST_STATES gstbin.c:2508:gst_bin_change_state_func:<pipeline0> child 'v4l2src0' failed to go to state 3(PAUSED)
0:00:00.498743852  1193    0x16050 INFO              GST_STATES gstelement.c:2658:gst_element_change_state:<pipeline0> have FAILURE change_state return
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not negotiate format
Additional debug info:
gstbasesrc.c(2778): gst_base_src_start (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Check your filtered caps, if any
Setting pipeline to NULL ...
0:00:00.503326186  1193    0x16050 INFO              GST_STATES gstelement.c:2314:gst_element_continue_state:<fakesink0> committing state from READY to READY, pending NULL, next NU                                               LL
0:00:00.503439260  1193    0x16050 INFO              GST_STATES gstelement.c:2323:gst_element_continue_state:<fakesink0> continue state change READY to NULL, final NULL
0:00:00.503524852  1193    0x16050 INFO              GST_STATES gstelement.c:2341:gst_element_continue_state:<fakesink0> completed state change to NULL
0:00:00.503579667  1193    0x16050 INFO              GST_STATES gstelement.c:2354:gst_element_continue_state:<fakesink0> posting state-changed READY to NULL
0:00:00.503657556  1193    0x16050 INFO              GST_STATES gstbin.c:2492:gst_bin_change_state_func:<pipeline0> child 'fakesink0' changed state to 1(NULL) successfully
0:00:00.503842852  1193    0x16050 INFO              GST_STATES gstelement.c:2341:gst_element_continue_state:<v4l2src0> completed state change to NULL
0:00:00.503970963  1193    0x16050 INFO              GST_STATES gstelement.c:2354:gst_element_continue_state:<v4l2src0> posting state-changed READY to NULL
0:00:00.504057186  1193    0x16050 INFO              GST_STATES gstbin.c:2492:gst_bin_change_state_func:<pipeline0> child 'v4l2src0' changed state to 1(NULL) successfully
0:00:00.504186408  1193    0x16050 INFO              GST_STATES gstelement.c:2341:gst_element_continue_state:<pipeline0> completed state change to NULL
0:00:00.504242778  1193    0x16050 INFO              GST_STATES gstelement.c:2354:gst_element_continue_state:<pipeline0> posting state-changed READY to NULL
0:00:00.504328482  1193    0x16050 INFO              GST_STATES gstbin.c:1759:gst_bin_get_state_func:<pipeline0> getting state
Freeing pipeline ...
0:00:00.504492371  1193    0x16050 INFO        GST_ELEMENT_PADS gstpad.c:1713:gst_pad_unlink: unlinking v4l2src0:src(0x13e000) and fakesink0:sink(0x13e0c8)
0:00:00.504608926  1193    0x16050 INFO        GST_ELEMENT_PADS gstpad.c:1758:gst_pad_unlink: unlinked v4l2src0:src and fakesink0:sink
0:00:00.504720741  1193    0x16050 INFO           GST_PARENTAGE gstbin.c:1440:gst_bin_remove_func:<pipeline0> removed child "fakesink0"
0:00:00.504814408  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:2955:gst_element_dispose:<fakesink0> dispose
0:00:00.504873000  1193    0x16050 INFO        GST_ELEMENT_PADS gstelement.c:835:gst_element_remove_pad:<fakesink0> removing pad 'sink'
0:00:00.505048889  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:2979:gst_element_dispose:<fakesink0> parent class dispose
0:00:00.505120593  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:3010:gst_element_finalize:<fakesink0> finalize
0:00:00.505179186  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:3021:gst_element_finalize:<fakesink0> finalize parent
0:00:00.505253112  1193    0x16050 INFO           GST_PARENTAGE gstbin.c:1440:gst_bin_remove_func:<pipeline0> removed child "v4l2src0"
0:00:00.505347037  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:2955:gst_element_dispose:<v4l2src0> dispose
0:00:00.505402371  1193    0x16050 INFO        GST_ELEMENT_PADS gstelement.c:835:gst_element_remove_pad:<v4l2src0> removing pad 'src'
0:00:00.505476741  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:2979:gst_element_dispose:<v4l2src0> parent class dispose
0:00:00.505596445  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:3010:gst_element_finalize:<v4l2src0> finalize
0:00:00.505659741  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:3021:gst_element_finalize:<v4l2src0> finalize parent
0:00:00.505719667  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:2955:gst_element_dispose:<pipeline0> dispose
0:00:00.505799704  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:2979:gst_element_dispose:<pipeline0> parent class dispose
0:00:00.505862260  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:3010:gst_element_finalize:<pipeline0> finalize
0:00:00.505917630  1193    0x16050 INFO         GST_REFCOUNTING gstelement.c:3021:gst_element_finalize:<pipeline0> finalize parent
0:00:00.506031334  1193    0x16050 INFO                GST_INIT gst.c:1010:gst_deinit: deinitializing GStreamer
0:00:00.522810815  1193    0x16050 INFO                GST_INIT gst.c:1113:gst_deinit: deinitialized GStreamer

Could it be an problem that I have no v4l2-subdev that can anwser the question from gstreamer for resolution and format?

  • 0
    Genius 3700 points

    Hi Christian Barth,

       Several comments about your questions:

    1. I guess that you are trying to capture 1080p, in that case you need to specify the caps for your pipeline so gstreamer could configure everything to use that resolution:

    gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1' !  omxbufferalloc numBuffers=8 ! gstperf ! omx_scaler  ! v4l2sink device=/dev/video1

    in your case you need at least:

    gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1' !  omxbufferalloc numBuffers=8 ! fakesink -v

    https://developer.ridgerun.com/wiki/index.php/Gstreamer_pipelines_for_DM816x_and_DM814x#Live_Preview_1080p.4060

    2. Please notice that you need to use omxbufferalloc since the V4l2 capture driver uses user pointer to handle the memory so basically omxbufferalloc will allocate the memory that will be used by the v4l2src.

    3. In order to be able to use an stable V4L2 capture driver you need to apply the following patches:

    http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/717/p/228258/809457.aspx#809457

    4. Also, Ridgerun has improved the plugins included with the EZSDK and we keep our own version that is already integrated with our EVAL SDK (which also has the patches mentioned in (3)), you can find more info:

    http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/717/t/327198.aspx

    5. Finally, removing the calls to the subdevices in the V4L2 capture driver is the right approach when you have an FPGA in front of your system, we have done something similar for other customers but also including dual capture:

    https://developer.ridgerun.com/wiki/index.php/Getting_Started_Guide_for_DM8168_EVM#Dual_Capture_Support

    I suppose that for some low latency requirement you have an FPGA in front of the DM816x, just in case Ridgerun also have the V4L2 driver for the ADV7611 already integrated in the DM816x.

    Hope this helps,

    -David

  • 0 in reply to David Soto - RidgeRun
    Intellectual 290 points

    Thx for your suggestions.

    1.+2.) I tested your pipeline and it changed something.

    In the old LOG there was a line: gstv4l2object.c:1525:gst_v4l2_object_get_caps_info:<v4l2src0> rowstride: 32768

    In the new pipeline: gstv4l2object.c:1525:gst_v4l2_object_get_caps_info:<v4l2src0> rowstride: 1920

    The result is the same, but I asume the resolution is correct. Next Problem could be a wrong Color-Mode. I use FVID2_DF_RGB24_888 for our Raw-Input in the hdvpss_capture_sdev_info in file ti81xx_vpss.c (line 257).

    Can I switch it via IOCTL or did I realy need to change the Kernel every time I need a raw RGB image?


    3.+4.) I look on this tomorrow.

    5.) thx

     

    Christian

  • 0 in reply to Christian Barth
    Genius 3700 points

    Hi Christian,

      From your post I think that you are able to get frames at least using the saLoopBack application, right?

       If that is the case then gstreamer should work too and you don't need to keep changing it, what gstreamer would do is to configure the capture subsytem to give you YUY2 or NV12, doesn't matter which color space is on the input.

    After applying the patches suggested on 3), try using:

    gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1' !  omxbufferalloc numBuffers=8 ! gstperf ! fakesink -v
    gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=1920,height=1080,framerate=(fraction)60/1' !  omxbufferalloc numBuffers=8 ! gstperf ! fakesink -v

    For more references about the code changes that we have made in the call you can check our eval SDK


    -David

  • 0 in reply to David Soto - RidgeRun
    Intellectual 290 points

    Hi,

    I checked number 3) in your first anwser. I applied all these patches.

    I made some research on the FPGA-Output of the converted Data (RGB-2-YUV). The FPGA has an IP-Core called "RGB to YCrCb Color-Space Converter". I captured the RAW-Data (24Bit) with our saLoopBack-derived tool. The converted output seems to be in form [Y, U, V] on 3 channels. I looked in the tvp7002 manual an i think the data have to be [Y, U/V] (2 Channels with swapping U and V components). I append a captured image with the converted (rgb2yuv) input.4466.capture1.zip

    It seems the problem is the output of the FPGA YUV-conversion?

  • 0 in reply to Christian Barth
    Genius 3700 points

    Hi Christian, 

    As far as I know the DM816x is able to capture BT1120 and it expect the data as is mentioned on section 2.11.2.3 of the technical reference manual (http://www.ti.com/lit/ug/sprugx8b/sprugx8b.pdf page 401) according to the configuration of the VPSS. I think it has to be  [Y Cb]  [Y Cr]... if you use 16 bits interface configuration

    Actually it expects the data in the same format given by the ADV7611, as I mentioned we have tried ADV7611 -> DM816x and the system captures without problems so why are you doing any conversion?, Also it seems that it can capture RGB in VIP0 too

    -David

  • 0 in reply to David Soto - RidgeRun
    Intellectual 290 points

    We not only using the ADV7611. We have several adaptercards for different signals. ADV7611 for HDMI; INAP375R for APIX2; DS90UR926R/DS90UR906R for FPD-Link. The FPGA is to setup the environment for these receivers. The common signal of all chips is 24Bit RGB. So we use this to capture raw images at low rates for quality masurements.

    A second task is to capture videostreams. In my understanding the Video Encoder is not able to use 24Bit RGB signales so I need to convert it to YCbCr in the FPGA. My approach was to use a gstreamer pipeline, because of the flexibility and the possible outputs (RDP, File).

  • 0 in reply to David Soto - RidgeRun
    Intellectual 290 points

    I made two diffs between our drivers and the arch directory to thehes in bsp-linux04.04.00.02.

    7178.diff.zip

  • 0 in reply to Christian Barth
    Genius 3700 points

    Hi,

      

    Christian Barth said:
    A second task is to capture videostreams. In my understanding the Video Encoder is not able to use 24Bit RGB signales so I need to convert it to YCbCr in the FPGA

    This is true, the encoder needs NV12, however I think that you can feed the VIP with RGB and it does the color space conversion to get YUY2 or NV12 you just need to configure the input port correctly to accept RGB and output YUY2 or NV12. As you can see in the caps above. You can provide more details if you contact us: http://ridgerun.com/www/index.php/contact-us.html we can get an NDA in place.