Linux/TMDXEVM8148: Some problems with capture component input

Hello,

Gstreamer point of view the error that you are observing means the video format (color space) you get from your v4l2src is not compatible with the sink in most of the cases.
I would recommend you to add --gst-debug=v4l2src:5 -v to the pipeline for more debug prints.
You could try also:
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1280,height=720' ! gstperf ! fakesink silent=true

BR
Margarita

Hi Margarita,

Below is the output of above pipelines, could you take a look at the output ? Both pipelines have the error "ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error"

I'm using EVM KIT with default SD Card, but I still get many errors when run gstreamer. !!!???

1. gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1280,height=720,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=8 ! gstperf ! omx_scaler  ! v4l2sink device=/dev/video1 --gst-debug=v4l2src:5 -v

Short log: 

src_capture_start:<v4l2src0> starting the capturing
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw-yuv-strided, forma
t=(fourcc)NV12, framerate=(fraction)60/1, width=(int)1280, height=(int)720, rowstride=(int)1
280
0:00:01.579049599  1502    0x16050 LOG                  v4l2src gstv4l2src.c:804:gst_v4l2src
_unlock_stop:<v4l2src0> No longer flushing
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
0:00:01.583910348  1502    0x16050 LOG                  v4l2src gstv4l2src.c:804:gst_v4l2src
_unlock_stop:<v4l2src0> No longer flushing
New clock: GstSystemClock
0:00:01.585608299  1502   0x180f08 DEBUG                v4l2src v4l2src_calls.c:110:gst_v4l2
src_grab_frame:<v4l2src0> grab frame
0:00:01.619383949  1502   0x180f08 INFO                 v4l2src gstv4l2src.c:1024:gst_v4l2sr
c_create:<v4l2src0> sync to 0:00:00.000000000
Recieved EOS event, press <CTRL+C> to terminate pipeline.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2574): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 37456850 ns.
Setting pipeline to PAUSED ...
0:00:01.622147398  1502    0x16050 LOG                  v4l2src gstv4l2src.c:793:gst_v4l2src
_unlock:<v4l2src0> Flushing
Setting pipeline to READY ...

0:00:01.623447698  1502    0x16050 LOG                  v4l2src gstv4l2src.c:793:gst_v4l2src
_unlock:<v4l2src0> Flushing
0:00:01.623536648  1502    0x16050 LOG                  v4l2src gstv4l2src.c:804:gst_v4l2src
_unlock_stop:<v4l2src0> No longer flushing
0:00:01.623822648  1502    0x16050 DEBUG                v4l2src v4l2src_calls.c:528:gst_v4l2
src_capture_stop:<v4l2src0> stopping capturing
0:00:01.633155948  1502    0x16050 DEBUG                v4l2src v4l2src_calls.c:562:gst_v4l2
src_capture_deinit:<v4l2src0> deinitting capture system
!!!!!!!!!unreference all the remaining buffers!!!!!!!!!/GstPipeline:pipeline0/GstV4l2Src:v4l
2src0.GstPad:src: caps = NULL
Setting pipeline to NULL ...
Freeing pipeline ...

Full log: https://drive.google.com/open?id=0B1sYeYUmGy08N3dXbGFSZUEzZVU

2. gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1280,height=720' ! gstperf ! fakesink silent=true

Short log:

src_capture_start:<v4l2src0> starting the capturing
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw-yuv-strided, forma
t=(fourcc)NV12, framerate=(fraction)100/1, width=(int)1280, height=(int)720, rowstride=(int)
1280
0:00:00.465240077  1511    0x16050 LOG                  v4l2src gstv4l2src.c:804:gst_v4l2src
_unlock_stop:<v4l2src0> No longer flushing
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
0:00:00.469940327  1511    0x16050 LOG                  v4l2src gstv4l2src.c:804:gst_v4l2src
_unlock_stop:<v4l2src0> No longer flushing
New clock: GstSystemClock
0:00:00.471455327  1511   0x16aaa8 DEBUG                v4l2src v4l2src_calls.c:110:gst_v4l2
src_grab_frame:<v4l2src0> grab frame
0:00:00.497549226  1511   0x16aaa8 INFO                 v4l2src gstv4l2src.c:1024:gst_v4l2sr
c_create:<v4l2src0> sync to 0:00:00.000000000
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2574): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 33095299 ns.
Setting pipeline to PAUSED ...
0:00:00.503672126  1511    0x16050 LOG                  v4l2src gstv4l2src.c:793:gst_v4l2src
_unlock:<v4l2src0> Flushing
Setting pipeline to READY ...
0:00:00.504606876  1511    0x16050 LOG                  v4l2src gstv4l2src.c:793:gst_v4l2src
_unlock:<v4l2src0> Flushing
0:00:00.504695176  1511    0x16050 LOG                  v4l2src gstv4l2src.c:804:gst_v4l2src
_unlock_stop:<v4l2src0> No longer flushing
0:00:00.504989226  1511    0x16050 DEBUG                v4l2src v4l2src_calls.c:528:gst_v4l2
src_capture_stop:<v4l2src0> stopping capturing
0:00:00.515228627  1511    0x16050 DEBUG                v4l2src v4l2src_calls.c:562:gst_v4l2
src_capture_deinit:<v4l2src0> deinitting capture system
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = NULL
Setting pipeline to NULL ...
Freeing pipeline ...

Hello,

What is the video source connected to the board and how?
What is the format that it provides?
In the log I do not see the debug messages before the pipeline is moved to playing state.

BR
Margarita

I connect camera into component input. Camera output is 720p (HD analog) and I can capture input with saloopback example.
Here is full log (--gst-debug=v4l2src:5 -v): drive.google.com/open

Hi,

With saloopback example, I choose user pointer buffer (V4L2_MEMORY_USERPTR). Moreover, capture driver only support user pointer buffer.

With above pipeline, I think v4l2src still use memory mapped buffer.

0:00:00.407333943  1444    0x16050 INFO                 v4l2src v4l2src_calls.c:452:gst_v4l2src_capture_init:<v4l2src0> capturing buffers via mmap()

However, this pipeline uses omxbufferalloc, v4l2src must be in user pointer buffer mode.

Do you have other comments about this problem ?

Hello,

I am sorry for the delay.

In the log it seems that the capture mode is set correctly.

In the debug log I see that gst_pad_push is failing. As I said the error that you are observing streaming task paused, reason not-negotiated (-4)  means that you've tried to link incompatible elements or some of the involved elements is not handling the caps correctly.

I assume the second case(capsfilter).

Have you tried pipeline like?

gst-launch -v --gst-debug=v4l2src:1 v4l2src device=/dev/video0 always-copy=false queue-size=12 !  video/x-raw-yuv-strided,format=NV12,width=1280,height=720,framerate=60/1  ! fakesink silent=true

BR
Margarita

Hi,

In general, I can't run any pipeline which contains v4l2src

root@dm814x-evm:/home/shang/capture# gst-launch -v --gst-debug=v4l2src:1 v4l2src device=/dev/video0 always-copy=false queue-size=12 !  video/x-raw-yuv-strided,format=NV12,width=1280,height=720,framerate=60/1  ! fakesink silent=true

WARNING: erroneous pipeline: could not link v4l2src0 to fakesink0

Here is some lines in verbose log:

0:00:00.222618700  1413    0x16050 INFO        GST_PADS gstpad.c:1941:gst_pad_link_prepare: trying to link v4l2src0:src and capsfilter0:sink                                     
0:00:00.223804300  1413    0x16050 INFO        GST_PADS gstpad.c:2124:gst_pad_link_full: linked v4l2src0:src and capsfilter0:sink, successful                                    
0:00:00.224002200  1413    0x16050 INFO        GST_ELEMENT_PADS gstutils.c:1568:gst_element_link_pads_full: trying to link element capsfilter0:src to element fakesink0:(any)

           
0:00:00.224097700  1413    0x16050 INFO        GST_ELEMENT_PADS gstelement.c:963:gst_element_get_static_pad: found pad capsfilter0:src                                                   
0:00:00.224196000  1413    0x16050 INFO        GST_PADS gstutils.c:1028:gst_pad_check_link: trying to link capsfilter0:src and fakesink0:sink                                    
0:00:00.232356450  1413    0x16050 INFO        GST_ELEMENT_PADS gstutils.c:1203:gst_element_get_compatible_pad:<fakesink0> Could not find a compatible pad to link to capsfilter0:src    

0:00:00.232485150  1413    0x16050 INFO        default gstutils.c:1892:gst_element_link_pads_filtered: Could not link pads: capsfilter:src -fakesink0:(null)

0:00:00.232596650  1413    0x16050 INFO        GST_STATES gstelement.c:2341:gst_element_continue_state:<capsfilter0> completed state change to NULL       

Hello,

Could you try to connect your video source to PC and use gstreamer version 0.10 for capture?
Could you try this pipeline on board also?
gst-launch -v --gst-debug=v4l2src:1 v4l2src device=/dev/video0 queue-size=12 ! fakesink silent=false
I have suspicions that probably you will get this error could not link v4l2src0 to fakesink0 because the caps filter is not set.

BR
Margarita

Hi,
Here is the output
root@dm814x-evm:/home/shang/capture# gst-launch -v --gst-debug=v4l2src:1 v4l2src device=/dev
/video0 queue-size=12 ! fakesink silent=false
Setting pipeline to PAUSED ...

Mode set is 720P60
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not negotiate format
Additional debug info:
gstbasesrc.c(2778): gst_base_src_start (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Check your filtered caps, if any
Setting pipeline to NULL ...
Freeing pipeline ...

Now, I don't have any video grabber which supports component input. I will feedback you later.

Hello,

This error was expected.

Are you sure the format is correct?
In this picture is seems like format is wrong:
drive.google.com/.../view

BR
Margarita

I have a software to set camera output and I set 720p60.
With saloopback example, it always detects correct preset 720p60.

Do you tell me about output format of v4l2src ? In saloopback, I set output format as below:
+ capt.fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV
Capture and display are fine.

+ capt.fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_NV12
With this format, I don't display frame, I only save some raw frames and use ffmpeg on PC to convert and display them. Frame data is wrong. Therefore, I want to know about NV12 format which v4l2 driver pack data.

With gstreamer, I haven't run any pipeline ( both YUYV and NV12)

Hello,

Please check this thread:
e2e.ti.com/.../524827
It could provides you some ideas + patch for omxbufferalloc(page 2).

BR
Margarita

Hi,
I think I'm using EVM Kit with default EZSDK, gstreamer will always run correctly.
I will check above thread.
Thank for your support !!!

Hi,

My pipeline could run very well. This patch is also in component-sources/gst-openmax_GST_DM81XX_00_07_00_00/patches. 

However, ARM load was very high, ~ 70 % CPU. saloopback example only spent 1% CPU.

Could you give me some comments about this problem ?

Hello,

Please post the pipeline that you are using.

BR
Margarita

Hi,

Here is my pipeline

root@dm814x-evm:~# gst-launch v4l2src always-copy=false queue-size=8  ! 'video/x-raw-yuv-strided,format=(fourcc)NV12,width=1280,height=720,framerate=(fraction)60/1' !  omxbufferalloc numBuffers=8 ! omx_scaler !  gstperf ! v4l2sink device=/dev/video1
Setting pipeline to PAUSED ...

** (gst-launch-0.10:1362): CRITICAL **: gst_v4l2sink_sync_rotation: assertion `v4l2_ioctl (fd, VIDIOC_S_CTRL, &control) >= 0' failed

** (gst-launch-0.10:1362): CRITICAL **: gst_v4l2sink_sync_rotation: assertion `v4l2_ioctl (fd, VIDIOC_S_CTRL, &control) >= 0' failed

** (gst-launch-0.10:1362): CRITICAL **: gst_v4l2sink_sync_flip: assertion `v4l2_ioctl (fd, VIDIOC_S_CTRL, &control) >= 0' failed

Mode set is 720P60
allocating 8 buffers of size:1382400!!
allocated outbuf:0x40bef080
allocated outbuf:0x40d40880
allocated outbuf:0x40e92080
allocated outbuf:0x40fe3880
allocated outbuf:0x41135080
allocated outbuf:0x41286880
allocated outbuf:0x413d8080
allocated outbuf:0x41529880
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
perf0: frames: 46       current: 45.56   average: 45.56 arm-load: 18
perf0: frames: 96       current: 49.15   average: 47.36 arm-load: 67
perf0: frames: 148      current: 50.59   average: 48.45 arm-load: 65
perf0: frames: 201      current: 52.43   average: 49.44 arm-load: 61
perf0: frames: 251      current: 49.28   average: 49.40 arm-load: 59
perf0: frames: 301      current: 49.82   average: 49.47 arm-load: 68
perf0: frames: 351      current: 49.38   average: 49.46 arm-load: 65
perf0: frames: 401      current: 49.21   average: 49.43 arm-load: 65

Hello,

This is normal to observe higher CPU load when you are using gstreamer instead of saLoopBack.
saLoopBack demo is using low level v4l2 capture driver.

However, you could try to enlarge queue-size properties of the v4l2src element (v4l2src queue-size=16).
I would recommend you to add some queue elements in the pipeline and set the properties.
By default queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 uses these properties.
Please enlarge the buffers in queue. Even if it is not necessary increasing the buffer-sizes does not do any harm to add them in the pipeline. Big buffers only increase the pipeline's latency which does not matter for capturing purposes.

BR
Margarita

Hi,
I think both v4l2src and saloopback use v4l2 API directly (Open device, set up format, set up buffers, stream on, queue and dequeue buffers).
V4l2 subsystem will control HDVPSS to capture frame, so arm-load should be small.

Could you write down your pipeline ?
Here is my new pipeline, it still spent too much CPU load.

root@dm814x-evm:~# gst-launch v4l2src always-copy=false queue-size=16 ! 'video/x-raw-yuv-strided,format=(fourcc)NV12,width=1280,height=720,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=16 ! queue max-size-buffers=16 ! omx_scaler ! gstperf ! v4l2sink device=/dev/video1
Setting pipeline to PAUSED ...

** (gst-launch-0.10:1366): CRITICAL **: gst_v4l2sink_sync_rotation: assertion `v4l2_ioctl (fd, VIDIOC_S_CTRL, &control) >= 0' failed

** (gst-launch-0.10:1366): CRITICAL **: gst_v4l2sink_sync_rotation: assertion `v4l2_ioctl (fd, VIDIOC_S_CTRL, &control) >= 0' failed

** (gst-launch-0.10:1366): CRITICAL **: gst_v4l2sink_sync_flip: assertion `v4l2_ioctl (fd, VIDIOC_S_CTRL, &control) >= 0' failed

Mode set is 720P60
allocating 16 buffers of size:1382400!!
allocated outbuf:0x40cca080
allocated outbuf:0x40e1b880
allocated outbuf:0x40f6d080
allocated outbuf:0x410be880
allocated outbuf:0x41210080
allocated outbuf:0x41361880
allocated outbuf:0x414b3080
allocated outbuf:0x41604880
allocated outbuf:0x41756080
allocated outbuf:0x418a7880
allocated outbuf:0x419f9080
allocated outbuf:0x41b4a880
allocated outbuf:0x41c9c080
allocated outbuf:0x41ded880
allocated outbuf:0x41f3f080
allocated outbuf:0x42090880
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
perf0: frames: 46 current: 45.67 average: 45.67 arm-load: 37
perf0: frames: 95 current: 48.24 average: 46.96 arm-load: 73
perf0: frames: 146 current: 49.70 average: 47.88 arm-load: 70
perf0: frames: 195 current: 47.73 average: 47.84 arm-load: 69
perf0: frames: 245 current: 49.86 average: 48.24 arm-load: 73
perf0: frames: 296 current: 50.20 average: 48.57 arm-load: 66
perf0: frames: 345 current: 48.86 average: 48.61 arm-load: 73
perf0: frames: 394 current: 48.38 average: 48.58 arm-load: 68
perf0: frames: 443 current: 48.20 average: 48.54 arm-load: 63
perf0: frames: 491 current: 47.09 average: 48.39 arm-load: 69

Hello,


Please try to add the queue in this places:

gst-launch v4l2src queue-size=16 ! 'video/x-raw-yuv-strided,format=(fourcc)NV12,width=1280,height=720,framerate=(fraction)60/1' ! queue max-size-buffers=100 max-size-time=0 max-size-bytes=0 ! omxbufferalloc numBuffers=16 ! queue queue max-size-buffers=100 max-size-time=0 max-size-bytes=0 ! omx_scaler ! gstperf ! v4l2sink device=/dev/video1

You could try to increase max-size-buffers more.

BR
Margarita

Hi,

When inserting queue element with max-size-buffers=20, 100 or 200, arm-load approximated to 60% in all cases.

I also found a strange behavior when running gstreamer. Sometimes, I pressed Ctrl C to stop the pipeline, but it stuck at "Setting pipeline to PAUSED ...". I had to press Ctrl C one more to stop. After that, if I ran the pipeline again, arm-load only approximated to 30%.

Beside using gstreamer, my application also runs others task on A8, so I really want to optimize this pipeline.

Ridge Run has improved TI gst-openmax. Should I buy their packages ? 

Hello,

Kiet Phi77 said:

I also found a strange behavior when running gstreamer. Sometimes, I pressed Ctrl C to stop the pipeline, but it stuck at "Setting pipeline to PAUSED ...". I had to press Ctrl C one more to stop

You could add -e in the pipeline.

Kiet Phi77 said:

Beside using gstreamer, my application also runs others task on A8,

This is could be the reason of the load.

Kiet Phi77 said:

Ridge Run has improved TI gst-openmax. Should I buy their packages ? 

I am sorry I am not aware with this.

BR
Margarita

Hi,
I am only testing with above gstreamer pipeline. I will try to debug this problem later.
Thank for your support !

Hi Margarita,
Could you send me source code of firmware dm816x_hdvpss_v4l2.xem3 ? I want to check some configurations.
Thanks !

Hello,

This source code is part of overlay package and it is under NDA so I can not send it.

BR
Margarita

Hi Margarita,

I have a small question, could you help me for a second ?

Does TI81XX Video Capture driver support non-blocking mode ? I open /dev/video0 in non-blocking mode (O_RDWR | O_NONBLOCK), and use select function to check for IO. However, select function always returns TIMEOUT.

FD_ZERO (&fds);
FD_SET (fd, &fds);
/* Timeout. */
tv.tv_sec = 2;
tv.tv_usec = 0;

int ret = select (fd + 1, &fds, NULL, NULL, &tv);