AM5728: Interlaced video capture issue

DeVdistress

Part Number: AM5728
Other Parts Discussed in Thread: TMDSEVM572X

Hi TI, I am developing a custom am5728 board on SDK05.02.00.10 which can be used to capture interlace video.
Just now i can capture the video successfully, but the video have some jitter.
My am5728 capture is 50 fields per second (used tw2964 decoder), the field is width 704, height 280 (non-standard resolution after video croped 720/288)
I can get 50 interrupts from the vin3a per second.
I doubt whether am5728 take each field as a frame to display which result in jitter.

I studied topic in the forum:

But I did not find a solution to the problem in them, but I found a promise to release a patch from Manisha.

Sep 19, 2017 2:40 AM

We will look into the vpe and v4l2src gstreamer plugin to root cause the jitter with gstreamer pipeline. Please expect the patch with the fix in next 3 weeks or so.

Regards,

Manisha

I tried to use :

gst-launch-1.0 v4l2src device=/dev/video1 io-mode=4 ! 'video/x-raw,format=(string)YUY2, width=(int)704, height=(int)280, framerate=(fraction)50/1' ! vpe num-input-buffers=8 ! 'video/x-raw,format=(string)YUY2, width=(int)704, height=(int)560, framerate=(fraction)50/1, interlace-mode=progressive' ! queue ! waylandsink use-drm=true

It will successful display non deinterlaced video with jitter.

If I insert " interlace-mode=interlaced " in v4l2src caps:

gst-launch-1.0 v4l2src device=/dev/video1 io-mode=4 ! 'video/x-raw,format=(string)YUY2, width=(int)704, height=(int)280, framerate=(fraction)50/1, interlace-mode=interlaced' ! vpe num-input-buffers=8 ! 'video/x-raw,format=(string)YUY2, width=(int)704, height=(int)560, framerate=(fraction)50/1, interlace-mode=progressive' ! queue ! waylandsink use-drm=true

It will report "ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error."

Please help solve the problem, how can deinterlace the video with gstreamer?

over 6 years ago

0 Manisha Agrawal over 6 years ago

TI__Mastermind 22946 points

Hi,

Interlace processing on AM57x device is not supported use case. Display and capture drivers, along with streamer plugin doesn't have right support for it. Along with it, interlace display is not supported by DSS IP on the DPI interface.

Regards,

Manisha

0 DeVdistress over 6 years ago in reply to Manisha Agrawal

Intellectual 635 points

manisha said:
Interlace processing on AM57x device is not supported use case

I'm a little confused, is there support for the deinterlacer in the hardware, but is not used in the Linux drivers?

http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_Kernel_Drivers.html#vpe

Hardware Architecture

VPE(Video Processing Engine) is an IP found on DRA7xx, and in some past TI multimedia SoCs which don’t have baseport support in the mainline kernel.

VPE is a memory to memory block used for performing de-interlacing, scaling and color conversion on input buffers. It’s primarily used to de-interlace decoded DVD/Blu Ray video buffers, and provide the content to progressive display or do some other post processing. VPE can also be used for other tasks like fast color space conversion, scaling and chrominance up/down sampling. The scaler in particular is based on a polyphase filter and supports 32 phases and 5/7 taps.

In the description for the Vpe driver, is there deinterlacer support right?

But in the description of Dss it is written

http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_Kernel_Drivers.html#dss

Unsupported Features/Limitations

Interlaced content is not supported.
Information about interlace top/bottom fields is not given to the userspace, and the userspace has no control if a buffer is shown on top/bottom.

In order for the deinterlaser to work with gstreamer, do me need to write a plug-in for gstreamer, or is it not possible in principle, or do I need something else?

Can you write me a list of drivers components for modification?

0 DeVdistress over 6 years ago in reply to Manisha Agrawal

Intellectual 635 points

Hi, Can anyone help me?

I need to get a deinterlacing video, can you advise how to do it?

Can I change the SDK to the version where it worked?

It is very important for me because my project depends on it.

I'm trying to run capturevpedisplay ( https://git.ti.com/glsdk/omapdrmtest/trees/master ), but I have a error.

./capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 1:704x560
vpe:/dev/video0 open success!!!
vip open success!!!
ERROR:connector_find_mode:492: failed to find mode "704x560"
using 1 connectors, 0x0 display, multiplanar: 1
ERROR:alloc_buffer:182: drmModeAddFB2 failed: Invalid argument (-22)
ERROR:alloc_buffers:226: allocation failed
Segmentation fault (core dumped)

0 DeVdistress over 6 years ago in reply to DeVdistress

Intellectual 635 points

Hi, Can anyone help me?

How can I get a frame of two half frames on am5728 using components linux sdk?

0 Manisha Agrawal over 6 years ago in reply to DeVdistress

TI__Mastermind 22946 points

Hi Oleg,

There are many pieces for the interlace feature to work - involved IPs itself, their drivers and the middleware framework. On AM57x SOC, the interlace feature is broken at different level for different IPs.

VIP and VPE: The IP and the driver supports it but it is broken by the middleware framework Gstreamer. This is due to protocol incompatibility between opensrc v4l2src plugin and TI provided VIP/VPE driver. The interlace field capture can be arranged in two modes - alternate field mode or the interleaved frame mode. VIP/VPE driver understands only the alternate mode of interlace capture while the v4l2src gstreamer plugin only supports the interleaved frame mode. This prevents successful de-interlacing using the gstreamer pipeline.

Option for you to either enhance the v4l2src open source gstreamer plugin to understand "alternate field" mode or enhance the VIP and VPE driver to work in "interleaved frame" mode. I am not sure if VIP/VPE IP has ability to support interleaved frame mode as we haven't tried this, so my suggestion will be to start with modification of v4l2src code or don't work with gstreamer framework.

DSS: Interlaced RGB is not supported on DSS DPI outputs (VOUT1/2/3) because the VSYNC signal's polarity is noncompliant. Some monitors that are fully compliant will exhibit display artifacts. However there are some monitors which accept the noncompliant signal without artifacts.

On the HDMI output side, while there is no issue with the IP, the driver doesn't support interlace display. The biggest missing piece is managing the interlace top/bottom field aspect, so when the userspace commits a new frame, it cannot specify whether it's top/bottom. It can output interlace, when the source frame is progressive, and the userspace has no control on the top/bottom.

Hope this helps.

Regards,

Manisha

0 DeVdistress over 6 years ago in reply to Manisha Agrawal

Intellectual 635 points

Thanks for the answer. If I understand correctly, then without gstreamer, the deinterlacer should work. Will capturevpedisplay ( ) work?

0 Manisha Agrawal over 6 years ago in reply to DeVdistress

TI__Mastermind 22946 points

Hi Oleg,

capturevpedisplay application is not written and tested for interlace use case. I don't think it will work out the box. Moreover, as I mentioned the display DSS IP/driver doesn't comply for interlace display as it doesn't track the top/bottom field type, that can introduce jitter for different reasons when field goes out of order in middle of the processing.

Regards,

Manisha

0 DeVdistress over 6 years ago in reply to Manisha Agrawal

Intellectual 635 points

manisha said:
capturevpedisplay application is not written and tested for interlace use case

please tell me a working example for interlacing?

0 Manisha Agrawal over 6 years ago in reply to DeVdistress

TI__Mastermind 22946 points

Oleg Butko said:

I'm trying to run capturevpedisplay ( https://git.ti.com/glsdk/omapdrmtest/trees/master ), but I have a error.

./capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 1:704x560
vpe:/dev/video0 open success!!!
vip open success!!!
ERROR:connector_find_mode:492: failed to find mode "704x560"
using 1 connectors, 0x0 display, multiplanar: 1
ERROR:alloc_buffer:182: drmModeAddFB2 failed: Invalid argument (-22)
ERROR:alloc_buffers:226: allocation failed
Segmentation fault (core dumped)

The error you are getting is because of incorrect display resolution. The display resolution of "704x560" is not supported by your display device.

720x560 might work (an example below). You can run "modetest" application to find out the supported resolution by your display monitor and try using that as the output res.

#./capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 1:720x560

0 Manisha Agrawal over 6 years ago in reply to DeVdistress

TI__Mastermind 22946 points

Hi Oleg,

There's no working example for interlacing as this is not a supported use case on AM57x Processor SDK.

Regards,

Manisha

0 DeVdistress over 6 years ago in reply to Manisha Agrawal

Intellectual 635 points

Hi, sorry for the delay in reply I had a weekend.

manisha said:
There's no working example for interlacing as this is not a supported use case on AM57x Processor SDK.

Does this apply to all sdk versions, even older ones versions?

manisha said:
The error you are getting is because of incorrect display resolution. The display resolution of "704x560" is not supported by your display device.
720x560 might work (an example below). You can run "modetest" application to find out the supported resolution by your display monitor and try using that as the output res.

#./capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 1:720x560

Today I’ll check and write about the result a little later.

0 DeVdistress over 6 years ago in reply to DeVdistress

Intellectual 635 points

Manisha, can I throw out all V4L2_FIELD_BOTTOM frames with the help of the vip driver modernization ( static irqreturn_t vip_irq(int irq_vip, void *data) )?

This idea is to display only V4L2_FIELD_TOP frames, this will reduce the frame rate to 25Hz and remove jitter

0 DeVdistress over 5 years ago in reply to DeVdistress

Intellectual 635 points

Hi, Can anyone help me?

I still have questions. Invalid status "TI Thinks Resolved"

Сan I throw out all V4L2_FIELD_BOTTOM frames with the help of the vip driver modernization ( static irqreturn_t vip_irq(int irq_vip, void *data) )?

manisha said:
There's no working example for interlacing as this is not a supported use case on AM57x Processor SDK.

Does this apply to all sdk versions, even older ones versions?

0 Manisha Agrawal over 5 years ago in reply to DeVdistress

TI__Mastermind 22946 points

1. You can try that inside VIP or if the filed id status is available in the use space, you can ignore consumption of that buffer.

2. Yes. this is applicable to all versions of the SDK.

0 DeVdistress over 5 years ago in reply to Manisha Agrawal

Intellectual 635 points

manisha said:

1. You can try that inside VIP or if the filed id status is available in the use space, you can ignore consumption of that buffer.

2. Yes. this is applicable to all versions of the SDK.

Thanks for the answer.

manisha said:

manisha

The error you are getting is because of incorrect display resolution. The display resolution of "704x560" is not supported by your display device.
720x560 might work (an example below). You can run "modetest" application to find out the supported resolution by your display monitor and try using that as the output res.

#./capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 1:720x560

Please do not close this topic. I have not yet verified the work of capturevpedisplay.

In the coming time, I will check and write the result.

0 DeVdistress over 5 years ago in reply to DeVdistress

Intellectual 635 points

I did a modetest.

root@am57xx-evm:~# modetest 
trying to open device 'i915'...failed
trying to open device 'amdgpu'...failed
trying to open device 'radeon'...failed
trying to open device 'nouveau'...failed
trying to open device 'vmwgfx'...failed
trying to open device 'omapdrm'...done
Encoders:
id	crtc	type	possible crtcs	possible clones	
34	39	TMDS	0x00000001	0x00000000
40	0	TMDS	0x00000002	0x00000000

Connectors:
id	encoder	status		name		size (mm)	modes	encoders
35	34	connected	DPI-1          	0x0		1	34
  modes:
	name refresh (Hz) hdisp hss hse htot vdisp vss vse vtot)
  800x480 60 800 1010 1040 1056 480 502 515 525 33000 flags: nhsync, nvsync; type: preferred, driver
  props:
	1 EDID:
		flags: immutable blob
		blobs:

		value:
	2 DPMS:
		flags: enum
		enums: On=0 Standby=1 Suspend=2 Off=3
		value: 0
	5 link-status:
		flags: enum
		enums: Good=0 Bad=1
		value: 0
41	0	disconnected	HDMI-A-1       	0x0		0	40
  props:
	1 EDID:
		flags: immutable blob
		blobs:

		value:
	2 DPMS:
		flags: enum
		enums: On=0 Standby=1 Suspend=2 Off=3
		value: 0
	5 link-status:
		flags: enum
		enums: Good=0 Bad=1
		value: 0

CRTCs:
id	fb	pos	size
39	63	(0,0)	(800x480)
  800x480 60 800 1010 1040 1056 480 502 515 525 33000 flags: nhsync, nvsync; type: preferred, driver
  props:
	23 CTM:
		flags: blob
		blobs:

		value:
	24 GAMMA_LUT:
		flags: blob
		blobs:

		value:
	25 GAMMA_LUT_SIZE:
		flags: immutable range
		values: 0 4294967295
		value: 256
	30 background:
		flags: range
		values: 0 16777215
		value: 0
	31 trans-key-mode:
		flags: enum
		enums: disable=0 gfx-dst=1 vid-src=2
		value: 0
	32 trans-key:
		flags: range
		values: 0 16777215
		value: 0
	33 alpha_blender:
		flags: range
		values: 0 1
		value: 0
	37 rotation:
		flags: bitmask
		values: rotate-0=0x1 rotate-90=0x2 rotate-180=0x4 rotate-270=0x8 reflect-x=0x10 reflect-y=0x20
		value: 1
	27 zorder:
		flags: range
		values: 0 3
		value: 0
47	0	(0,0)	(0x0)
   0 0 0 0 0 0 0 0 0 0 flags: ; type: 
  props:
	23 CTM:
		flags: blob
		blobs:

		value:
	24 GAMMA_LUT:
		flags: blob
		blobs:

		value:
	25 GAMMA_LUT_SIZE:
		flags: immutable range
		values: 0 4294967295
		value: 256
	30 background:
		flags: range
		values: 0 16777215
		value: 0
	31 trans-key-mode:
		flags: enum
		enums: disable=0 gfx-dst=1 vid-src=2
		value: 0
	32 trans-key:
		flags: range
		values: 0 16777215
		value: 0
	33 alpha_blender:
		flags: range
		values: 0 1
		value: 0
	43 rotation:
		flags: bitmask
		values: rotate-0=0x1 rotate-90=0x2 rotate-180=0x4 rotate-270=0x8 reflect-x=0x10 reflect-y=0x20
		value: 1
	27 zorder:
		flags: range
		values: 0 3
		value: 0

Planes:
id	crtc	fb	CRTC x,y	x,y	gamma size	possible crtcs
36	39	63	0,0		0,0	0       	0x00000003
  formats: RX12 AR12 RG16 XR24 RG24 AR24 RA24 RX24 AR15 XR12 RA12 XR15
  props:
	6 type:
		flags: immutable enum
		enums: Overlay=0 Primary=1 Cursor=2
		value: 1
	37 rotation:
		flags: bitmask
		values: rotate-0=0x1 rotate-90=0x2 rotate-180=0x4 rotate-270=0x8 reflect-x=0x10 reflect-y=0x20
		value: 1
	27 zorder:
		flags: range
		values: 0 3
		value: 0
	38 zpos:
		flags: range
		values: 0 3
		value: 0
	28 global_alpha:
		flags: range
		values: 0 255
		value: 255
	29 pre_mult_alpha:
		flags: range
		values: 0 1
		value: 0
42	0	0	0,0		0,0	0       	0x00000003
  formats: RG16 RX12 YUYV AR15 RA24 NV12 RA12 XR24 RG24 UYVY AR12 XR15 AR24 XR12 RX24
  props:
	6 type:
		flags: immutable enum
		enums: Overlay=0 Primary=1 Cursor=2
		value: 1
	43 rotation:
		flags: bitmask
		values: rotate-0=0x1 rotate-90=0x2 rotate-180=0x4 rotate-270=0x8 reflect-x=0x10 reflect-y=0x20
		value: 1
	27 zorder:
		flags: range
		values: 0 3
		value: 0
	44 zpos:
		flags: range
		values: 0 3
		value: 0
	28 global_alpha:
		flags: range
		values: 0 255
		value: 255
	29 pre_mult_alpha:
		flags: range
		values: 0 1
		value: 0
48	0	0	0,0		0,0	0       	0x00000003
  formats: RG16 RX12 YUYV AR15 RA24 NV12 RA12 XR24 RG24 UYVY AR12 XR15 AR24 XR12 RX24
  props:
	6 type:
		flags: immutable enum
		enums: Overlay=0 Primary=1 Cursor=2
		value: 0
	49 rotation:
		flags: bitmask
		values: rotate-0=0x1 rotate-90=0x2 rotate-180=0x4 rotate-270=0x8 reflect-x=0x10 reflect-y=0x20
		value: 1
	27 zorder:
		flags: range
		values: 0 3
		value: 2
	50 zpos:
		flags: range
		values: 0 3
		value: 2
	28 global_alpha:
		flags: range
		values: 0 255
		value: 255
	29 pre_mult_alpha:
		flags: range
		values: 0 1
		value: 0
53	0	0	0,0		0,0	0       	0x00000003
  formats: RG16 RX12 YUYV AR15 RA24 NV12 RA12 XR24 RG24 UYVY AR12 XR15 AR24 XR12 RX24
  props:
	6 type:
		flags: immutable enum
		enums: Overlay=0 Primary=1 Cursor=2
		value: 0
	54 rotation:
		flags: bitmask
		values: rotate-0=0x1 rotate-90=0x2 rotate-180=0x4 rotate-270=0x8 reflect-x=0x10 reflect-y=0x20
		value: 1
	27 zorder:
		flags: range
		values: 0 3
		value: 3
	55 zpos:
		flags: range
		values: 0 3
		value: 3
	28 global_alpha:
		flags: range
		values: 0 255
		value: 255
	29 pre_mult_alpha:
		flags: range
		values: 0 1
		value: 0

Frame buffers:
id	size	pitch

I tried to make for different id but everywhere an error

root@am57xx-evm:~# capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 35:800x480
vpe:/dev/video0 open success!!!
vip open success!!!
using 1 connectors, 800x480 display, multiplanar: 1
Setting mode 800x480 on connector 35, crtc 39
ERROR:post_buffer:298: Could not post buffer on crtc 39: Permission denied (-13)
vip: G_FMT(start): width = 704, height = 280, 4cc = YUYV
vpe i/p: G_FMT: width = 704, height = 280, 4cc = YUYV
vpe o/p: G_FMT: width = 704, height = 560, 4cc = YUYV
ERROR:post_buffer:298: Could not post buffer on crtc 39: Permission denied (-13)
ERROR:post_buffer:325: Timeout waiting for flip complete: Permission denied (0)
streaming started...
ERROR:post_vid_buffer:419: failed to enable plane 48: Permission denied
disp post vid buf failed
root@am57xx-evm:~# capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 35:704x560
vpe:/dev/video0 open success!!!
vip open success!!!
using 1 connectors, 800x480 display, multiplanar: 1
Setting mode 704x560 on connector 35, crtc 39
ERROR:post_buffer:298: Could not post buffer on crtc 39: Permission denied (-13)
vip: G_FMT(start): width = 704, height = 280, 4cc = YUYV
vpe i/p: G_FMT: width = 704, height = 280, 4cc = YUYV
vpe o/p: G_FMT: width = 704, height = 560, 4cc = YUYV
ERROR:post_buffer:298: Could not post buffer on crtc 39: Permission denied (-13)
ERROR:post_buffer:325: Timeout waiting for flip complete: Permission denied (0)
streaming started...
ERROR:post_vid_buffer:419: failed to enable plane 48: Permission denied
disp post vid buf failed
root@am57xx-evm:~# capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 34:800x480
vpe:/dev/video0 open success!!!
vip open success!!!
ERROR:connector_find_mode:492: failed to find mode "800x480"
using 1 connectors, 0x0 display, multiplanar: 1
ERROR:alloc_buffer:182: drmModeAddFB2 failed: Invalid argument (-22)
ERROR:alloc_buffers:226: allocation failed
Segmentation fault (core dumped)
root@am57xx-evm:~# capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 39:800x480
vpe:/dev/video0 open success!!!
vip open success!!!
ERROR:connector_find_mode:492: failed to find mode "800x480"
using 1 connectors, 0x0 display, multiplanar: 1
ERROR:alloc_buffer:182: drmModeAddFB2 failed: Invalid argument (-22)
ERROR:alloc_buffers:226: allocation failed
Segmentation fault (core dumped)

I use starter kit TMDSEVM572x. Where is the error?

0 Manisha Agrawal over 5 years ago in reply to DeVdistress

TI__Mastermind 22946 points

Hi Oleg,

There's only single CRTC and connector in your display and it shows 800x480 resolution. So your display resolution is 800x480.

It seems Wayland/Weston is running while you are running the capturevpedisplay application. This is causing the conflict with DRM resource allocation. There can be only master for that resource.

Please exit the Weston and try running the application again -

#/etc/init.d/weston stop

Regards,

Manisha

0 DeVdistress over 5 years ago in reply to Manisha Agrawal

Intellectual 635 points

manisha said:
Please exit the Weston and try running the application again

Thank you very much.

/etc/init.d/weston stop && sleep 1 && capturevpedisplay 704 280 yuyv 704 560 yuyv 1 3 -s 35:800x480

Everything works perfectly.

There is no jitter.

Vpe works with deinterlacer in mode Alternating Field.

This is a working example of a deinterlacer in mode Alternating Field for sdk SDK05.02.00.10

Processors

Processors forum

AM5728: Interlaced video capture issue