Linux/DRA745: v4l2 userptr access

sinkeu

Intellectual 320 points

Part Number: DRA745

Tool/software: Linux

Hi,

our bsp is based on 6AM.1.2.

I make a test using type V4L2_MEMORY_USERPTR to communicate with v4l2-core.

unfortunately, test failed.

"contiguous mapping is too small" print in kernel log.

my question is:

1. i see this code below in function alloc_stream() (file drivers/media/platform/ti-vpe/vip.c),

q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;

why VB2_USERPTR is not at here, not support?

2. if support VB2_USERPTR, what can i do to make my test pass?

Best Regards!

Sinkeu

over 7 years ago

0 Yordan Kamenov over 7 years ago

TI__Mastermind 42515 points

Hi Sinkeu,

the VIP driver supports only MMAP and DMABUF buffering methods:
processors.wiki.ti.com/index.php

Regards,
Yordan

0 sinkeu over 7 years ago in reply to Yordan Kamenov

Intellectual 320 points

Hi Yordan,

thank you for your reply.
however, I am still confused about one question.
I am trying to find a way rendering video buffer(allocated by function vb2_dc_alloc in file videobuf2-dma-contig.c) to LCD panel as soon as possible. I have no solution now. currently we using V4L2_MEMORY_MMAP then copy the mapped buf to GraphicBuffer, moreover SurfaceFlinger will perform layer compositing. nevertheless it takes a little bit longer time within this whole procedure . I want to replace it with for example V4L2_MEMORY_USERPTR, V4L2_MEMORY_OVERLAY, but the VIP driver is not support these two features according to processors.wiki.ti.com/index.php.

so, is there any solution to make it more faster? such as a way that passing video buffer address(it is dma_addr in fact) to DSS directly, I am not clarify if your other customers have used this way or likely?

Regards,
Sinkeu

0 Yordan Kamenov over 7 years ago in reply to sinkeu

TI__Mastermind 42515 points

Hi Sinkeu,

I have pinged V4L2 experts for further help.

Regards,
Yordan

0 Nikhil Devshatwar over 7 years ago in reply to sinkeu

TI__Genius 9905 points

Why not use dmabuf?

dmabuf is a common mechanism in Linux to share video buffers across drivers without doing any memcpy

Android surfaceflinger should be able to allocate a buffer, export it as dmabuf

Then you can give this buffer to VIP driver for capture and once the capture is done, use it in the surfaceFlinger

I hope this helps

Nikhil D

0 sinkeu over 7 years ago in reply to Nikhil Devshatwar

Intellectual 320 points

Hi Nikhil,

thank's a lot for your advice, it's very useful for me if memcpy will be remove.
I search through web, maybe I use a not suitable keywords, it seems there have a small amount of information about dmabuf. however I have not get a way to use dmabuf in my case.

I saw two demos, it seems they are not match with my case:
the first one: github.com/.../omapdrmtest
the second one:Test application for DMABUF sharing between V4L2 and DRM: lists.freedesktop.org/.../021182.html

now I still confused about:
1. how to export a buffer(allocated by GraphicBufferMapper.lock) as dmabuf? use VIDIOC_EXPBUF?
2. if capture is done, how SurfaceFlinger receive the 'done' signal?
3. is there a demo similar with my case?

thanks in advance.
Sinkeu

0 Nikhil Devshatwar over 7 years ago in reply to sinkeu

TI__Genius 9905 points

Hi,

There is a dmabuf exporter and an importer.

Typical embedded usecases will have the graphics or DRM driver as the main dmabuf exporter.

Check how the android buffers are exported (It uses DRM, so it should support dmabuf export)

Then this video buffer will be given to Video driver for capture.

When the blocking DQBUF ioctl returns, the capture is DONE and now the buffer can be disaplayed.

You can find a demo code at git.ti.com/.../omapdrmtest

Note that, here we are using DRM APIs to allocate and export and display.
There might be something similar in Android

Regards,

Nikhil D

0 sinkeu over 7 years ago in reply to Nikhil Devshatwar

Intellectual 320 points

Hi,

demo omapdrmtest is share buffer between v4l2 and display.
take display-kms.c for example, the key point is kms.c by means of libdrm(omap_bo_new, omap_bo_dmabuf) to export buffer.

however for my case, buffer I use seems have no relationship with libdrm.
it just like, maybe a not good example: char *p = malloc(size); here is a buffer pointed by pointer p, how to export it as dmabuf?

Regards,
Sinkeu

0 Nikhil Devshatwar over 7 years ago in reply to sinkeu

TI__Genius 9905 points

Hi,

All buffers given the hardware (Capture, display) needs to be physically contiguous.

Malloc ed buffers aren't continuous, you cannot pass that buffer to the hardware.

It doesn't know how to access the scattered buffer pages.

For these special needs, you have to ask drivers to allocate buffers with special needs.

That's where DRM APIs come in handy to create these buffers.

e.g. omap_bo_new, ioctl(DRM_DUMB_CREATE), etc

Regards,

Nikhil D

0 sinkeu over 7 years ago in reply to Nikhil Devshatwar

Intellectual 320 points

Hi,

"On SoCs with TILER (OMAP4/5, AM5, DRA7) the driver supports scatter-gather lists for both allocated and imported buffers."

quote from: http://processors.wiki.ti.com/index.php/Linux_Core_DSS_User's_Guide#Buffers

if it is to say SoC DRA7 support non physically contiguous for display?

malloc is really not a good example :-(

another example, there is a buffer, it is physically contiguous, how to export it as dmabuf?

Regards,

Sinkeu

0 Nikhil Devshatwar over 7 years ago in reply to sinkeu

TI__Genius 9905 points

Hi,

These are questions not related to TI software.

You can find more details about the dmabuf in the Linux kernel documentation.

Each driver has its own method of exporting the dmabuf.

For V4L2, its VIDIOC_EXPBUF, For DRM, its DRM_PRIME_HANDLE_TO_FD, etc

Only the dmabufAPIs in the kernel are generic. Userspace usage may depend on the driver who is exporting / importing.

As I explained in my last post, DRM driver should be able to export buffers (e.g. using omap_bo_dmabuf) and then queue them to capture driver using VIDIOC_QBUF with memory = dmabuf (other details in filled in the buf.m.fd)

Please refer to the dmabuftest example mentioned above.

Regards,

NIkhil D

0 sinkeu over 7 years ago in reply to Nikhil Devshatwar

Intellectual 320 points

Hi,

I am trying demo omapdrmtest on my android device.

command: ./dmabuftest -s 32:1568x1080 -d /dev/video0 -c 1568x1080@NV12

1568x1080 is my camera resolution.

something error please refer to attechment.

static int
post_vid_buffer(struct display *disp, struct buffer *buf,
uint32_t x, uint32_t y, uint32_t w, uint32_t h) //util/display-kms.c

the third params 'buf' is represent of a bufer stores one frame of camera video capture data? or, if I write one frame to a file, which buf or poiner I can use?

2. my test failed on drmModeSetPlane call,

failed log: "ERROR:post_vid_buffer:447: failed to enable plane 37: Permission denied"

this error will appear if framebuffer is in use?

3. we know dmabuf is a mechanism sharing buffer between two devices/subsystem. however, where after all the buffer they use come from? is alloced by alloc_bo() called by alloc_buffer() in file display-kms.c? I see __vb2_queue_alloc(videobuf2-core.c) is not alloc memory for type VB2_MEMORY_DMABUF.

Regards,

Sinkeu

Fullscreen drmtest_log.txt Download

root@jacinto6evm:/data/omapdrmtest # ./dmabuftest -s 32:1568x1080 -d /dev/video0 -c 1568x1080@NV12
/dmabuftest -s 32:1568x1080 -d /dev/video0 -c 1568x1080@NV12                  <
Opening Display..
disp_kms_open 32 -1 1568x1080
using 1 connectors, 1920x1080 display, multiplanar: 1
capture buf is w:1920 h:1080
Capture one frame saved
Setting mode 1568x1080 on connector 32, crtc 34
ERROR:post_buffer:312: Could not post buffer on crtc 34: Permission denied (-13)
Opening V4L2..
-1093490196 -1093490192 NV12
get vid buffer - fourcc:842094158 w:1568 h:1080

Exported buffer fd = 18

Exported buffer fd = 19

Exported buffer fd = 20

Exported buffer fd = 21

Exported buffer fd = 22

Exported buffer fd = 23

vid capture buf is w:1568 h:1080
Capture one vid frame saved
ERROR:post_vid_buffer:447: failed to enable plane 37: Permission denied
ERROR:capture_loop:80: Post buffer failed



KERNEL LOG:
root@jacinto6evm:/data/omapdrmtest # dmesg -c
dmesg -c
[ 1952.313677] vip1-s1: vip_open
[ 1952.313707] singulato_360view_get_fmt code=8198,colorspace=1,width=1568,height=1080,field=7 ok
[ 1952.314832] vip1-s1: vip_init_port: g_mbus_fmt subdev mbus_code: 2006 fourcc:NV12 size: 1568x1080
[ 1952.314851] vip1-s1: calc_format_size: fourcc:NV12 size: 1568x1080 bpl:1568 img_size:2540160
[ 1952.314863] vip1-s1: init_stream fourcc:NV12 size: 1568x1080 bpl:1568 img_size:2540160
[ 1952.314873] vip1-s1: init_stream vpdma data type: 0x02
[ 1952.314888] vip1-s1: vip_init_stream: stream instance 0xc020ae70ee379800
[ 1952.314917] vip1-s1: g_fmt fourcc:NV12 code: 2006 size: 1568x1080 bpl:1568 img_size:2540160
[ 1952.314927] vip1-s1: g_fmt vpdma data type: 0x02
[ 1952.314939] video0: VIDIOC_G_FMT: type=vid-cap, width=1568, height=1080, pixelformat=NV12, field=alternate, bytesperline=1568, sizeimage=2540160, colorspace=1, flags=0x0, ycbcr_enc=0, quantization=
0, xfer_func=0
[ 1952.314990] vip1-s1: s_fmt input fourcc:NV12 size: 1568x1080
[ 1952.315003] vip1-s1: try_fmt fourcc:NV12 size: 1568x1080
[ 1952.315013] singulato_360view_enum_frame_size pix=1568x1080 ok
[ 1952.316734] vip1-s1: try_fmt loop:0 fourcc:NV12 size: 1568x540
[ 1952.316752] vip1-s1: try_fmt loop:0 found new larger: 1568x540
[ 1952.316765] vip1-s1: try_fmt best subdev size: 1568x1080
[ 1952.316779] vip1-s1: Scaler active on Port A: requesting 1568x1080
[ 1952.316789] vip1-s1: Scaler: got 1568x1080
[ 1952.316801] vip1-s1: calc_format_size: fourcc:NV12 size: 1568x1080 bpl:1568 img_size:2540160
[ 1952.316811] vip1-s1: s_fmt try_fmt fourcc:NV12 size: 1568x1080
[ 1952.316823] vip1-s1: s_fmt fourcc:NV12 size: 1568x1080 bpl:1568 img_size:2540160
[ 1952.316835] vip1-s1: s_fmt pix_to_mbus mbus_code: 2006 size: 1568x1080
[ 1952.316847] singulato_360view_set_fmt code=8198,colorspace=1,width=1568,height=1080 ok
[ 1952.318270] vip1-s1: s_fmt subdev fmt mbus_code: 2006 size: 1568x1080
[ 1952.318284] vip1-s1: s_fmt vpdma data type: 0x02
[ 1952.318295] video0: VIDIOC_S_FMT: type=vid-cap, width=1568, height=1080, pixelformat=NV12, field=alternate, bytesperline=1568, sizeimage=2540160, colorspace=1, flags=0x0, ycbcr_enc=0, quantization=
0, xfer_func=0
[ 1952.319332] vip1-s1: get 6 buffer(s) of size 2540160 each.
[ 1952.319355] vb2: __vb2_queue_alloc: allocated 6 buffers, 1 plane(s) each
[ 1952.320272] video0: VIDIOC_REQBUFS: count=6, type=vid-cap, memory=dmabuf
[ 1952.320413] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=0, type=vid-cap, flags=0x00002000, field=any, sequence=0, memory=dmabuf, bytesused=0, offset/userptr=0x0, length=2540160
[ 1952.320460] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 1952.320748] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=1, type=vid-cap, flags=0x00002000, field=any, sequence=0, memory=dmabuf, bytesused=0, offset/userptr=0x0, length=2540160
[ 1952.320793] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 1952.320865] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=2, type=vid-cap, flags=0x00002000, field=any, sequence=0, memory=dmabuf, bytesused=0, offset/userptr=0x0, length=2540160
[ 1952.320907] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 1952.320976] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=3, type=vid-cap, flags=0x00002000, field=any, sequence=0, memory=dmabuf, bytesused=0, offset/userptr=0x0, length=2540160
[ 1952.321020] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 1952.321090] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=4, type=vid-cap, flags=0x00002000, field=any, sequence=0, memory=dmabuf, bytesused=0, offset/userptr=0x0, length=2540160
[ 1952.321132] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 1952.321196] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=5, type=vid-cap, flags=0x00002000, field=any, sequence=0, memory=dmabuf, bytesused=0, offset/userptr=0x0, length=2540160
[ 1952.321239] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 1952.321261] vb2: __qbuf_dmabuf: buffer for plane 0 changed
[ 1952.330045] vb2: vb2_core_qbuf: qbuf of buffer 0 succeeded
[ 1952.330785] vb2: __qbuf_dmabuf: buffer for plane 0 changed
[ 1952.339022] vb2: vb2_core_qbuf: qbuf of buffer 1 succeeded
[ 1952.339751] vb2: __qbuf_dmabuf: buffer for plane 0 changed
[ 1952.348067] vb2: vb2_core_qbuf: qbuf of buffer 2 succeeded
[ 1952.349006] vb2: __qbuf_dmabuf: buffer for plane 0 changed
[ 1952.355766] vb2: vb2_core_qbuf: qbuf of buffer 3 succeeded
[ 1952.356501] vb2: __qbuf_dmabuf: buffer for plane 0 changed
[ 1952.363665] vb2: vb2_core_qbuf: qbuf of buffer 4 succeeded
[ 1952.364442] vb2: __qbuf_dmabuf: buffer for plane 0 changed
[ 1952.371926] vb2: vb2_core_qbuf: qbuf of buffer 5 succeeded
[ 1952.372719] vip1-s1: vip_set_slice_path:
[ 1952.372734] vip1-s1: vip_set_slice_path: DATA_PATH_SELECT(00000110): 00000404
[ 1952.373031] vip1-s1: start_streaming: buf 0xe3ec7800 6
[ 1952.373046] singulato_360view_s_stream ok
[ 1952.374017] vip1-s1: start_streaming: start_dma buf 0xe3ec7800
[ 1952.374038] vb2: vb2_core_streamon: successful
[ 1952.374616] video0: VIDIOC_STREAMON: type=vid-cap
[ 1952.374659] vb2: vb2_core_dqbuf: returning done buffer
[ 1952.375433] vb2: vb2_core_dqbuf: dqbuf of buffer 0, with state 0
[ 1952.412478] vip1-s1: vip_release
[ 1952.412497] vip1-s1: vip_stop_streaming:
[ 1952.412506] vip1-s1: vip_disable_sc_path:
[ 1952.412516] singulato_360view_s_stream ok
[ 1952.413055] vip1-s1: Clear channel no: 172
[ 1952.413065] vip1-s1: Clear channel no: 173
[ 1952.413430] vip1-s1: vip_release_stream: stream instance 0xee715cc0ee379800
[ 1952.413444] vip1-s1: vip_release_port: port instance 0xee353400ee353010
root@jacinto6evm:/data/omapdrmtest #

0 Nikhil Devshatwar over 7 years ago in reply to sinkeu

TI__Genius 9905 points

Hello,

Just to summarize our discussion:-
USERPTR is not supported for the TI capture driver.
dmabuf is the preferred method to do video capture, omapdrmtest is an example for it

Now, you are facing an issue with the display - post_buf giving permission denied error is because android UI is already running and some other process is trying to execute the setCrtc or setPlane APIs.
If you want to verify that the capture is running, you can comment out the post_vid_buffer call and instead dump the buffer into a file to confirm that the video capture is happening.

You should raise a separate thread for the display related issues.

Regards,
Nikhil D

0 sinkeu over 7 years ago in reply to Nikhil Devshatwar

Intellectual 320 points

Hi Nikhil,

demo omapdrmtest is running ok. thank you for your support.

Processors

Processors forum

Linux/DRA745: v4l2 userptr access