PROCESSOR-SDK-AM62X: can csi2rx handle RGB888 stream?

Hi Leijie,

I'm out of office today. Let me get back to you in a day or so.

Thanks and regards,

Jianzhong

Hi Leijie,

Xia Leijie said:

you can see in entity 13: dummy_csi2_sensor, the format is RGB888_1X24, while in entity 7: cdns_csi2rx.30101000.csi-bridge,

the format is YUYV8_2X8, is it fine? any comments?

No, this is not fine. You won't be able to capture camera data correctly if the senor format and Cadence CSI bridge format are different.

Currently RGB888 is not supported by the Cadence CSI driver.

Regards,

Jianzhong

hi jianzhong,

is there a way to capture frame directly to memory using may be raw12?

Hi Leijie,

Yes, either GStreamer or Yavta can be used to capture raw frames and save to a file. Is that what you were looking for?

Regards,

Jianzhong

can you share commands using gstreamer or yavta for capture raw frame?

I got  error with below commands, can you help check?

# v4l2-ctl --device /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RGGB --stream-mmap --stream-to=frame.raw --stream-count=10
[13717.067853] cdns-csi2rx 30101000.csi-bridge: Failed to configure external DPHY: -95
VIDIOC_STREAMON returned -1 (Operation not supported)

I have one more query, is it possible to add RGB888 support in Cadence CSI driver?

the background is I'm testing internal colorbar, I guess the colorbar format can't be modified, it's always RGB888 in my case,

even though the data type has been changed.

Please check the Yavta command from here: https://software-dl.ti.com/processor-sdk-linux/esd/AM62AX/08_06_00_45/exports/docs/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX.html#capturing-raw-frames.

Xia Leijie said:

I have one more query, is it possible to add RGB888 support in Cadence CSI driver?

We're planning to add this support in next release.

Regards,

Jianzhong

Hi Jianzhong

in the page, I see this:

where can I find this script? also how can I know which video node is the one I need operate, here is /dev/video2

[   15.958706] init_script.sh[859]: CSI Camera 0 detected
[   15.981126] init_script.sh[859]:     device = /dev/video2
[   15.981439] init_script.sh[859]:     name = imx219
[   15.981604] init_script.sh[859]:     format = [fmt:SRGGB8_1X8/1920x1080]
[   15.981740] init_script.sh[859]:     subdev_id = /dev/v4l-subdev2
[   15.981862] init_script.sh[859]:     isp_required = yes

Hi Leijie,

Xia Leijie said:

where can I find this script?

Please check /opt/edgeai-gst-apps/init_script.sh and /opt/edgeai-gst-apps/scripts.

Xia Leijie said:

how can I know which video node is the one I need operate, here is /dev/video2

When you connect only 1 camera, the first /dev/videoX is the video node for that camera. If you have multiple cameras connected, you'll need to look at the topology through "media-ctl -p".

Regards,

Jianzhong

Hi jianzhong,

currently I have one problem on deserializer configuration, so I want to test csi functions first. 

does sitara has csid generator for test?

Hi Leijie,

Are you asking about the CSID test generator such as this one: https://www.kernel.org/doc/html/v4.14/media/v4l-drivers/qcom_camss.html#qualcomm-camera-subsystem-hardware? We don't have this IP on AM62A.

Regards,
Jianzhong

Hi Jianzhong,

is it already supported yet? or do you have a detail date when it can get supported?

Hi Leijie,

Were you asking about RGB888 support? We haven't planed for that yet.

Regards,

Jianzhong

Hi Jianzhong

do you have any idea if there's some way to capture RGB888 stream? we are block from it.

Hi Leijie,

I'll be out for the first half of next week. Let me try to get back to you later next week.

Regards,

Jianzhong

Hi,

Please see below example patch that enables RGB888 formats in CSI driver.
While capturing you will need to use 32-bit container format (Like RGBX32) as 4 bits padding are added by CSI wrapper IP before storing in memory.

diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
index cf44c00c7421..724446a4504d 100644
--- a/drivers/media/platform/cadence/cdns-csi2rx.c
+++ b/drivers/media/platform/cadence/cdns-csi2rx.c
@@ -185,6 +185,18 @@ static const struct csi2rx_fmt formats[] = {
                .code   = MEDIA_BUS_FMT_SRGGB12_1X12,
                .bpp    = 12,
        },
+       {
+               .code   = MEDIA_BUS_FMT_RGB888_1X24,
+               .bpp    = 24,
+       },
+       {
+               .code   = MEDIA_BUS_FMT_BGR888_1X24,
+               .bpp    = 24,
+       },
+       {
+               .code   = MEDIA_BUS_FMT_RGB565_1X16,
+               .bpp    = 16,
+       },
 };

 static const struct csi2rx_fmt *csi2rx_get_fmt_by_code(u32 code)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index 092805df1e3f..62824af6f8ae 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -44,6 +44,7 @@
 #define CSI_DF_YUV420                  0x18
 #define CSI_DF_YUV422                  0x1e
 #define CSI_DF_RGB444                  0x20
+#define CSI_DF_RGB565                  0x22
 #define CSI_DF_RGB888                  0x24
 #define CSI_DF_RAW8                    0x2a
 #define CSI_DF_RAW10                   0x2b
@@ -306,6 +307,27 @@ static const struct ti_csi2rx_fmt formats[] = {
                .csi_df                 = CSI_DF_RAW10,
                .bpp                    = 16,
                .size                   = SHIM_DMACNTX_SIZE_16,
+       }, {
+               .fourcc                 = V4L2_PIX_FMT_RGBX32,
+               .code                   = MEDIA_BUS_FMT_RGB888_1X24,
+               .colorspace             = V4L2_COLORSPACE_SRGB,
+               .csi_df                 = CSI_DF_RGB888,
+               .bpp                    = 32,
+               .size                   = SHIM_DMACNTX_SIZE_32,
+       }, {
+               .fourcc                 = V4L2_PIX_FMT_BGRX32,
+               .code                   = MEDIA_BUS_FMT_BGR888_1X24,
+               .colorspace             = V4L2_COLORSPACE_SRGB,
+               .csi_df                 = CSI_DF_RGB888,
+               .bpp                    = 32,
+               .size                   = SHIM_DMACNTX_SIZE_32,
+       }, {
+               .fourcc                 = V4L2_PIX_FMT_RGB565,
+               .code                   = MEDIA_BUS_FMT_RGB565_1X16,
+               .colorspace             = V4L2_COLORSPACE_SRGB,
+               .csi_df                 = CSI_DF_RGB565,
+               .bpp                    = 16,
+               .size                   = SHIM_DMACNTX_SIZE_16,
        },

        /* More formats can be supported but they are not listed for now. */

Hi Jai,

so the solution was really adding RGB format support only, no more needed?

were you able to test stream with this patch for RGB888 pattern? I will test it and give feedback .

Thanks for your reply.

Hi Leijie,

I was able to capture RGB888 with OV5640 using v4l2-ctl. Yavta was giving some error for the RGBX format.

root@am62xx-evm:~# v4l2-ctl --stream-mmap -d0 --set-fmt-video=width=640,height=480,pixelformat=XB24
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 44.99 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 44.99 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 45.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 45.00 fps

This fix will be included in 9.0 release, patch shared is just for reference.

Thanks,
Jai

Hi Jai

I still have some problems on it. please check below log.

with 1920*1080, I got a memory alloc exception.

with 640*480, I got a dphy configure fail.

and before capturing, the deserializer is already sending a 1920*1080 colorbar.

root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714# media-ctl --set-v4l2 "'dummy_csi2_sensor':0 [fmt:RGB888_1X24/1920x1080 field:none]"
root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714# media-ctl --set-v4l2 "'30102000.ticsi2rx':0 [fmt:RGB888_1X24/1920x1080 field:none]"
root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714# v4l2-ctl --stream-mmap -d0 --set-fmt-video=width=1920,height=1080,pixelformat=XB24
[  111.270261] ------------[ cut here ]------------
[  111.274907] WARNING: CPU: 0 PID: 1025 at mm/page_alloc.c:4975 __alloc_pages_nodemask+0x29c/0x2e8
[  111.283671] Modules linked in: rpmsg_char cdns_csi2rx crct10dif_ce ti_k3_r5_remoteproc rti_wdt virtio_rpmsg_bus ti_k3_m4_remoteproc sa2ul sha512_generic authenc mcrc j721e_csi2rx videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 cdns_dphy videobuf2_common pvrsrvkm(O) sch_fq_codel cryptodev(O) ipv6
[  111.310270] CPU: 0 PID: 1025 Comm: v4l2-ctl Tainted: G           O      5.10.168364501dcd #1
[  111.318686] Hardware name: HARMAN Z067 DMS (DT)
[  111.323204] pstate: 20000005 (nzCv daif -PAN -UAO -TCO BTYPE=--)
[  111.329196] pc : __alloc_pages_nodemask+0x29c/0x2e8
[  111.334064] lr : __dma_direct_alloc_pages+0x13c/0x1a0
[  111.339098] sp : ffff800011de38e0
[  111.342399] x29: ffff800011de38e0 x28: ffff800011de3ba8
[  111.347698] x27: ffff0000076e1550 x26: ffff000001978860
[  111.352997] x25: 00000000007e9000 x24: ffff8000100e5db8
[  111.358296] x23: ffff000000336810 x22: 000000000000000b
[  111.363596] x21: 0000000000000cc0 x20: ffff000003f0e3c0
[  111.368895] x19: ffff000000336810 x18: 0000000000000000
[  111.374194] x17: 0000000000000000 x16: 0000000000000000
[  111.379492] x15: 0000ffffdb98b138 x14: 0000000000000000
[  111.384791] x13: 0000000000000000 x12: 0000000000000000
[  111.390090] x11: 0000000000000000 x10: 0000000000000000
[  111.395388] x9 : 0000000000000000 x8 : ffff00000553db00
[  111.400687] x7 : 0000000000000000 x6 : 000000000000003f
[  111.405986] x5 : 00000000007e9000 x4 : 0000000000000000
[  111.411285] x3 : 0000000000000000 x2 : 0000000000000000
[  111.416584] x1 : 0000000000000000 x0 : 0000000000000cc0
[  111.421884] Call trace:
[  111.424323]  __alloc_pages_nodemask+0x29c/0x2e8
[  111.428841]  __dma_direct_alloc_pages+0x13c/0x1a0
[  111.433532]  dma_direct_alloc+0x70/0x328
[  111.437442]  dma_alloc_attrs+0x80/0xe8
[  111.441189]  vb2_dc_alloc+0x6c/0x128 [videobuf2_dma_contig]
[  111.446761]  __vb2_queue_alloc+0x18c/0x458 [videobuf2_common]
[  111.452498]  vb2_core_reqbufs+0x1fc/0x478 [videobuf2_common]
[  111.458151]  vb2_ioctl_reqbufs+0x84/0xc0 [videobuf2_v4l2]
[  111.463539]  v4l_reqbufs+0x4c/0x60
[  111.466929]  __video_do_ioctl+0x184/0x3e8
[  111.470926]  video_usercopy+0x18c/0x6d8
[  111.474750]  video_ioctl2+0x18/0x60
[  111.478226]  v4l2_ioctl+0x40/0x60
[  111.481533]  __arm64_sys_ioctl+0xa8/0xf0
[  111.485447]  el0_svc_common.constprop.0+0x78/0x1c8
[  111.490225]  do_el0_svc+0x24/0xa0
[  111.493533]  el0_svc+0x14/0x20
[  111.496576]  el0_sync_handler+0xb0/0xb8
[  111.500402]  el0_sync+0x180/0x1c0
[  111.503706] ---[ end trace 5c8ef45ba90f9eba ]---
[  111.508624] ti-udma 485c0100.dma-controller: dma_alloc_coherent of size 8294400 failed
                VIDIOC_REQBUFS returned -1 (Cannot allocate memory)
root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714#
root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714#
root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714# media-ctl -p
Media controller API version 5.10.168

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial
bus info        platform:30102000.ticsi2rx
hw revision     0x1
driver version  5.10.168

Device topology
- entity 1: 30102000.ticsi2rx (5 pads, 5 links, 1 route)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none]
                <- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none]
                -> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
        pad2: Source
                -> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
        pad3: Source
                -> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
        pad4: Source
                -> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]

- entity 7: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links, 0 route)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                <- "dummy_csi2_sensor":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
        pad2: Source
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
        pad3: Source
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
        pad4: Source
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]

- entity 13: dummy_csi2_sensor (1 pad, 1 link, 0 route)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev2
        pad0: Source
                [stream:0 fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]

- entity 19: 30102000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video0
        pad0: Sink
                <- "30102000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 25: 30102000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video1
        pad0: Sink
                <- "30102000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 31: 30102000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video2
        pad0: Sink
                <- "30102000.ticsi2rx":3 [ENABLED,IMMUTABLE]

- entity 37: 30102000.ticsi2rx context 3 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video3
        pad0: Sink
                <- "30102000.ticsi2rx":4 [ENABLED,IMMUTABLE]
    

root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714# v4l2-ctl --stream-mmap -d0 --set-fmt-video=width=640,height=480,pixelformat=XB24
[  316.273628] cdns-csi2rx 30101000.csi-bridge: Failed to configure external DPHY: -95
                VIDIOC_STREAMON returned -1 (Operation not supported)

Hi,

From the logs you have shared looks like 5MB buffer allocation is failing. Do you have enough CMA memory for streaming?

You can look at output of cat /proc/meminfo to check

And from media graph it looks like camera is sending 1920x1080, so your buffer also needs to be of that size.

[  316.273628] cdns-csi2rx 30101000.csi-bridge: Failed to configure external DPHY: -95

Is your sensor implementing link-frequencies or pixel-rate APIs? One of those will be required for DPHY to be configured properly, preferably link-frequencies.
Please look at other sensor drivers that are known to work like ov5640 or imx219.

Thanks,
Jai

Hi Jai,

I have 2 questions:

1. can you test if you set wrong width and height, then you will also get dphy configure fail error?  I got this when the stream is 1920*1080, but I set 640*480.

2. how to increase the CMA memory for streaming.

see the cma memory below

root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714# cat /proc/meminfo | grep cma -i
CmaTotal: 0 kB
CmaFree: 0 kB

Hi Jai

I have successfully adjust CMA memory.

root@am62xx-z067-dms:/biosOnLinux/project/video-in/MAX96714# cat /proc/meminfo | grep cma -i
CmaTotal: 65536 kB
CmaFree: 64804 kB

but I still met dphy configure fail issue, as you mentioned "link-frequencies", I do see imx219 dts has such node,

while in ov5640 there's no such node, can you explain? and how can I set it.

in imx219 it has been set like this:

./k3-am62a7-sk-csi2-imx219.dts:54:                                              link-frequencies = /bits/ 64 <456000000>;

update：

finally I make it work, thanks for your help Jai.

Hi Leijie,

Would you mind sharing what the problem was?

Regards,

Jianzhong

Hi jianzhong,

now I can use v4l2-tools capture RGB888 raw frame. but the frame seems a little strange.

can you help have a look at the frame? how can I upload it here? I'm not able to insert the file here.

Hi Leijie,

You can upload files by click "Insert" and then "Image/video/file". Then in the pop-up window click "upload" and select the file to upload.

Regards,

Jianzhong

video.txt

Hi Jianzhong,

I'm not able to upload raw image, so I rename it as txt file, please check.

it should be a colorbar.

Hi Jai,

I don't know the order, I'm not able to see the image with image viewer tools, this is why I say the image a little strange in my side

I'm fine with this image. I think this means it's working correct.

Hi Jai,

Thank you very much for your support here.

Leijie,

Very glad that it is working for you. I'll close this thread for now. If you have additional questions, please open a new inquiry.

Regards,

Jianzhong