This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SK-AM62A-LP: Monochrome format support in csi driver

Part Number: SK-AM62A-LP

Tool/software:

Hello,

We are using monochrome sensor and it will give Y8/Y10 data. Currently we are at sensor bring-up stage. and trying to test with EVM AM62A-LP.
In image sensor driver our v4l2 set_pad_format is implemented as below:

static int set_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_state *sd_state,
struct v4l2_subdev_format *fmt)
{
struct i2c_client *client = v4l2_get_subdevdata(sd);
struct sensor *priv = to_sensor(client);
const struct sensor_mode *mode;

if (fmt->which == V4L2_SUBDEV_FORMAT_TRY)
return 0;

mode = sensor_find_best_fit(fmt);
if(mode->sensor_depth==8)
fmt->format.code = MEDIA_BUS_FMT_Y8_1X8;
if(mode->sensor_depth==10) {
fmt->format.code = MEDIA_BUS_FMT_Y10_1X10;
}
fmt->format.width = mode->width;
fmt->format.height = mode->height;
fmt->format.field = V4L2_FIELD_NONE;
priv->mode = mode;

return 0;
}

With above we are receiving error as below when trying to capture image using yavta tool.

root@am62axx-evm:/opt# ./yavta -F /dev/video-rpi-cam0 --capture=2 --list-controls
Device /dev/video-rpi-cam0 opened.
Device `j721e-csi2rx' on `platform:30102000.ticsi2rx' (driver 'j721e-csi2rx') supports video, capture, without mplanes.
unable to query control 0xc0000000: Inappropriate ioctl for device (25).
Video format: UYVY (59565955) 1920x1080 (stride 3840) field none buffer size 4147200
8 buffers requested.
length: 4147200 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffff8503b000.
length: 4147200 offset: 4149248 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xffff84c46000.
length: 4147200 offset: 8298496 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffff84851000.
length: 4147200 offset: 12447744 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffff8445c000.
length: 4147200 offset: 16596992 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffff84067000.
length: 4147200 offset: 20746240 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffff83c72000.
length: 4147200 offset: 24895488 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffff8387d000.
length: 4147200 offset: 29044736 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffff83488000.
Unable to start streaming: Broken pipe (32).
8 buffers released.
root@am62axx-evm:/opt#

Then in further investigation we tried to set image format using below command:

root@am62axx-evm:~# media-ctl -V '"ar0235 4-0036":0 [fmt:Y10_1X10/1920x1200 field:none]' --verbose
Opening media device /dev/media0
Enumerating entities
looking up device: 81:9
looking up device: 81:10
looking up device: 81:11
looking up device: 81:3
looking up device: 81:4
looking up device: 81:5
looking up device: 81:6
looking up device: 81:7
looking up device: 81:8
Found 9 entities
Enumerating pads and links
Setting up format Y10_1X10 1920x1200 on pad ar0235 4-0036/0/0
Format set: Y10_1X10 1920x1200
Setting up format Y10_1X10 1920x1200 on pad cdns_csi2rx.30101000.csi-bridge/0/0
Format set: YUYV8_1X16 1920x1200


In above output we comes to know that the cdns_csi2rx driver is not supporting this format so it is keeping as YUYV8_1X16 1920x1200 instead of Y10. Is there any patch available to support this format in cdns_csi2rx driver?

Just to check impact with other supported format changes below line in set pad format function of driver

Removed => fmt->format.code = MEDIA_BUS_FMT_Y10_1X10;
Replaced With => fmt->format.code = MEDIA_BUS_FMT_SRGGB10_1X10;

With this we are able to set SRGGB10 format at both as below:

root@am62axx-evm:~# media-ctl -V '"ar0235 4-0036":0 [fmt:SRGGB10_1X10/1920x1200 field:none]' --verbose
Opening media device /dev/media0
Enumerating entities
looking up device: 81:9
looking up device: 81:10
looking up device: 81:11
looking up device: 81:3
looking up device: 81:4
looking up device: 81:5
looking up device: 81:6
looking up device: 81:7
looking up device: 81:8
Found 9 entities
Enumerating pads and links
Setting up format SRGGB10_1X10 1920x1200 on pad ar0235 4-0036/0/0
Format set: SRGGB10_1X10 1920x1200
Setting up format SRGGB10_1X10 1920x1200 on pad cdns_csi2rx.30101000.csi-bridge/0/0
Format set: SRGGB10_1X10 1920x1200
root@am62axx-evm:~#

Questions:
1. Does CSI Driver supports Y10_1X10 and Y8_1X8 formats?
2. If Y10_1X10 formats are not supported then how we can get monochrome 8 and 10 bit data from image sensor?




Thanks,
Jaimin

  • Hi,

    Our camera expert is traveling this week. Please expect delayed response.

  • Hello Team,

    Any update on this ?

    Thanks 

  • Hello Team,

    Further we tried to use MEDIA_BUS_FMT_SRGGB10_1X10 format for monochrome 10 bit input in our driver as MEDIA_BUS_FMT_Y10_1X10 is not supported in CSI driver. With that we are not able to receive any frames at application layer and with further debug it is found that the dma callback is not executing in CSI RX driver means frames are not receiving at CSI driver.

    Please find attached output for "media-ctl -p" command and topology file for our image sensor.

    Please let us know what is wrong with configuration causing frames are not receiving at driver layer.

    Monochrome Image sensor is sending data with below config:

    1920x1200 Resolution

    10 Bit (Raw)

    850 Mbps/lane On 4 Lanes

    root@am62axx-evm:~# media-ctl -p
    Media controller API version 6.1.46
    
    Media device information
    ------------------------
    driver          j721e-csi2rx
    model           TI-CSI2RX
    serial
    bus info        platform:30102000.ticsi2rx
    hw revision     0x1
    driver version  6.1.46
    
    Device topology
    - entity 1: 30102000.ticsi2rx (7 pads, 7 links, 1 route)
                type V4L2 subdev subtype Unknown flags 0
                device node name /dev/v4l-subdev0
            routes:
                    0/0 -> 1/0 [ACTIVE]
            pad0: Sink
                    [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                    <- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
            pad1: Source
                    [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                    -> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
            pad2: Source
                    -> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
            pad3: Source
                    -> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
            pad4: Source
                    -> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
            pad5: Source
                    -> "30102000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
            pad6: Source
                    -> "30102000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]
    
    - entity 9: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links, 1 route)
                type V4L2 subdev subtype Unknown flags 0
                device node name /dev/v4l-subdev1
            routes:
                    0/0 -> 1/0 [ACTIVE]
            pad0: Sink
                    [stream:0 fmt:SRGGB10_1X10/1920x1200 field:none]
                    <- "ar0235 4-0036":0 [ENABLED,IMMUTABLE]
            pad1: Source
                    [stream:0 fmt:SRGGB10_1X10/1920x1200 field:none]
                    -> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
            pad2: Source
            pad3: Source
            pad4: Source
    
    - entity 15: ar0235 4-0036 (1 pad, 1 link, 0 route)
                 type V4L2 subdev subtype Sensor flags 0
                 device node name /dev/v4l-subdev2
            pad0: Source
                    [stream:0 fmt:SRGGB10_1X10/1920x1200 field:none]
                    -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
    
    - entity 21: 30102000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video3
            pad0: Sink
                    <- "30102000.ticsi2rx":1 [ENABLED,IMMUTABLE]
    
    - entity 27: 30102000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video4
            pad0: Sink
                    <- "30102000.ticsi2rx":2 [ENABLED,IMMUTABLE]
    
    - entity 33: 30102000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video5
            pad0: Sink
                    <- "30102000.ticsi2rx":3 [ENABLED,IMMUTABLE]
    
    - entity 39: 30102000.ticsi2rx context 3 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video6
            pad0: Sink
                    <- "30102000.ticsi2rx":4 [ENABLED,IMMUTABLE]
    
    - entity 45: 30102000.ticsi2rx context 4 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video7
            pad0: Sink
                    <- "30102000.ticsi2rx":5 [ENABLED,IMMUTABLE]
    
    - entity 51: 30102000.ticsi2rx context 5 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video8
            pad0: Sink
                    <- "30102000.ticsi2rx":6 [ENABLED,IMMUTABLE]

    Below is the output of command when we try to capture using yavta command. It get stuck and not processing further without any error.

    root@am62axx-evm:/opt# ./yavta -c -Fcapture -s 1920x1200 -f SRGGB10 /dev/video3
    Device /dev/video3 opened.
    Device `j721e-csi2rx' on `platform:30102000.ticsi2rx' (driver 'j721e-csi2rx') supports video, capture, without mplanes.
    Video format set: SRGGB10 (30314752) 1920x1200 (stride 3840) field none buffer size 4608000
    Video format: SRGGB10 (30314752) 1920x1200 (stride 3840) field none buffer size 4608000
    8 buffers requested.
    length: 4608000 offset: 0 timestamp type/source: mono/EoF
    Buffer 0/0 mapped at address 0xffffa867b000.
    length: 4608000 offset: 4608000 timestamp type/source: mono/EoF
    Buffer 1/0 mapped at address 0xffffa8216000.
    length: 4608000 offset: 9216000 timestamp type/source: mono/EoF
    Buffer 2/0 mapped at address 0xffffa7db1000.
    length: 4608000 offset: 13824000 timestamp type/source: mono/EoF
    Buffer 3/0 mapped at address 0xffffa794c000.
    length: 4608000 offset: 18432000 timestamp type/source: mono/EoF
    Buffer 4/0 mapped at address 0xffffa74e7000.
    length: 4608000 offset: 23040000 timestamp type/source: mono/EoF
    Buffer 5/0 mapped at address 0xffffa7082000.
    length: 4608000 offset: 27648000 timestamp type/source: mono/EoF
    Buffer 6/0 mapped at address 0xffffa6c1d000.
    length: 4608000 offset: 32256000 timestamp type/source: mono/EoF
    Buffer 7/0 mapped at address 0xffffa67b8000.

    Let us know if any further information required.

    Thanks,

  • Hello Jamin,

    Please try the following.

    1. add the following to Cadence CSI bridge driver: git.ti.com/.../cdns-csi2rx.c

    	{
    		.code	= MEDIA_BUS_FMT_Y8_1X8,
    		.bpp	= 8,
    	},
    	{
    		.code	= MEDIA_BUS_FMT_Y10_1X10,
    		.bpp	= 10,
    	}
    

    2. add the following to TI's CSI-2 RX driver: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c?h=ti-linux-6.1.y#n284

            {
                    .fourcc                 = V4L2_PIX_FMT_GREY,
                    .code                   = MEDIA_BUS_FMT_Y8_1X8,
                    .csi_dt                 = MIPI_CSI2_DT_RAW8,
                    .bpp                    = 8,
                    .size                   = SHIM_DMACNTX_SIZE_8,
            }, {
                    .fourcc                 = V4L2_PIX_FMT_Y10,
                    .code                   = MEDIA_BUS_FMT_Y10_1X10,
                    .csi_dt                 = MIPI_CSI2_DT_RAW10,
                    .bpp                    = 16,
                    .size                   = SHIM_DMACNTX_SIZE_16,
            },

    Regards,

    Jianzhong

  • Hello Jianzhong,

    We have added above code changes in respective drivers and with that we are able to set Y10_1X10 format but our issue remains and not able to get any stream from driver. i.e DMA buffer callback is not executing in driver with any commands (gst-lauch or yavta)

    Please find attached "media-ctl -p" command output for sensor.

    root@am62axx-evm:~# media-ctl -p
    Media controller API version 6.1.46
    
    Media device information
    ------------------------
    driver          j721e-csi2rx
    model           TI-CSI2RX
    serial
    bus info        platform:30102000.ticsi2rx
    hw revision     0x1
    driver version  6.1.46
    
    Device topology
    - entity 1: 30102000.ticsi2rx (7 pads, 7 links, 1 route)
                type V4L2 subdev subtype Unknown flags 0
                device node name /dev/v4l-subdev0
            routes:
                    0/0 -> 1/0 [ACTIVE]
            pad0: Sink
                    [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                    <- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
            pad1: Source
                    [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                    -> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
            pad2: Source
                    -> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
            pad3: Source
                    -> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
            pad4: Source
                    -> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
            pad5: Source
                    -> "30102000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
            pad6: Source
                    -> "30102000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]
    
    - entity 9: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links, 1 route)
                type V4L2 subdev subtype Unknown flags 0
                device node name /dev/v4l-subdev1
            routes:
                    0/0 -> 1/0 [ACTIVE]
            pad0: Sink
                    [stream:0 fmt:Y10_1X10/1920x1200 field:none]
                    <- "ar0235 4-0036":0 [ENABLED,IMMUTABLE]
            pad1: Source
                    [stream:0 fmt:Y10_1X10/1920x1200 field:none]
                    -> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
            pad2: Source
            pad3: Source
            pad4: Source
    
    - entity 15: ar0235 4-0036 (1 pad, 1 link, 0 route)
                 type V4L2 subdev subtype Sensor flags 0
                 device node name /dev/v4l-subdev2
            pad0: Source
                    [stream:0 fmt:Y10_1X10/1920x1200 field:none]
                    -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
    
    - entity 21: 30102000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video3
            pad0: Sink
                    <- "30102000.ticsi2rx":1 [ENABLED,IMMUTABLE]
    
    - entity 27: 30102000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video4
            pad0: Sink
                    <- "30102000.ticsi2rx":2 [ENABLED,IMMUTABLE]
    
    - entity 33: 30102000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video5
            pad0: Sink
                    <- "30102000.ticsi2rx":3 [ENABLED,IMMUTABLE]
    
    - entity 39: 30102000.ticsi2rx context 3 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video6
            pad0: Sink
                    <- "30102000.ticsi2rx":4 [ENABLED,IMMUTABLE]
    
    - entity 45: 30102000.ticsi2rx context 4 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video7
            pad0: Sink
                    <- "30102000.ticsi2rx":5 [ENABLED,IMMUTABLE]
    
    - entity 51: 30102000.ticsi2rx context 5 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video8
            pad0: Sink
                    <- "30102000.ticsi2rx":6 [ENABLED,IMMUTABLE]
    

    We have tried following command to capture image and get stream:

    1.  ./yavta  -F /dev/video-rpi-cam0 --capture=2 -f Y10 -s 1920x1200

    2. gst-launch-1.0 -v v4l2src device=/dev/video-rpi-cam0 ! video/x-raw, width=1920, height=1200, framerate=120/1 ! fakesink

    Please let us know what are the reasons for dma callback is not performing at driver. We are using 1920x1200@120 FPS resolution.

    Please provide the information what parameter we can check to debug this issue further?

    Thanks

  • Hello Jaimin,

    I'm going to be on an emergent leave. Let me try to find somebody else to help you out while I'm gone.

    Thanks for your patience.

    Regards,

    Jianzhong

  • Hi Jaimin,

    Not seeing any frames can happen due to many reasons, like a broken sensor, faulty HW connection (FFC cables go bad easily), or wrong link frequency being programmed at the DPHY-RX.

    Can you please check what link frequency the sensor driver reports through the V4L2_CID_LINK_FREQ control?

    Also, has this particular sensor module and driver been validated on some other platform previously? That can help us rule out any hardware issues.

    Thanks,
    Jai

  • Hello,

    This image sensor is not validated on any platform and we connect to to EVM for development. We are able to get dma buffers from csi driver now and we are analysing it.

    We have configured link frequency 425000000 as MIPI lane data transfer rate is configured as 850mbps on image sensor.

    For pixelrate configuration we are following below equation:
    pixelrate=link_frequency(425000000)*2*num_of_lanes(4)/bpp(10)

    1. Is above pixel_rate equation is correct?
    2. Does csi/phy driver use pixel_rate parameter for image synchronization?
    3. Our understanding is pixel_rate is not dependent on resolution height/width and only need to update when we configure 8 bpp mode on image sensor. Is this correct?

    Thanks

  • Hi Jaimin,

    1. Is above pixel_rate equation is correct?

    Yes, that is correct.

    Does csi/phy driver use pixel_rate parameter for image synchronization?

    CSI doesn't use the link frequency, but D-PHY uses the link frequency to set the clock so that the sensor and the SoC D-PHY both run at the same clock frequency.

     

    Link frequency must be configured at the sensor side and advertised to CSI2-RX via V4L2 framework. The CSI2-RX driver obtains this information by v4l2_get_link_freq(

    https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/v4l2-core/v4l2-common.c?h=ti-linux-6.1.y#n450) and then configures the D-PHY. If the sensor driver doesn't advertise the link frequency, the CSI2-RX driver will try to get the pixel rate and estimate the link frequency.

    Our understanding is pixel_rate is not dependent on resolution height/width and only need to update when we configure 8 bpp mode on image sensor. Is this correct?

    Pixel rate depends on resolution and frame rate: pixel_rate = (width+ horizontal blanking) * (height + vertical blanking) * frame_rate.

    Regards,

    Jianzhong