AM625: MIPI CSI Virtual Channels on Linux

Michał Fita

Part Number: AM625

I'm trying to figure out how to use MIPI CSI Virtual Channels with Linux drivers available on AM625? The ticsi2rx driver has five media pads:

Device topology
- entity 1: 30102000.ticsi2rx (5 pads, 5 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
		<- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
		-> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
	pad3: Source
		-> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
	pad4: Source
		-> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]

The sink is linked with csi-bridge, that itself has five media pads as well:

- entity 7: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
	pad0: Sink
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
		<- "imx219 4-0010":0 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
		-> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
	pad2: Source
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
	pad3: Source
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
	pad4: Source
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]

The sink of it is connected to imx219 I managed to configure via device tree overlay, but that's not my ultimate goal. I'm going to have combined multiple sensors each streaming on different virtual channel.

What I need to understand is either whether source pads of the csi-bridge are mapped to virtual channels of the CSI interface or if there's a way to tell drivers what camera streams on each virtual channel? Would I need to configure more instances of ticsi2rx subdevices to get separate streams? How to configure formats on them as at the moment sink of csi-bridge seem to be fixed to one format?

Any hints welcome.

Thank you.

over 2 years ago

0 Jianzhong Xu over 2 years ago

TI__Guru 62580 points

Hello Michal,

You'll need to manually setup the routes and formats. Attached is a script of setting up multiple IMX219 sensors for your reference. https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/setup_2D00_imx219.sh

Regards,

Jianzhong

0 Michał Fita over 2 years ago in reply to Jianzhong Xu

Prodigy 11 points

Thanks Jianzhong,

But this doesn't address my problem at all. I have my imx219 camera set up via device tree overlay at the moment. But I'm going to use different device that stream on more than one Virtual Channel (by the MIPI CSI standard that's ID on the frame transferred over the bus).

My problem is how to split 4 virtual channels on a single receiver into separate video stream on Linux.

Thanks.

Kind regards,
Michał Fita

0 Jianzhong Xu over 2 years ago in reply to Michał Fita

TI__Guru 62580 points

Hi Michal,

Sorry for not completely understanding your question. You'll need to use a CSI aggregator that takes the data from all cameras and send the aggregated data to the SoC through the CSI interface. The CSI aggregator's driver and TI CSI RX driver will be responsible for managing the virtual channels.

For example, the FPD-Link SerDes, supported by the AM62x SDK, manages the virtual channels. The below picture shows the HW setup:

The DeSer driver will map each sensor's I2C address and virtual channel id to a port on the DeSer board. Please refer to the following:

Serializer (UB953) driver: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/i2c/ds90ub953.c?h=ti-linux-6.1.y
Deserializer (UB960) driver: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/i2c/ds90ub960.c?h=ti-linux-6.1.y
TI CSI2 RX driver: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c?h=ti-linux-6.1.y
Deserializer device tree: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-am62a7-sk-fusion.dtso?h=ti-linux-6.1.y
Sesor device tree:
- Sensor 0: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-fpdlink-imx390-rcm-0-0.dtso?h=ti-linux-6.1.y
- Sensor 1: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-fpdlink-imx390-rcm-0-1.dtso?h=ti-linux-6.1.y

Regards,

Jianzhong

0 Jianzhong Xu over 2 years ago in reply to Jianzhong Xu

TI__Guru 62580 points

Some additional and specific information:

1. https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/i2c/ds90ub960.c?h=ti-linux-6.1.y#n2278: UB960 deserializer aggregates data from multiple cameras, each connected to a port on the deserializer. UB960 creates a virtual channel (VC) number for each camera stream and sends the aggregated streams through a single CSI Tx interface (just one pad).

2. The cdns.csi-bridge is a pass-through device and does not filter any VCs, so all streams (stream0 and stream1 for this example) should just pass though from pad0 to pad1

3. https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/i2c/ds90ub960.c?h=ti-linux-6.1.y#n2765: The CSI-RX driver receives multiple sensor streams from a same pad, but the driver can query which stream maps to which VC using the get_frame_desc hook populated by UB960.

4. https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c?h=ti-linux-6.1.y#n919: the CSI-RX driver finds the VC for each incoming stream.

5. https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c?h=ti-linux-6.1.y#n638: after getting the VC, CSI-RX driver configures each output DMA channel (i.e. /dev/videoX) to only receive data for that particular VC.

6. Userspace has to make sure that multiple streams are routed correctly to each v4l-subdev using the Media Graph APIs (media-ctl command).
For example, for 2x IMX219s connected to UB960, it has to route them to stream0 and stream1 from that subdev to the next, to ultimately be split into different /dev/videoX devices. This is what the script I shared earlier does.

Regards,

Jianzhong

Processors

Processors forum

AM625: MIPI CSI Virtual Channels on Linux