AM62A7: CSI2RX: Multi camera with single GSML Deserializer hub

Hi Tri,

Could you please refer the below document to understand the development of multiple camera application? so that you can get an idea

Link:- https://www.ti.com/lit/an/spradh2a/spradh2a.pdf?ts=1768299909431&ref_url=https%253A%252F%252Fwww.bing.com%252F

Regards,

Dilna K

Thanks Dilna.

Have you brought up Maxim 9296/9295 Serdes pair yet? Is the kernel driver available?

Thank you,

Best regards,

Tri

Hi Tri,

Below are the available Max drivers in the SDK.

Note:- You can also check the path ti-processor-sdk-linux-edgeai-am62a-evm-11.01.07.05/board-support/ti-linux-kernel-6.12.35+git-ti/drivers/media/i2c

for other available drivers.

Regards,

Dilna K

Thanks, Dilna

So the driver for the serdes pair is not available yet. Currently what we have been trying is instead of implementing kernel driver for the serdes, we use bash script to init those. And we are able to see the sensors on the I2C bus and able to capture video from them separately

However, we are struggling on getting the device tree to work for the 2 cameras case. We have been testing the following dtso file with aim to have 2 camera appear as 2 v4l2 video device for capture. But the system doesn't boot up and uboot raise the following error. Could you please help to point out what is wrong in this dtso.

Device Tree

// SPDX-License-Identifier: GPL-2.0
/*
 * OX01H1B Camera Module
 * Copyright (C) 2023-2024 Texas Instruments Incorporated - https://www.ti.com/
 */

/dts-v1/;
/plugin/;

#include <dt-bindings/gpio/gpio.h>

&{/} {
	clk_sensor_fixed: ox-xclk {
		compatible = "fixed-clock";
		#clock-cells = <0>;
		clock-frequency = <24000000>;
	};
};

&main_i2c2 {
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	i2c-switch@71 {
		compatible = "nxp,pca9543";
		#address-cells = <1>;
		#size-cells = <0>;
		reg = <0x71>;

		/* CAM port */
		i2c@1 {
			#address-cells = <1>;
			#size-cells = <0>;
			reg = <1>;

			ox05b1s: camera@36 {
				compatible = "ovti,ox05b";
				reg = <0x36>;

				clocks = <&clk_sensor_fixed>;
				clock-names = "inck";

				#pwdn-gpios = <&exp1 13 GPIO_ACTIVE_LOW>;

				port {
					csi2_cam0: endpoint {
						remote-endpoint = <&csi2rx0_in_sensor>;
						link-frequencies = /bits/ 64 <480000000>;
						clock-lanes = <0>;
						data-lanes = <1 2 3 4>;
					};
				};
			};
			ox01h1b: camera@10 {
				compatible = "ovti,ox01h";
				reg = <0x10>;

				clocks = <&clk_sensor_fixed>;
				clock-names = "inck";

				#pwdn-gpios = <&exp1 13 GPIO_ACTIVE_LOW>;

				port {
					csi2_cam1: endpoint {
						remote-endpoint = <&csi2rx1_in_sensor>;
						link-frequencies = /bits/ 64 <480000000>;
						clock-lanes = <0>;
						data-lanes = <1 2 3 4>;
					};
				};
			};

		};
	};
};

&cdns_csi2rx0 {
	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		csi0_port0: port@0 {
			reg = <0>;
			status = "okay";

			csi2rx0_in_sensor: endpoint {
				remote-endpoint = <&csi2_cam0>;
				bus-type = <4>; /* CSI2 DPHY. */
				clock-lanes = <0>;
				data-lanes = <1 2 3 4>;
			};
		};
	};
};

&cdns_csi2rx1 {
	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		csi1_port0: port@0 {
			reg = <0>;
			status = "okay";

			csi2rx1_in_sensor: endpoint {
				remote-endpoint = <&csi2_cam1>;
				bus-type = <4>; /* CSI2 DPHY. */
				clock-lanes = <0>;
				data-lanes = <1 2 3 4>;
			};
		};
	};
};
&ti_csi2rx0 {
	status = "okay";
};

&ti_csi2rx1 {
	status = "okay";
};
&dphy0 {
	status = "okay";
};
&dphy1 {
	status = "okay";
};

Uboot Error

Loading Environment from nowhere... OK
In:    serial@2800000
Out:   serial@2800000
Err:   serial@2800000
Net:   eth0: ethernet@8000000port@1
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc1 is current device
SD/MMC found on device 1
923 bytes read in 2 ms (450.2 KiB/s)
Loaded env from uEnv.txt
Importing environment from mmc1 ...
22323712 bytes read in 241 ms (88.3 MiB/s)
61891 bytes read in 5 ms (11.8 MiB/s)
Working FDT set to 88000000
564 bytes read in 5 ms (109.4 KiB/s)
3572 bytes read in 5 ms (697.3 KiB/s)
failed on fdt_overlay_apply(): FDT_ERR_NOTFOUND
2151 bytes read in 5 ms (419.9 KiB/s)
failed on fdt_overlay_apply(): FDT_ERR_BADMAGIC
base fdt does not have a /__symbols__ node
make sure you've compiled with -@
ERROR: Did not find a cmdline Flattened Device Tree
Could not find a valid device tree
Scanning for bootflows in all bootdevs
Seq  Method       State   Uclass    Part  Name                      Filename
---  -----------  ------  --------  ----  ------------------------  ----------------
Scanning bootdev 'mmc@fa00000.bootdev':
Scanning bootdev 'mmc@fa10000.bootdev':
Bus usb@31100000: generic_phy_get_bulk : no phys property

Thank you

Hi Ti,

I am out of office today. please expect a delay in response.

Regards,

Dilna k

Hi Tri,

Could you please give me a clarity of your current multicamera set up? Now how are you connecting multiple cameras to the SOC at a time? 

1. Could you please ensure main_i2c2, exp1, and cdns_csi2rx0/1 actually enabled/present in your base DTB.

2. Check uEnv.txt: Ensure the filenames in your environment match the compiled .dtbo files exactly.

Note 1:-

Please refer the the section Applying sensor overlays in the below link.

Link:- https://software-dl.ti.com/processor-sdk-linux/esd/AM62AX/latest/exports/docs/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX.html

Regards,

Dilna K

Hi Dilna,

Yes, we have 2 cameras with their own serializer max9295 and both connected to 1 de-serializer max9296

1. In which dts/dtso file are those cdns_csi2rx0/1 defined ?

a. exp1 is not used in my dtso, it was commented out

b. main_i2c2 was enabled, and I can confirm it work for single camera test

2. Yes, I have checked and confirmed the file name is correct

Thank  you,

Best regards,

Tri

Hi Tri,

In the dts/dtso files, each cdns_csi2rx node corresponds to the CSI2RX hardware present on the SoC. In the AM62A, there is one CSIRX so only cdns_csi2rx0 is defined. For the correct layout of the device tree please refer to these two overlays(UB960-UB953 serdes combo with the IMX219 camera) from our official SDK:

1. https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-am62x-sk-csi2-v3link-fusion.dtso?h=11.01.07
2. https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-v3link-imx219-0-0.dtso?h=11.01.07

Of these, [1] adds the UB960 deserializer and [2] adds the IMX219 camera along with UB953 serializer. The topology would look something like this:

The device tree error in uboot logs is most likely because of the cdns_csi2rx1 node is not defined.

The cdns_csi2rx0 node is defined in the following dtsi: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-am62a-main.dtsi?h=11.01.07#n1086

Regards,
Jay

Thank you, Dilna.

You are right. After removing cdns_csi2rx1 node, the system can boot up.

So what we currently trying. In the device tree, we assign 1 camera to port 0 and 1 camera to port 1 of csi2_rx0.

With this, system can boot up, both of the camera kernel driver can be loaded ( one of the kernel driver is ox05b1s.c in your BSP and the other one we implement following 0x05b1s.c structure). However the only camera that is assigned to port 0 appear in the media topology, while the one assigned to port 1 doesn't. Could you please help to review to see if we missed anything here.

// SPDX-License-Identifier: GPL-2.0
/*
 * OX01H1B Camera Module
 * Copyright (C) 2023-2024 Texas Instruments Incorporated - https://www.ti.com/
 */

/dts-v1/;
/plugin/;

#include <dt-bindings/gpio/gpio.h>

&{/} {
	clk_sensor_fixed: ox-xclk {
		compatible = "fixed-clock";
		#clock-cells = <0>;
		clock-frequency = <24000000>;
	};
};

&main_i2c2 {
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	i2c-switch@71 {
		compatible = "nxp,pca9543";
		#address-cells = <1>;
		#size-cells = <0>;
		reg = <0x71>;

		/* CAM port */
		i2c@1 {
			#address-cells = <1>;
			#size-cells = <0>;
			reg = <1>;

			ox05b1s: camera@36 {
				compatible = "ovti,ox05b";
				reg = <0x36>;
				clocks = <&clk_sensor_fixed>;
				clock-names = "inck";
				port {
					csi2_cam1: endpoint {
						remote-endpoint = <&csi2rx1_in_sensor>;
						link-frequencies = /bits/ 64 <480000000>;
						clock-lanes = <0>;
						data-lanes = <1 2 3 4>;
					};
				};
			};
			ox01h1b: camera@10 {
				compatible = "ovti,ox01h";
				reg = <0x10>;

				clocks = <&clk_sensor_fixed>;
				clock-names = "inck";
				port {
					csi2_cam0: endpoint {
						remote-endpoint = <&csi2rx0_in_sensor>;
						link-frequencies = /bits/ 64 <480000000>;
						clock-lanes = <0>;
						data-lanes = <1 2 3 4>;
					};
				};
			};
		};
	};
};

&cdns_csi2rx0 {
	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		csi0_port0: port@0 {          
			reg = <0>;
			status = "okay";
			csi2rx0_in_sensor: endpoint {
				remote-endpoint = <&csi2_cam0>;
				bus-type = <4>; /* CSI2 DPHY. */
				clock-lanes = <0>;
				data-lanes = <1 2 3 4>;
			};
		};
		csi0_port1: port@1 {          
			reg = <1>;
			status = "okay";
			csi2rx1_in_sensor: endpoint {
				remote-endpoint = <&csi2_cam1>;
				bus-type = <4>; /* CSI2 DPHY. */
				clock-lanes = <0>;
				data-lanes = <1 2 3 4>;
			};
		};
	};
};

&ti_csi2rx0 {
	status = "okay";
};

&dphy0 {
	status = "okay";
};

In our case, we are thinking we are actually using port0 for both cameras ( we configure the de-serializer to assign VC id of 0 to camera 0 and VC ID of 1 to stream from camera 1). So our device tree as above might not be correct. But we have no idea how to configure csi2_rx0 for the case of 2 camera, single port, 2 VC IDs. Could you please help us with this?

Thank you,

Tri

Hi Tri,

Could you please follow the below device tree overlays for the reference

1.  https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-am62x-sk-csi2-v3link-fusion.dtso?h=11.01.07

This will provide you an idea of how to configure the deserializer ,DPHY

2. The device tree overlay for the sensor ox05b1s.c is available in the below path you can refer the same for configuring serializer and two cameras.

ti-processor-sdk-linux-edgeai-am62a-evm-11.01.07.05/board-support/ti-linux-kernel-6.12.35+git-ti/arch/arm64/boot/dts/ti/k3-am62a7-sk-csi2-ox05b1s.dtso

3. You said that " you have no idea how to configure csi2_rx0 for the case of 2 camera, single port, 2 VC IDs"

You can compare reference code of connecting UB960-UB953 serdes combo with the multiple IMX219 cameras so that you can get an idea of VC configuration.

camera 1:-

https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-v3link-imx219-0-0.dtso?h=11.01.07

camera 2:-

https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-v3link-imx219-0-1.dtso?h=11.01.07

Note:- 

You can create 2 dtso files like, k3-am62a7-sk-csi2-ox05b1s-0-0.dtso,k3-am62a7-sk-csi2-ox05b1s-0-1.dtso  and add it in the uEnv.txt with k3-am62x-sk-csi2-v3link-fusion.dtso

Regards,

Dilna K

Hi Tri

Could you please let me clarify, do you have the driver for the serdes pair now?or still you are using bash script? as you mentioned earlier?

Regards,

Dilna K

Hi Dilna,

No, we haven't implemented the serdes kernel driver yet, we are still using bash script to initialize and setup the serdes before we install the camera kernel drivers

Thanks,

Tri

So just want to add more clarity here:

1. We use the serdes i2c bash script to initialize and setup the de-serializer and serializers

2. If we test each camera independently, using csi2_rx0 port0, using the device tree as mentioned in 3.2.2.3.4 here. They both work well 

3. Now we configure the deserialiser with 2 input streams from 2 serialisers as VC0 and VC1. How can we update the csi2_rx0 node in device tree to get this to work?

Thank you,

Tri

Hi Tri,

Ok. thanks for the confirmation. 

3. You can try the way that I have motioned in the above response.

Regards,

Dilna K

Hi Dilna,

I have tried your suggestion

You can create 2 dtso files like, k3-am62a7-sk-csi2-ox05b1s-0-0.dtso,k3-am62a7-sk-csi2-ox05b1s-0-1.dtso  and add it in the uEnv.txt with k3-am62x-sk-csi2-v3link-fusion.dtso

name_overlays=ti/k3-am62a7-sk-edgeai.dtbo k3-am62x-sk-csi2-v3link-fusion.dtbo ti/k3-am62a7-sk-csi2-ox05b1s.dtbo ti/k3-am62a7-sk-csi2-ox01h1b.dtbo

But only the camera appears later in the overlay command work ( in this case ox01h1b, and if I swap the order in the command line only 0x05b1s works). Is there any params in csi2rx that I can modify to specify the virtual channel ID to differentiate the 2?

Do you know if there is max9296 kernel driver in any kernel branch that I can look into for reference or bring into SDK to use? 

Thank you,

Best regards

Hi Tri,

Tri Nguyen4 said:

But only the camera appears later in the overlay command work ( in this case ox01h1b, and if I swap the order in the command line only 0x05b1s works). Is there any params in csi2rx that I can modify to specify the virtual channel ID to differentiate the 2?

The DTS files that you are using specify the the deserializer or the camera as the end point of CSIRX port 0. This results in the overlay overwriting the configuration done by the previous overlay.

Tri Nguyen4 said:

So what we currently trying. In the device tree, we assign 1 camera to port 0 and 1 camera to port 1 of csi2_rx0.

This will not work because port1-port4 of the csi_rx0 are output ports. They can not receive data.

Dilna K said:

camera 1:-

https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-v3link-imx219-0-0.dtso?h=11.01.07

camera 2:-

https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-v3link-imx219-0-1.dtso?h=11.01.07

The overlays mentioned by Dilna here highlight the correct topology for the camera sensors. They specify the camera+serializer combination as the end point of the deserializer.

Dilna K said:

Could you please follow the below device tree overlays for the reference

1.  https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-am62x-sk-csi2-v3link-fusion.dtso?h=11.01.07

This device tree adds the deserialiizer as the end point of the CSIRX.

What I would suggest doing is implementing a "dummy driver" for the deserializer that you have. It needs to register a V4L2 subdevice and that would take in two input streams from the camera. It need not configure anything if using the actual hardware. This is needed because the CSIRX accepts a single input subdevice. As a result the deserializer needs to be present in the device tree.

In the long run, the deserializer should have a working driver and should be configured using the same.

Hope this explanation helped.

Regards,
Jay