• State
  • Locked Locked
  • Replies 9 replies
  • Subscribers 101 subscribers
  • Views 192 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SK-AM62A-LP: Add a new SerDes camera

Terry pan
Prodigy 10 points
Part Number: SK-AM62A-LP

Tool/software:

Hi TI experts,

I want to bring up a ser/des rgb+ir camera on sk-am62a-lp.

The devices connect like this:

AM62a SOC -- CSI2 -- UB9702 ---FDP LINK --- ub971 -- gw5400 isp yuv

The sensor include rgb and ir channel.

I use SDK 9.02 and modify ds90ub960.c and ov2312.c for this case and refer v3link solution.

ub960 device tree change port because custom ub9702 board output the data from CSI1 not CSI0.

	/* CSI-2 TX */
	port@5 {
		reg = <5>;
		ds90ub960_0_csi_out: endpoint {
			data-lanes = <1 2 3 4>;
			clock-lanes = <0>;
			link-frequencies = /bits/ 64 <800000000>;
			remote-endpoint = <&csi2_phy0>;
		};
	};

After modify related files, it can not start the stream on board.

The error message:

Arago 2023.10 am62axx-evm -

am62axx-evm login: [   14.451050] audit: type=1334 audit(1651203585.664:13): prog-id=12 op=UNLOAD
[   14.458199] audit: type=1334 audit(1651203585.664:14): prog-id=11 op=UNLOAD
root
[   79.475358] audit: type=1006 audit(1651203650.688:15): pid=1889 uid=0 old-auid=4294967295 auid=0 tty=(none) old-ses=4294967295 ses=3 res=1
[   79.487877] audit: type=1300 audit(1651203650.688:15): arch=c00000b7 syscall=64 success=yes exit=1 a0=8 a1=fffff2cfc5c8 a2=1 a3=0 items=0 ppid=1 pid=1889 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3 comm="(systemd)" exe="/lib/systemd/systemd" key=(null)
[   79.513379] audit: type=1327 audit(1651203650.688:15): proctitle="(systemd)"
[   79.520480] audit: type=1334 audit(1651203650.712:16): prog-id=13 op=LOAD
[   79.527312] audit: type=1300 audit(1651203650.712:16): arch=c00000b7 syscall=280 success=yes exit=8 a0=5 a1=ffffc101c360 a2=78 a3=0 items=0 ppid=1 pid=1889 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3 comm="systemd" exe="/lib/systemd/systemd" key=(null)
[   79.552802] audit: type=1327 audit(1651203650.712:16): proctitle="(systemd)"
[   79.559892] audit: type=1334 audit(1651203650.724:17): prog-id=13 op=UNLOAD
[   79.566885] audit: type=1334 audit(1651203650.724:18): prog-id=14 op=LOAD
[   79.573719] audit: type=1300 audit(1651203650.724:18): arch=c00000b7 syscall=280 success=yes exit=8 a0=5 a1=ffffc101c400 a2=78 a3=0 items=0 ppid=1 pid=1889 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3 comm="systemd" exe="/lib/systemd/systemd" key=(null)
[   79.599192] audit: type=1327 audit(1651203650.724:18): proctitle="(systemd)"
OV2312 Camera 0 detected
    device IR = /dev/video-ov2312-ir-cam0
    device RGB = /dev/video-ov2312-rgb-cam0
    name = ov2312
    format = [fmt:UYVY8_1X16/2560x1984 field: none]
    subdev_id = /dev/v4l-ov2312-subdev0
    isp_required = no
    ldc_required = no
root@am62axx-evm:/opt/edgeai-gst-apps# yavta -c1 -fUYVY -Frgb.yuv -s2560x1984 /dev/video-ov2312-rgb-cam0
Device /dev/video-ov2312-rgb-cam0 opened.
Device `j721e-csi2rx' on `platform:30102000.ticsi2rx' (driver 'j721e-csi2rx') supports video, capture, without mplanes.
Video format set: UYVY (59565955) 2560x1984 (stride 5120) field none buffer size 10158080
Video format: UYVY (59565955) 2560x1984 (stride 5120) field none buffer size 10158080
8 buffers requested.
length: 10158080 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffff89140000.
[  135.909874] ds90ub960 4-0030: Prepare for streaming
[  135.916969] cdns-csi2rx 30101000.csi-bridge: Failed to start streams 0x3 on subdev
length: 10158080 offset: 10158080 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xffff88790000.
length: 10158080 offset: 20316160 timestamp type/source: mono/EoF
[  135.940604] cdns-csi2rx 30101000.csi-bridge: Failed to stop stream0
Buffer 2/0 mapped at address 0xffff87de0000.
length: 10158080 offset: 30474240 timestamp type/source: mono/EoF
[  135.957476] cdns-csi2rx 30101000.csi-bridge: Failed to stop stream1
Buffer 3/0 mapped at address 0xffff87430000.
length: 10158080 offset: 40632320 timestamp type/source: mono/EoF
[  135.974883] cdns-csi2rx 30101000.csi-bridge: Failed to stop stream2
Buffer 4/0 mapped at address 0xffff86a80000.
length: 10158080 offset: 50790400 timestamp type/source: mono/EoF
[  135.992163] cdns-csi2rx 30101000.csi-bridge: Failed to stop stream3
Buffer 5/0 mapped at address 0xffff860d0000.
length: 10158080 offset: 60948480 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffff85720000.
length: 10158080 offset: 71106560 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffff84d70000.
Unable to start streaming: Invalid argument (22).
8 buffers released.
root@am62axx-evm:/opt/edgeai-gst-apps#

media-ctl -p output:

root@am62axx-evm:/opt/edgeai-gst-apps# media-ctl -p
Media controller API version 6.1.80

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial
bus info        platform:30102000.ticsi2rx
hw revision     0x1
driver version  6.1.80

Device topology
- entity 1: 30102000.ticsi2rx (7 pads, 7 links, 2 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        routes:
                0/0 -> 1/0 [ACTIVE]
                0/1 -> 2/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                [stream:1 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
        pad2: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
        pad3: Source
                -> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
        pad4: Source
                -> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
        pad5: Source
                -> "30102000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
        pad6: Source
                -> "30102000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]

- entity 9: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links, 2 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        routes:
                0/0 -> 1/0 [ACTIVE]
                0/1 -> 1/1 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/2560x1984 field:none]
                [stream:1 fmt:UYVY8_1X16/2560x1984 field:none]
                <- "ds90ub960 4-0030":4 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_1X16/2560x1984 field:none]
                [stream:1 fmt:UYVY8_1X16/2560x1984 field:none]
                -> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
        pad2: Source
        pad3: Source
        pad4: Source

- entity 15: ds90ub960 4-0030 (6 pads, 2 links, 2 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
        routes:
                0/0 -> 4/0 [ACTIVE]
                0/1 -> 4/1 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/2560x1984 field:none]
                [stream:1 fmt:UYVY8_1X16/2560x1984 field:none]
                <- "ds90ub953 4-0044":1 [ENABLED,IMMUTABLE]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Source
                [stream:0 fmt:UYVY8_1X16/2560x1984 field:none]
                [stream:1 fmt:UYVY8_1X16/2560x1984 field:none]
                -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
        pad5: Source

- entity 24: ds90ub953 4-0044 (2 pads, 2 links, 2 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev3
        routes:
                0/0 -> 1/0 [ACTIVE]
                0/1 -> 1/1 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/2560x1984 field:none]
                [stream:1 fmt:UYVY8_1X16/2560x1984 field:none]
                <- "ov2312 6-0010":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_1X16/2560x1984 field:none]
                [stream:1 fmt:UYVY8_1X16/2560x1984 field:none]
                -> "ds90ub960 4-0030":0 [ENABLED,IMMUTABLE]

- entity 29: ov2312 6-0010 (1 pad, 1 link, 2 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev4
        routes:
                0/0 -> 0/0 [ACTIVE]
                0/0 -> 0/1 [ACTIVE]
        pad0: Source
                [stream:0 fmt:UYVY8_1X16/2560x1984@1/10 field:none colorspace:srgb]
                [stream:1 fmt:UYVY8_1X16/2560x1984@1/10 field:none colorspace:srgb]
                -> "ds90ub953 4-0044":0 [ENABLED,IMMUTABLE]

- entity 35: 30102000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video3
        pad0: Sink
                <- "30102000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 41: 30102000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video4
        pad0: Sink
                <- "30102000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 47: 30102000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video5
        pad0: Sink
                <- "30102000.ticsi2rx":3 [ENABLED,IMMUTABLE]

 I'm not very sure if the ub960 pad 4 source is right since I set pad 5 to output data in dtso.

For experiment, I set port 4 in dtso but still set the data output at CSI1.

It will hang up:

root@am62axx-evm:/opt/edgeai-gst-apps# yavta -c1 -fUYVY -Frgb.yuv -s2560x1984 /dev/video-ov2312-rgb-cam0
Device /dev/video-ov2312-rgb-cam0 opened.
Device `j721e-csi2rx' on `platform:30102000.ticsi2rx' (driver 'j721e-csi2rx') supports video, capture, without mplanes.
Video format set: UYVY (59565955) 2560x1984 (stride 5120) field none buffer size 10158080
Video format: UYVY (59565955) 2560x1984 (stride 5120) field none buffer size 10158080
8 buffers requested.
length: 10158080 offset: 0 timestamp type/source: mono/EoF
[   41.670271] ds90ub960 4-0030: Prepare for streaming
Buffer 0/0 mapped at address 0xffff9a790000.
[   41.677412] ds90ub960 4-0030: enable TX port 0
length: 10158080 offset: 10158080 timestamp type/source: mono/EoF
[   41.687388] ds90ub960 4-0030: enable RX port 0
Buffer 1/0 mapped at address 0xffff99de0000.
length: 10158080 offset: 20316160 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffff99430000.
length: 10158080 offset: 30474240 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffff98a80000.
length: 10158080 offset: 40632320 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffff980d0000.
length: 10158080 offset: 50790400 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffff97720000.
length: 10158080 offset: 60948480 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffff96d70000.
length: 10158080 offset: 71106560 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffff963c0000.


^C[   61.554127] ds90ub960 4-0030: disable RX port 0
[   61.558714] ds90ub960 4-0030: disable TX port 0

root@am62axx-evm:/opt/edgeai-gst-apps#

Any help would be appreciated.

Thank you

Terry

  • 0
    TI__Mastermind 47395 points

    Hello Terry,

    UB9702 is FPD-Link IV deserializer. I don't think the UB960 (FPD-Link III) driver can work with UB9702.

    Regards,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Prodigy 10 points

    Hi Jianzhong,

    In SDK 09.02.00.05 , I can see the ub9702 api in ub960 driver.

    I just modify the ub9702 register settings that verfied by another platform.

    ub9702 mipi output signal  was measured with oscilloscope.

    Regards,

    Terry

  • 0 in reply to Terry pan
    TI__Mastermind 47395 points

    Hi Terry,

    Terry pan said:
    I'm not very sure if the ub960 pad 4 source is right since I set pad 5 to output data in dtso.

    I think this is correct and you should use port 5 in the device tree if CSI Tx port 1 is used. Which pad to use is determined by the driver. It simply uses the first available source pad which is pad 4.

    You modified the ub960 driver, correct? Would you mind sharing the modified driver and the device tree for ub9702?

    Thank you.

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Prodigy 10 points

    Hi Jianzhong,

    ub9720 driver and device tree are modified from the ub960's in SDK.

    In driver, I mark all original register setting but use the regs settings from partner company.

    I will ask the partner if regs settings is required.

    I have check the rx0 in and csi tx1 out in ub9702 setting.

    k3-am62x-sk-csi2-v3link-fusion.dtso

    // SPDX-License-Identifier: GPL-2.0
/*
 * DT Overlay for Arducam V3Link UC-A09 board
 * https://www.arducam.com/fpd-link-3-cameras/
 *
 * Copyright (C) 2024 Texas Instruments Incorporated - http://www.ti.com/
 */

/dts-v1/;
/plugin/;

#include <dt-bindings/gpio/gpio.h>

&{/} {
	clk_fusion_25M_fixed: fixed-clock-25M {
		compatible = "fixed-clock";
		#clock-cells = <0>;
		clock-frequency = <25000000>;
	};
};

&main_i2c2 {
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	i2c-switch@71 {
		compatible = "nxp,pca9543";
		#address-cells = <1>;
		#size-cells = <0>;
		reg = <0x71>;

		i2c@1 {
			#address-cells = <1>;
			#size-cells = <0>;
			reg = <1>;

			deser@30 {
				compatible = "ti,ds90ub9702-q1";
				reg = <0x30>;

				clock-names = "refclk";
				clocks = <&clk_fusion_25M_fixed>;

				i2c-alias-pool = <0x4a 0x4b 0x4c 0x4d 0x4e 0x4f>;

				ds90ub960_0_ports: ports {
					#address-cells = <1>;
					#size-cells = <0>;

					/* CSI-2 TX */
					port@5 {
						reg = <5>;
						ds90ub960_0_csi_out: endpoint {
							data-lanes = <1 2 3 4>;
							clock-lanes = <0>;
							link-frequencies = /bits/ 64 <800000000>;
							remote-endpoint = <&csi2_phy0>;
						};
					};
				};

				ds90ub960_0_links: links {
					#address-cells = <1>;
					#size-cells = <0>;
				};
			};
		};
	};
};

&csi0_port0 {
	status = "okay";

	csi2_phy0: endpoint {
		remote-endpoint = <&ds90ub960_0_csi_out>;
		data-lanes = <1 2 3 4>;
		clock-lanes = <0>;
		link-frequencies = /bits/ 64 <800000000>;
	};
};

    When set port 5 in DTS, there is a bug in ds90ub960.c:

    static int ub960_configure_ports_for_streaming(struct ub960_data *priv,
					       struct v4l2_subdev_state *state)
{
	u8 fwd_ctl;
	struct {
		u32 num_streams;
		u8 pixel_dt;
		u8 meta_dt;
		u32 meta_lines;
		u32 tx_port;
	} rx_data[UB960_MAX_RX_NPORTS] = {};
	u8 vc_map[UB960_MAX_RX_NPORTS] = {};
	struct v4l2_subdev_route *route;
	unsigned int nport;

	ub960_get_vc_maps(priv, vc_map);

	for_each_active_route(&state->routing, route) {
		struct ub960_rxport *rxport;
		struct ub960_txport *txport;
		struct v4l2_mbus_framefmt *fmt;
		const struct ub960_format_info *ub960_fmt;
		unsigned int nport;

		nport = ub960_pad_to_port(priv, route->sink_pad);

		rxport = priv->rxports[nport];
		if (!rxport)
			return -EINVAL;

		//txport = priv->txports[ub960_pad_to_port(priv, route->source_pad)];
		// priv->txports[0] is null since we use csi tx port 1 not 0
		txport = priv->txports[1];
		if (!txport)
			return -EINVAL;

		//rx_data[nport].tx_port = ub960_pad_to_port(priv, route->source_pad);
		rx_data[nport].tx_port = 1;

		rx_data[nport].num_streams++;

.......
.......

    all ub960_pad_to_port(priv, route->source_pad)  seems shoud to be 1

    But it will hang up like set port 4 in DTS.

    root@am62axx-evm:/opt/edgeai-gst-apps# yavta -c1 -fUYVY -Frgb.yuv -s2560x1984 /dev/video-ov2312-rgb-cam0
Device /dev/video-ov2312-rgb-cam0 opened.
Device `j721e-csi2rx' on `platform:30102000.ticsi2rx' (driver 'j721e-csi2rx') supports video, capture, without mplanes.
Video format set: UYVY (59565955) 2560x1984 (stride 5120) field none buffer size 10158080
Video format: UYVY (59565955) 2560x1984 (stride 5120) field none buffer size 10158080
8 buffers requested.
length: 10158080 offset: 0 timestamp type/source: mono/EoF
[   41.670271] ds90ub960 4-0030: Prepare for streaming
Buffer 0/0 mapped at address 0xffff9a790000.
[   41.677412] ds90ub960 4-0030: enable TX port 0
length: 10158080 offset: 10158080 timestamp type/source: mono/EoF
[   41.687388] ds90ub960 4-0030: enable RX port 0
Buffer 1/0 mapped at address 0xffff99de0000.
length: 10158080 offset: 20316160 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffff99430000.
length: 10158080 offset: 30474240 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffff98a80000.
length: 10158080 offset: 40632320 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffff980d0000.
length: 10158080 offset: 50790400 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffff97720000.
length: 10158080 offset: 60948480 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffff96d70000.
length: 10158080 offset: 71106560 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffff963c0000.


^C[   61.554127] ds90ub960 4-0030: disable RX port 0
[   61.558714] ds90ub960 4-0030: disable TX port 0

root@am62axx-evm:/opt/edgeai-gst-apps#

    Does it mean it can not get any image?

    Regards,

    Terry

  • 0 in reply to Terry pan
    TI__Mastermind 47395 points

    Hi Terry,

    There might be some issues when using CSI Tx port 1 instead of port 0. Let me check internally.

    Is it possible for you to try CSI Tx port 0 just for debugging purpose?

    Regards,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Prodigy 10 points

    Hi Jianzhong,

    The ub9702 board only provide CSI TX port 1 connector.

    So it is diffciult for me to use CSI0 pins on 9702 IC.

    Regards,

    Terry

  • 0 in reply to Terry pan
    TI__Mastermind 47395 points

    Terry,

    Please give me a few days to see if I can provide debug patch to use CSI Tx port 1.

    Regards,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Prodigy 10 points

    Hi Jianzhong,

    Have you found the issue or solved it ?

    I try to solve it still.

    Regards,

    Terry

  • 0 in reply to Terry pan
    TI__Mastermind 47395 points

    Hello Terry,

    Sorry for the delay, but unfortunately we haven't been able to work on a debug patch. Using CSI TX port 1 without port 0 is not cleanly supported. I'm afraid we are not able to provide you a solution any time soon.

    Regards,

    Jianzhong