SK-AM68: port imx477 to SK-AM68

csscyt

Prodigy 30 points

Part Number: SK-AM68

make all log:

make_all.log

Tool/software:

Hi,

I need to port imx477 to SK-AM68 platform,
I follow this
https://dev.ti.com/tirex/explore/content/am62ax_academy_9_02_00_00_v1/_build_am62ax_academy_9_02_00_00_v1/source/linux/ch-develop/dev-use-camera.html#enable-a-new-csi-2-sensor

Enable A New CSI-2 Sensor

The Processor SDK supports a few sensors out-of-box. Both the driver modules and the device tree overlays for the supported sensors are pre-built and included in the SDK. To enable a new CSI-2 sensor, the following listed steps are needed. Some steps can be skipped if the driver is already included in the SDK.

Add the sensor driver source code to drivers/media/i2c/<sensor_name>.c (path is relative to <sdk root>/board-support/ti-linux-kernel).

Add the sensor configuration in drivers/media/i2c/Kconfig. For example:

config VIDEO_<SENSOR_NAME>
        tristate "<sensor_name> support"
        depends on VIDEO_DEV && I2C
        depends on ACPI || COMPILE_TEST
        select MEDIA_CONTROLLER
        select VIDEO_V4L2_SUBDEV_API
        select V4L2_FWNODE
        help
          This is a Video4Linux2 sensor driver for the <sensor_name>

Enable the compilation of the sensor driver in drivers/media/i2c/Makefile:
obj-$(CONFIG_VIDEO_<SENSOR_NAME>) += <sensor_name>.o
Enable the kernel module for the sensor in arch/arm64/configs/defconfig:
CONFIG_VIDEO_<SENSOR_NAME>=m
Create a device tree overlay for the sensor and add it to arch/arm64/boot/dts/ti:
arch/arm64/boot/dts/ti/<device_tree_overlay_name>.dtso
Add the device tree overlay to arch/arm64/boot/dts/ti/Makefile:
dtb-$(CONFIG_ARCH_K3) += <device_tree_overlay_name>.dtso
Rebuild and install Linux kernel according to the instructions given in section Build Kernel.
Boot the EVM with the updated kernel. Apply the device tree overlay according to instructions given in the Camera Evaluation section.
Verify the sensor is probed according to Camera Evaluation.

the kernel module(imx477.ko) has been updated to the target board, and enable it by dtb overlay file in /run/media/BOOT-mmcblk1p1/uEnv.txt as below,

name_overlays=k3-j721s2-edgeai-apps.dtbo k3-am68-sk-bb-rpi-cam-imx477.dtbo

but i cann't see the imx477 log during the booting the EVM, did I miss any steps? thanks

14 days ago

0 Fabiana Jaimes 14 days ago

TI__Genius 10020 points

Hi,

What was the result of the camera verification? Could you share the output of dmesg | grep -i imx and dmesg | grep -i csi ? If you haven't already, I recommend adding imx477 to the setup_cameras.sh script located in the /opt/edgeai-gst-apps/scripts directory.

Thank you,

Fabiana

0 csscyt 13 days ago in reply to Fabiana Jaimes

Prodigy 30 points

the output log：

root@am68a-sk:/opt/edgeai-gst-apps# dmesg | grep imx

[ 4.811001] imx477 4-0010: supply dovdd not found, using dummy regulator
[ 4.822631] imx477 4-0010: supply avdd not found, using dummy regulator
[ 4.830100] imx477 4-0010: supply dvdd not found, using dummy regulator
[ 4.863887] imx477 4-0010: failed to find sensor: -5
[ 4.880402] imx477: probe of 4-0010 failed with error -5

[ 4.971633] imx477 5-0010: supply dovdd not found, using dummy regulator
[ 5.069309] imx477 5-0010: supply avdd not found, using dummy regulator
[ 5.133913] imx477 5-0010: supply dvdd not found, using dummy regulator

[ 5.171249] imx477 5-0010: before read reg
[ 5.183112] imx477 5-0010: client->addr is 16
[ 5.192762] imx477 5-0010: failed to find sensor: -5
[ 5.273839] imx477: probe of 5-0010 failed with error -5

and in imx477.c

/* Chip ID */

#define IMX477_REG_ID 0x0016

#define IMX477_ID 0x477

the k3-am68-sk-bb-rpi-cam-imx477.dtso

/dts-v1/;
/plugin/;

#include <dt-bindings/gpio/gpio.h>
#include "k3-pinctrl.h"

&{/} {
	clk_imx477_fixed: imx477-xclk {
		compatible = "fixed-clock";
		#clock-cells = <0>;
		clock-frequency = <24000000>;
	};
};

&exp3 {
	p01-hog {
		/* CSI_MUX_SEL_2 */
		gpio-hog;
		gpios = <1 GPIO_ACTIVE_HIGH>;
		output-high;
		line-name = "CSI_MUX_SEL_2";
	};
};

&main_i2c1 {
	status = "okay";
	#address-cells = <1>;
	#size-cells = <0>;

	i2c-switch@70 {
		compatible = "nxp,pca9543";
		#address-cells = <1>;
		#size-cells = <0>;
		reg = <0x70>;

		i2c-alias-pool = /bits/ 16 <0x10 0x11>;

		/* CAM0 I2C */
		cam0_i2c: i2c@0 {
			#address-cells = <1>;
			#size-cells = <0>;
			reg = <0>;

			imx477_0: imx477_0@10 {
				compatible = "sony,imx477";
				reg = <0x10>;

				clocks = <&clk_imx477_fixed>;
				clock-names = "xclk";

				port {
					csi2_cam0: endpoint {
						remote-endpoint = <&csi2rx0_in_sensor>;
						link-frequencies = /bits/ 64 <456000000>;
						clock-lanes = <0>;
						data-lanes = <1 2>;
					};
				};
			};
		};

		/* CAM1 I2C */
		cam1_i2c: i2c@1 {
			#address-cells = <1>;
			#size-cells = <0>;
			reg = <1>;

			imx477_1: imx477_1@10 {
				compatible = "sony,imx477";
				reg = <0x10>;

				clocks = <&clk_imx477_fixed>;
				clock-names = "xclk";

				port {
					csi2_cam1: endpoint {
						remote-endpoint = <&csi2rx1_in_sensor>;
						link-frequencies = /bits/ 64 <456000000>;
						clock-lanes = <0>;
						data-lanes = <1 2>;
					};
				};
			};
		};
	};
};

&csi0_port0 {
	status = "okay";
	csi2rx0_in_sensor: endpoint {
		remote-endpoint = <&csi2_cam0>;
		bus-type = <4>; /* CSI2 DPHY. */
		clock-lanes = <0>;
		data-lanes = <1 2>;
	};
};

&csi1_port0 {
	status = "okay";
	csi2rx1_in_sensor: endpoint {
		remote-endpoint = <&csi2_cam1>;
		bus-type = <4>; /* CSI2 DPHY. */
		clock-lanes = <0>;
		data-lanes = <1 2>;
	};
};

Could you help me confirm which step is wrong or what is missing? thanks

in k3-am68-sk-bb-rpi-cam-imx477.dtso the device reg addr is 0x10 or 0x1A?

0 Fabiana Jaimes 13 days ago in reply to csscyt

TI__Genius 10020 points

Hi,

Thank you for the data. Due to a US holiday, there will a delay in my response. Thanks for understanding.

-Fabiana

0 csscyt 12 days ago in reply to Fabiana Jaimes

Prodigy 30 points

root@am68a-sk:/opt/edgeai-gst-apps# dmesg | grep -i csi
[ 0.428166] SCSI subsystem initialized
[ 0.712024] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[ 1.137471] i2c 4-001a: Fixed dependency cycle(s) with /bus@100000/ticsi2rx@4500000/csi-bridge@4504000
[ 1.152089] i2c 5-001a: Fixed dependency cycle(s) with /bus@100000/ticsi2rx@4510000/csi-bridge@4514000
[ 1.185712] gpio-481 (CSI_MUX_SEL_2): hogged as output/high
[ 4.931977] imx477 4-001a: gpio info: CSI0_B_GPIO1
[ 6.379594] imx477 5-001a: gpio info: CSI0_B_GPIO1
[ 7.021885] cdns-csi2rx 4504000.csi-bridge: Probed CSI2RX with 2/4 lanes, 4 streams, external D-PHY
[ 7.090799] cdns-csi2rx 4514000.csi-bridge: Probed CSI2RX with 2/4 lanes, 4 streams, external D-PHY

0 Fabiana Jaimes 12 days ago in reply to csscyt

TI__Genius 10020 points

Hi,

Please expect a delay in response as I am out of office.

Thank you,

Fabiana

0 csscyt 9 days ago in reply to Fabiana Jaimes

Prodigy 30 points

i checked the hardware configutation and now imx477 works.

 
 root@am68a-sk:/opt/edgeai-gst-apps# 
  CSI Camera 0 detected
    device = /dev/video-rpi-cam0
    name = imx477
    format = [fmt:SRGGB12_1X12/2028x1080]
    subdev_id = /dev/v4l-rpi-subdev0
    isp_required = yes
[   15.576636] sh (986): drop_caches: 1
root@am68a-sk:/opt/edgeai-gst-apps#

and check imx477 camera:

root@am68a-sk:/opt/edgeai-gst-apps#
root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes pad=0
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0,stream=0)
   0x3012: MEDIA_BUS_FMT_SRGGB12_1X12
   0x300f: MEDIA_BUS_FMT_SRGGB10_1X10
root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x300f
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0,stream=0)
   Size Range: 1332x990 - 1332x990
root@am68a-sk:/opt/edgeai-gst-apps#

   root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x3012
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0,stream=0)
   Size Range: 4056x3040 - 4056x3040
   Size Range: 2028x1520 - 2028x1520
   Size Range: 2028x1080 - 2028x1080

Modify the /opt/edgeai-gst-apps/scripts/setup_cameras.sh

setup_imx477(){
    IMX477_CAM_FMT='[fmt:SRGGB12_1X12/2028x1080]' 


 if runs the edgeai-gst-app, select camera it will crash



the crash log:
[   32.371914] ------------[ cut here ]------------
[   32.376529] WARNING: CPU: 1 PID: 1235 at drivers/media/common/videobuf2/videobuf2-core.c:1647 vb2_start_streaming+0xd8/0x15c [videobuf2_common]
[   32.389388] Modules linked in: xt_conntrack xt_MASQUERADE iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_addrtype iptable_filter ip_tables x_tables br_netfilter bridge stp )
[   32.389507]  fuse drm drm_panel_orientation_quirks ipv6
[   32.481655] CPU: 1 PID: 1235 Comm: v4l2src0:src Tainted: G           O       6.1.46 #1
[   32.489552] Hardware name: Texas Instruments AM68 SK (DT)
[   32.494933] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   32.501876] pc : vb2_start_streaming+0xd8/0x15c [videobuf2_common]
[   32.508047] lr : vb2_start_streaming+0x68/0x15c [videobuf2_common]
[   32.514214] sp : ffff80000b4cbb60
[   32.517514] x29: ffff80000b4cbb60 x28: ffff000832059400 x27: ffff000830ea14b8
[   32.524632] x26: 0000000000000000 x25: 0000000040045612 x24: 0000000000000000
[   32.531750] x23: ffff80000b4cbcc8 x22: ffff00082f479700 x21: ffff000830ea1438
[   32.538868] x20: ffff000830ea1450 x19: 00000000ffffffa1 x18: 00000000002feac5
[   32.545986] x17: 0000000000000020 x16: fffffc002405f808 x15: 0000000000000000
[   32.553104] x14: 000000000000025d x13: 0000000000000000 x12: 0000000000000000
[   32.560222] x11: 0000000000000000 x10: 00000000000009b0 x9 : ffff80000b4cba30
[   32.567339] x8 : ffff00084204df90 x7 : ffff000b7e1b3340 x6 : 0000000000000000
[   32.574457] x5 : 00000000410fd080 x4 : 0000000000c0000e x3 : ffff00082eddaa48
[   32.581574] x2 : 0000000000000000 x1 : ffff800000e6d000 x0 : ffff00084213c9f0
[   32.588693] Call trace:
[   32.591127]  vb2_start_streaming+0xd8/0x15c [videobuf2_common]
[   32.596951]  vb2_core_streamon+0x90/0x190 [videobuf2_common]
[   32.602598]  vb2_ioctl_streamon+0x5c/0xb0 [videobuf2_v4l2]
[   32.608078]  v4l_streamon+0x24/0x30 [videodev]
[   32.612534]  __video_do_ioctl+0x18c/0x3dc [videodev]
[   32.617504]  video_usercopy+0x21c/0x6d0 [videodev]
[   32.622307]  video_ioctl2+0x18/0x30 [videodev]
[   32.626758]  v4l2_ioctl+0x40/0x60 [videodev]
[   32.631034]  __arm64_sys_ioctl+0xa8/0xf0
[   32.634948]  invoke_syscall+0x48/0x114
[   32.638687]  el0_svc_common.constprop.0+0xd4/0xfc
[   32.643377]  do_el0_svc+0x30/0xd0
[   32.646680]  el0_svc+0x2c/0x84
[   32.649725]  el0t_64_sync_handler+0xbc/0x140
[   32.653981]  el0t_64_sync+0x18c/0x190
[   32.657631] ---[ end trace 0000000000000000 ]---
[   34.119055] cdns-csi2rx 4504000.csi-bridge: Failed to start streams 0x1 on subdev
[   34.136691] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream0
[   34.152889] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream1
[   34.169118] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream2
[   34.185300] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream3
[   34.196974] ------------[ cut here ]------------
[   34.201593] WARNING: CPU: 1 PID: 1248 at drivers/media/common/videobuf2/videobuf2-core.c:1647 vb2_start_streaming+0xd8/0x15c [videobuf2_common]
[   34.214455] Modules linked in: xt_conntrack xt_MASQUERADE iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_addrtype iptable_filter ip_tables x_tables br_netfilter bridge stp )
[   34.214582]  fuse drm drm_panel_orientation_quirks ipv6
[   34.306733] CPU: 1 PID: 1248 Comm: v4l2src0:src Tainted: G        W  O       6.1.46 #1
[   34.314631] Hardware name: Texas Instruments AM68 SK (DT)
[   34.320011] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   34.326954] pc : vb2_start_streaming+0xd8/0x15c [videobuf2_common]
[   34.333125] lr : vb2_start_streaming+0x68/0x15c [videobuf2_common]
[   34.339294] sp : ffff80000b7a3b60
[   34.342594] x29: ffff80000b7a3b60 x28: ffff000830ca4900 x27: ffff000830ea14b8
[   34.349713] x26: 0000000000000000 x25: 0000000040045612 x24: 0000000000000000
[   34.356831] x23: ffff80000b7a3cc8 x22: ffff000833959f00 x21: ffff000830ea1438
[   34.363949] x20: ffff000830ea1450 x19: 00000000ffffffa1 x18: 00000000002fb867
[   34.371067] x17: 0000000000000020 x16: fffffc002405f808 x15: 0000003deadedbfe
[   34.378185] x14: 0000000000000322 x13: 0000000000000000 x12: 0000000000000000
[   34.385302] x11: 0000000000000000 x10: 00000000000009b0 x9 : ffff80000b7a3a30
[   34.392420] x8 : ffff00082d78a690 x7 : ffff000b7e1b3340 x6 : 0000000000000000
[   34.399537] x5 : 00000000410fd080 x4 : 0000000000c0000e x3 : ffff00082edda348
[   34.406655] x2 : 0000000000000000 x1 : ffff800000e6d000 x0 : ffff00084214b9f0
[   34.413773] Call trace:
[   34.416207]  vb2_start_streaming+0xd8/0x15c [videobuf2_common]
[   34.422030]  vb2_core_streamon+0x90/0x190 [videobuf2_common]
[   34.427677]  vb2_ioctl_streamon+0x5c/0xb0 [videobuf2_v4l2]
[   34.433155]  v4l_streamon+0x24/0x30 [videodev]
[   34.437612]  __video_do_ioctl+0x18c/0x3dc [videodev]
[   34.442584]  video_usercopy+0x21c/0x6d0 [videodev]
[   34.447380]  video_ioctl2+0x18/0x30 [videodev]
[   34.451830]  v4l2_ioctl+0x40/0x60 [videodev]
[   34.456106]  __arm64_sys_ioctl+0xa8/0xf0
[   34.460020]  invoke_syscall+0x48/0x114
[   34.463759]  el0_svc_common.constprop.0+0xd4/0xfc
[   34.468450]  do_el0_svc+0x30/0xd0
[   34.471754]  el0_svc+0x2c/0x84
[   34.474799]  el0t_64_sync_handler+0xbc/0x140
[   34.479057]  el0t_64_sync+0x18c/0x190
[   34.482707] ---[ end trace 0000000000000000 ]---

Could you help me confirm which step is wrong or what steps are still missing? thanks

0 Praveen Rao 9 days ago in reply to csscyt

TI__Mastermind 30111 points

Our expert handling this topic is currently out of the office; please expect a 1-2 day delay in response.

Thanks.

0 Fabiana Jaimes 8 days ago in reply to csscyt

TI__Genius 10020 points

Hello,

Try running the below command to grab a frame and share the output:

v4l2-ctl --verbose --device /dev/video-rpi-cam0 --set-fmt-video=width=2028,height=1080 --stream-mmap --stream-to=test-frame.raw --stream-count=1

Thanks,

Fabiana

0 csscyt 2 days ago in reply to Fabiana Jaimes

Prodigy 30 points

I used SDK version: ti-processor-sdk-linux-edgeai-j721s2-evm-09_02_00_05

port imx477.c from raspberrypi
and connected two cameras(imx219 and imx477),

The driver is probed and I2C communication works,
see the log:

root@am68a-sk:/opt/edgeai-gst-apps# dmesg | grep -i imx
[    4.575593] imx219 5-0010: supply VANA not found, using dummy regulator
[    4.592375] imx219 5-0010: supply VDIG not found, using dummy regulator
[    4.592410] imx477 4-001a: supply dovdd not found, using dummy regulator
[    4.607551] imx219 5-0010: supply VDDL not found, using dummy regulator
[    4.614484] imx477 4-001a: supply avdd not found, using dummy regulator
[    4.621566] imx477 4-001a: supply dvdd not found, using dummy regulator

[    7.145908] imx477 4-001a: Consider updating driver imx477 to match on endpoints
[    7.506788] imx219 5-0010: Consider updating driver imx219 to match on endpoints



IMX219 Camera 1 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB8_1X8/1920x1080]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes
IMX477 Camera 0 detected
    device = /dev/video-imx477-cam0
    name = imx477
    format = [fmt:SRGGB10_1X10/1332x990]
    subdev_id = /dev/v4l-imx477-subdev0
    isp_required = yes

imx219 camera can work，

but when i select IMX477 camera，there is no display and no incorrect log output

if I use v4l2-ctl command to grab a frame，they will stuck and can not return, Also including imx219

imx477:
root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl --verbose --device /dev/video-imx477-cam0  --set-fmt-video=width=1332,height=990 --stream-mmap --stream-to=test-frame.raw --stream-count=1                     
VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
   Width/Height      : 1328/990
   Pixel Format      : 'UYVY' (UYVY 4:2:2)
   Field             : None
   Bytes per Line    : 2656
   Size Image        : 2629440
   Colorspace        : sRGB
   Transfer Function : Default (maps to sRGB)
   YCbCr/HSV Encoding: Default (maps to ITU-R 601)
   Quantization      : Default (maps to Limited Range)
   Flags             : 
        VIDIOC_REQBUFS returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_STREAMON returned 0 (Success)
        
    Stuck here!!!!
  
  
  
    
    imx219:
    
    root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl --verbose --device /dev/video-imx219-cam0  --set-fmt-video=width=2028,height=1080 --stream-mmap --stream-to=test-frame.raw --stream-count=1                    
VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
   Width/Height      : 2024/1080
   Pixel Format      : 'UYVY' (UYVY 4:2:2)
   Field             : None
   Bytes per Line    : 4048
   Size[  244.125984] imx219 5-0010: imx219_start_streaming
 Image        : 4371840
   Colorspace        : sRGB
   Transfer Function : Default (maps to sRGB)
   YCbCr/HSV Encoding: Default (maps to ITU-R 601)
   Quantization      : Default (maps to Limited Range)
   Flags             : 
        VIDIOC_REQBUFS returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_STREAMON returned 0 (Success)
        
    stuck here

Do you know the reason why imx477 doesn't work and how to debug this issue

and does TI have plan to support IMX477 in the subsequent versions?

0 Fabiana Jaimes 2 days ago in reply to csscyt

TI__Genius 10020 points

Hello,

If the camera only produces raw RGB or RGBD images, then further ISP processing is required which can be handled by the Vision Imaging Sub System (VISS) hardware accelerator. For more on ISP tuning, please see the following application note: https://www.ti.com/lit/an/sprad86a/sprad86a.pdf

We are still discussing internally about looking into IMX477 or another similar sensor and if it is something we can support soon.

Thank you,

Fabiana

0 csscyt 1 day ago in reply to Fabiana Jaimes

Prodigy 30 points

Thank you for your reply.

followed the ISP tuning guide,
I have copied the generated DCC binary files to the target board,

/opt/imaging/imx477/linear/dcc_2a.bin

/opt/imaging/imx477/linear/dcc_ldc.bin

/opt/imaging/imx477/linear/dcc_viss.bin

if select the IMX477 camera via the below edgeAI GUI,

Do I need to set the path of the DCC file, that is,how can the GUI find the newly added DCC file of imx477

If it is needed, where to config it?

+1 Fabiana Jaimes 1 day ago in reply to csscyt

TI__Genius 10020 points

Hello,

The way the custom gui application works is by generating a yaml/configuration file to generate and run a GStreamer pipeline using the provided inputs. Because IMX477 is not supported out of box, the sensor id for this sensor is missing so the format cannot be set to rggb. When enabling a new sensor, you should test functionality by running GStreamer pipelines in the command line. Here is an example for IMX219:

gst-launch-1.0 v4l2src device=/dev/video-imx219-cam0 io-mode=5 ! queue leaky=2 ! \
video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb ! \
tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX219_RPI" \
dcc-isp-file=/opt/imaging/imx219/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/imx219/dcc_2a.bin format-msb=7 ! \
tiovxmultiscaler ! \
video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
kmssink driver-name=tidss sync=false

Thank you,

Fabiana

0 csscyt 8 hours ago in reply to Fabiana Jaimes

Prodigy 30 points

These are my two camera devices(media0 is IMX477, media1 is IMX219)

root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl --list-devices
j721e-csi2rx (platform:4500000.ticsi2rx):
        /dev/video2
        /dev/video3
        /dev/video4
        /dev/video5
        /dev/video6
        /dev/video7
        /dev/video8
        /dev/video9
        /dev/media0

j721e-csi2rx (platform:4510000.ticsi2rx):
        /dev/video10
        /dev/video11
        /dev/video12
        /dev/video13
        /dev/video14
        /dev/video15
        /dev/video16
        /dev/video17
        /dev/media1

wave5-dec (platform:wave5-dec):
        /dev/video0

wave5-enc (platform:wave5-enc):
        /dev/video1

check the media0 device topology,

root@am68a-sk:/opt/edgeai-gst-apps# media-ctl -p -d /dev/media0
Media controller API version 6.1.80

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial          
bus info        platform:4500000.ticsi2rx
hw revision     0x1
driver version  6.1.80

Device topology
- entity 1: 4500000.ticsi2rx (9 pads, 9 links, 1 route)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "cdns_csi2rx.4504000.csi-bridge":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "4500000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
        pad2: Source
                -> "4500000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
        pad3: Source
                -> "4500000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
        pad4: Source
                -> "4500000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
        pad5: Source
                -> "4500000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
        pad6: Source
                -> "4500000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]
        pad7: Source
                -> "4500000.ticsi2rx context 6":0 [ENABLED,IMMUTABLE]
        pad8: Source
                -> "4500000.ticsi2rx context 7":0 [ENABLED,IMMUTABLE]

- entity 11: cdns_csi2rx.4504000.csi-bridge (5 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:SRGGB10_1X10/4056x3040 field:none colorspace:raw xfer:none]
                <- "imx477 4-001a":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:SRGGB10_1X10/4056x3040 field:none colorspace:raw xfer:none]
                -> "4500000.ticsi2rx":0 [ENABLED,IMMUTABLE]
        pad2: Source
        pad3: Source
        pad4: Source

- entity 17: imx477 4-001a (1 pad, 1 link, 0 route)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev2
        pad0: Source
                [stream:0 fmt:SRGGB10_1X10/4056x3040 field:none colorspace:raw xfer:none]
                -> "cdns_csi2rx.4504000.csi-bridge":0 [ENABLED,IMMUTABLE]

- entity 23: 4500000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video2
        pad0: Sink
                <- "4500000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 29: 4500000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video3
        pad0: Sink
                <- "4500000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 35: 4500000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video4
        pad0: Sink
                <- "4500000.ticsi2rx":3 [ENABLED,IMMUTABLE]

- entity 41: 4500000.ticsi2rx context 3 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video5
        pad0: Sink
                <- "4500000.ticsi2rx":4 [ENABLED,IMMUTABLE]

- entity 47: 4500000.ticsi2rx context 4 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video6
        pad0: Sink
                <- "4500000.ticsi2rx":5 [ENABLED,IMMUTABLE]

- entity 53: 4500000.ticsi2rx context 5 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video7
        pad0: Sink
                <- "4500000.ticsi2rx":6 [ENABLED,IMMUTABLE]

- entity 59: 4500000.ticsi2rx context 6 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video8
        pad0: Sink
                <- "4500000.ticsi2rx":7 [ENABLED,IMMUTABLE]

- entity 65: 4500000.ticsi2rx context 7 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video9
        pad0: Sink
                <- "4500000.ticsi2rx":8 [ENABLED,IMMUTABLE]

check the media1 device topology,

root@am68a-sk:/opt/edgeai-gst-apps# media-ctl -p -d /dev/media1
Media controller API version 6.1.80

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial          
bus info        platform:4510000.ticsi2rx
hw revision     0x1
driver version  6.1.80

Device topology
- entity 1: 4510000.ticsi2rx (9 pads, 9 links, 1 route)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev3
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "cdns_csi2rx.4514000.csi-bridge":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "4510000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
        pad2: Source
                -> "4510000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
        pad3: Source
                -> "4510000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
        pad4: Source
                -> "4510000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
        pad5: Source
                -> "4510000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
        pad6: Source
                -> "4510000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]
        pad7: Source
                -> "4510000.ticsi2rx context 6":0 [ENABLED,IMMUTABLE]
        pad8: Source
                -> "4510000.ticsi2rx context 7":0 [ENABLED,IMMUTABLE]

- entity 11: cdns_csi2rx.4514000.csi-bridge (5 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev4
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:SRGGB8_1X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                <- "imx219 5-0010":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:SRGGB8_1X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "4510000.ticsi2rx":0 [ENABLED,IMMUTABLE]
        pad2: Source
        pad3: Source
        pad4: Source

- entity 17: imx219 5-0010 (1 pad, 1 link, 0 route)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev5
        pad0: Source
                [stream:0 fmt:SRGGB8_1X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
                 crop.bounds:(8,8)/3280x2464
                 crop:(688,700)/1920x1080]
                -> "cdns_csi2rx.4514000.csi-bridge":0 [ENABLED,IMMUTABLE]

- entity 23: 4510000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video10
        pad0: Sink
                <- "4510000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 29: 4510000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video11
        pad0: Sink
                <- "4510000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 35: 4510000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video12
        pad0: Sink
                <- "4510000.ticsi2rx":3 [ENABLED,IMMUTABLE]

- entity 41: 4510000.ticsi2rx context 3 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video13
        pad0: Sink
                <- "4510000.ticsi2rx":4 [ENABLED,IMMUTABLE]

- entity 47: 4510000.ticsi2rx context 4 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video14
        pad0: Sink
                <- "4510000.ticsi2rx":5 [ENABLED,IMMUTABLE]

- entity 53: 4510000.ticsi2rx context 5 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video15
        pad0: Sink
                <- "4510000.ticsi2rx":6 [ENABLED,IMMUTABLE]

- entity 59: 4510000.ticsi2rx context 6 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video16
        pad0: Sink
                <- "4510000.ticsi2rx":7 [ENABLED,IMMUTABLE]

- entity 65: 4510000.ticsi2rx context 7 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video17
        pad0: Sink
                <- "4510000.ticsi2rx":8 [ENABLED,IMMUTABLE]

for IMX219, the following command works,

gst-launch-1.0  v4l2src device=/dev/video-imx219-cam0 io-mode=5 ! queue leaky=2 ! \
video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sensor-name="SENSOR_SONY_IMX219_RPI" \
dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a.bin format-msb=7 ! \
tiovxmultiscaler ! \
video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
kmssink driver-name=tidss sync=false

for IMX477, I made the software changes followed the ISP tuning guide, and use the following command, it does't work,

could you help check this command, "tiovxisp sink_0::device=/dev/v4l-subdev1" it is correct accoring to

the above device tolopoly?

gst-launch-1.0  v4l2src device=/dev/video-imx477-cam0 io-mode=5 ! queue leaky=2 ! \
video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb ! \
tiovxisp sink_0::device=/dev/v4l-subdev1 sensor-name="SENSOR_SONY_IMX477_RPI" \
dcc-isp-file=/opt/imaging/imx477/linear/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/imx477/linear/dcc_2a.bin format-msb=7 ! \
tiovxmultiscaler ! \
video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
kmssink driver-name=tidss sync=false

I enabled the gst-debug log and added some logs in gsttivoxisp.c and gsttiovxmiso.c,

compared the logs between IMX219 and IMX477,

for IMX477, the initialization function(gst_tiovx_isp_init_module) was not called,and it will stuck there

Could you help me check whether there are still any missing or wrong configurations? thanks.

0 Fabiana Jaimes 5 hours ago in reply to csscyt

TI__Genius 10020 points

Hi,

Could you try running the following configuration and share the output?

gst-launch-1.0  v4l2src device=/dev/video-imx477-cam0 io-mode=5 ! queue leaky=2 ! \
video/x-bayer, width=1332, height=990, framerate=30/1, format=rggb10 ! \
tiovxisp sink_0::device=/dev/v4l-imx477-subdev0 sensor-name="SENSOR_SONY_IMX477_RPI" \
dcc-isp-file=/opt/imaging/imx477/linear/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/imx477/linear/dcc_2a.bin format-msb=7 ! \
tiovxmultiscaler ! \
video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
kmssink driver-name=tidss sync=false

Thank you,

Fabiana

Processors

Processors forum

SK-AM68: port imx477 to SK-AM68

Enable A New CSI-2 Sensor