SK-AM68: Adding a new camera sensor

It seems I can't attach a DTSO file to the post, so here it is in full

// SPDX-License-Identifier: GPL-2.0
/**
 * DT Overlay for RPi Camera V3 (Sony IMX708) interfaced with CSI2 on AM68-SK board - copied and adjusted from the IMX219 version by Dan
 * datasheets.raspberrypi.com/.../camera-module-3-schematics.pdf
 */

/dts-v1/;
/plugin/;

#include <dt-bindings/gpio/gpio.h>
#include "k3-pinctrl.h"

&{/} {
  clk_imx708_fixed: imx708-inclk {
    status = "okay";
    compatible = "fixed-clock";
    #clock-cells = <0>;
    clock-frequency = <24000000>;
  };
};

&exp3 {
  p01-hog {
    /* CSI_MUX_SEL_2 */
    gpio-hog;
    gpios = ;
    output-high;
    line-name = "CSI_MUX_SEL_2";
  };
};


&main_i2c1 {
  status = "okay";
  #address-cells = ;
  #size-cells = ;

  i2c-switch@70 {
    compatible = "nxp,pca9543";
    #address-cells = ;
    #size-cells = ;
    reg = ;

    i2c-alias-pool = /bits/ 16 ;

    /* CAM0 I2C */
    cam0_i2c: i2c@0 {
      #address-cells = ;
      #size-cells = ;
      reg = ;

      imx708_0: imx708_0@1a {
        compatible = "sony,imx708";
        reg = ;
        status = "disabled";

        clocks = <&clk_imx708_fixed>;
        clock-names = "inclk";

        rotation = ;
        orientation = ;

        reset-gpios = <&exp3 3 GPIO_ACTIVE_HIGH>;

        port {
          csi2_cam0: endpoint {
            remote-endpoint = <&csi2rx0_in_sensor>;
            link-frequencies = /bits/ 64 ;
            clock-lanes = ;
            data-lanes = ;
          };
        };
      };
    };

    /* CAM1 I2C */
    cam1_i2c: i2c@1 {
      #address-cells = ;
      #size-cells = ;
      reg = ;

      imx708_1: imx708_1@1a {
        compatible = "sony,imx708";
        reg = ;

        clocks = <&clk_imx708_fixed>;
        clock-names = "inclk";

        rotation = ;
        orientation = ;

        reset-gpios = <&exp3 3 GPIO_ACTIVE_HIGH>;

        port {
          csi2_cam1: endpoint {
            remote-endpoint = <&csi2rx1_in_sensor>;
            link-frequencies = /bits/ 64 ;
            clock-lanes = ;
            data-lanes = ;
          };
        };
      };
    };
  };
};

&csi0_port0 {
  status = "okay";
  csi2rx0_in_sensor: endpoint {
    remote-endpoint = <&csi2_cam0>;
    bus-type = ; /* CSI2 DPHY. */
    clock-lanes = ;
    data-lanes = ;
  };
};

&csi1_port0 {
  status = "okay";
  csi2rx1_in_sensor: endpoint {
    remote-endpoint = <&csi2_cam1>;
    bus-type = ; /* CSI2 DPHY. */
    clock-lanes = ;
    data-lanes = ;
  };
};

Hi Dan,

I would suggest following this guide to enable a new CSI-2 sensor. This is for AM62Ax devices but the idea is the same and are working on creating one for SK-AM68.

Could you share what changes were made to the imx708 dt files from the imx219 files besides swapping the names?

Dan Parnham said:

imx708_0: imx708_0@1a { compatible = "sony,imx708"; reg = ; status = "disabled";

What is the reason for having this status set to disabled?

Thank you,

Fabiana

Hi Fabiana,

Thanks for the reply. The guide you suggested is the exact one I followed since it was the only documentation I could find related to adding a new sensor.

The changes to the DTSO were as follows:

1. The clk section at the top was renamed
2. The compatible string was changed to refer to the imx708 kernel module.
3. Changed from "reg = <0x10>" to "reg=<0x1a>" in the imx708_0 and imx708_1 nodes (based on the DTSI in the rpi kernel).
4. Changed the clock-names from "xclk" to "inclk" since that is a difference I found in the i2c kernel modules.
5. Changed the link frequencies for the CSI endpoints from 456000000 to 450000000 since that is what was in the DTSI for the rpi kernel configuration.
   

The `status="disabled"` was left in accidentally from one of my many attempts to modify the file - but it doesn't work with that removed either.

Kind regards,

Dan

Hi Dan,

Thank you for sharing! It would also be helpful if you could send the driver as well as include the values that are missing in the dtso file you shared to better understand what you're currently working with.

Thank you,

Fabiana

Hi Fabiana,

This is the i2c driver that I incorporated into the AM68 kernel build using the TI sdk: https://github.com/raspberrypi/linux/blob/rpi-6.1.y/drivers/media/i2c/imx708.c

Apologies, I hadn't noticed that bits of the dtso were missing - it appears that the forum software makes pasting examples with triangular brackets rather difficult. Since it won't let me attach the dtso file either I've zipped it up instead:

k3-am68-sk-bb-rpi-cam-imx708.dtso.zip

Kind regards,

Dan

Hi again,

I think we have the i2c driver working now :). Having looked over the schematics for the rpi cam module and the documentation for the I2C GPIO Expander Table from the AM68-SK documentation we needed to set the voltage to 3.3V by adding

p00-hog {
  /* I2C voltage - set to 3.3V */
  gpio-hog;
  gpios = <0 GPIO_ACTIVE_HIGH>;
  output-high;
  line-name = "CSI_VIO_SEL";
};

to the "&exp3" section in the overlay. And now in dmesg we see that it has successfully read the chip ID

Hi Dan,

I'm glad you have a working driver! If your sensor produces only raw RGB or RGBD images, it will require further ISP processing which can be handled by the Vision Imaging Sub System (VISS) hardware accelerator. This will involve ISP/sensor tuning activity to produce artifacts that can be understood by the VISS hardware. Your pipeline would look more like the IMX219 pipeline:

gst-launch-1.0 v4l2src device=/dev/video-rpi-cam0 io-mode=5 ! queue leaky=2 ! video/x-bayer, width=1920, height=1080, format=rggb ! tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss.bin format-msb=7 sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a.bin sink_0::device=/dev/v4l-rpi-subdev0 ! video/x-raw,format=NV12, width=1920, height=1080 ! queue ! kmssink driver-name=tidss sync=false

To perform ISP tuning for a raw camera, please read this ISP tuning guide: AM6xA ISP Tuning Guide (ti.com)

I hope this helps!

Thank you,

Fabiana

Hi Fabiana,

Thanks, but as described in my previous message whilst it looks like the i2c is working now, we're unable to actually grab any data from the sensor. There's no point configuring the ISP layers until we can actually get raw data from the chip. When using a fakesink with gstreamer or --stream-to with v4l2-ctl both fail at the v4l2 capture stage and I'm unable to figure out why. 

The "VIDIOC_STREAMON returned -1 (Operation not supported)" error message reveals very little when searching on google.

The "ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory" produces more search results but nothing that has been able to help me understand the issue so far.

I see these errors in the kernel log when trying to capture

[  172.279031] cdns-csi2rx 4504000.csi-bridge: Failed to start streams 0x1 on subdev
[  172.296577] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream0
[  172.312779] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream1
[  172.328998] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream2
[  172.345184] cdns-csi2rx 4504000.csi-bridge: Failed to stop stream3

Kind regards,

Dan

Hi Dan,

I understand. So, was the only change you made between both driver and overlay files you shared just the below chunk of code to the "&exp3" section in the overlay?

p00-hog {
  /* I2C voltage - set to 3.3V */
  gpio-hog;
  gpios = <0 GPIO_ACTIVE_HIGH>;
  output-high;
  line-name = "CSI_VIO_SEL";
};

Please confirm this while I consult with my team on this.

Thank you,

Fabiana

Hi,

Yes that was the only change from the file I previously shared. But I've been examining the DTSO, schematics and register tables further and think there are a number of issues. 

Firstly the imx219 dtso that I based it on appears to use the 40-pin header and not the flex cable connections so I think the "p01-hog" section needs to be disabled since the imx708 is connected via flex.

Secondly, the imx708 has a definition for regulator supplies in the i2c module:

static const char * const imx708_supply_name[] = {
  /* Supplies can be enabled in any order */
  "vana1", /* Analog1 (2.8V) supply */
  "vana2", /* Analog2 (1.8V) supply */
  "vdig", /* Digital Core (1.1V) supply */
  "vddl", /* IF (1.8V) supply */
};

These are referred to in the DTS and DTSI for the raspberry pi kernel, but I'm unsure how to specify them for the am68-sk. The schematics for the camera suggest that those voltages are supplied on the camera module itself: https://datasheets.raspberrypi.com/camera/camera-module-3-schematics.pdf

Kind regards,

Dan

Hi Dan,

I am currently checking with our imaging experts to see if this issue may be due to a sensor resolution limitation. Expect a response from me later this week.

Thank you,

Fabiana