This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SK-AM62A-LP: Need guidance on writing DTS overlay for ov5647 (and other sensors)

Part Number: SK-AM62A-LP
Other Parts Discussed in Thread: AM62A7

Hello,

We're working on getting the ov5647 sensor up and running on the AM62A evk for some tests, but we've hit a few bumps. We got the driver and .dtbo compiled and loaded onto the evk using uboot, but no MIPI CSI-2 interfaces are showing up. At first, we thought it could be the clock speed, but after looking into it, the 25 MHz we're using seems right for the ov5647.  I've added my .dts code below. Can you check it out and let me know if you see any problems? A lot of the image sensors we plan to use, like the IMX415, aren't officially supported, so we want to be sure we're not missing anything important or messing up as we try to bring new ones online.

// SPDX-License-Identifier: GPL-2.0

/*
* Copyright (C) 2023 Texas Instruments Incorporated - https://www.ti.com/
*/

/dts-v1/;
/plugin/;

#include <dt-bindings/gpio/gpio.h>

/ {
fragment@101 {
target-path = "/";

__overlay__ {
clk_ov5647_fixed: ov5647-xclk {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <25000000>;
};
};
};
};

&main_i2c2 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";

i2c-switch@71 {
compatible = "nxp,pca9543";
#address-cells = <1>;
#size-cells = <0>;
reg = <0x71>;

/* CAM port */
i2c@1 {
#address-cells = <1>;
#size-cells = <0>;
reg = <1>;

ov5647: camera@36 {
compatible = "ovti,ov5647";
reg = <0x36>;

clocks = <&clk_ov5647_fixed>;
clock-names = "xclk";

reset-gpios = <&exp1 13 GPIO_ACTIVE_LOW>;

port {
csi2_cam0: endpoint {
remote-endpoint = <&csi2rx0_in_sensor>;
//link-frequencies = /bits/ 64 <456000000>;
clock-lanes = <0>;
data-lanes = <1 2>;
};
};
};
};
};
};

&csi0_port0 {
status = "okay";

csi2rx0_in_sensor: endpoint {
remote-endpoint = <&csi2_cam0>;
bus-type = <4>; /* CSI2 DPHY. */
clock-lanes = <0>;
data-lanes = <1 2>;
};
};

  • Hi Narayan,

    Thanks for your query.

    Seems like I would need a good background of what you are trying to achieve here. Although its mostly related to changes in the device tree source file, looks like you are creating device tree overlay which will be added on top of DTS and later provided to the kernel. I am assuming you know the steps for it.(I will also look into it)

    But a better approach would be to simply modify the dts file and check if its working as intended. Once you are sure its working, then you can make your own overlay files and use them.

    So right now can you do the following ?

    • Since I don't have ov5647 sensors(and other sensors), so I will be simply able to review the device tree source/overlay file.
    • Instead of using the overlay file, can you do the following ?
      • Instead of creating the device tree overlay file, can you put the changes in the original DTS file(k3-am62a7-sk.dts)
      • Compile dts to dtb using dtc
      • Check if the changes are working fine.
    • Can you send me the file? Instead of pasting its content above.

    Regards,

    Vaibhav

  • Hi Vaibhav,

    We are using the following tutorial https://dev.ti.com/tirex/explore/node?node=A__AIwN3jgf3NkdPtMYdoNKKw__AM62A-ACADEMY__WeZ9SsL__LATEST

    I based the DTS off of the imx219 dts for the AM62A that's already included in the SDK.  We just changed addressed and clock speed to accommodate the ov5647.  Is this not the correct method? I will send you the dtb as well.

    Best,

    Narayan

  • Hi Narayana,

    I think you are talking about the following.

    I am going through this and will let you know shortly.

    Also it would be nice if you could hand me the DTB file. I would probably convert that to DTS and check.

    Regards,

    Vaibhav

  • Hi Narayana,

    I have gone through the steps mentioned in the Academy for AM62A.

    I just wanted to confirm one thing.

    Are you able to verify the sensor using utilities like “v4l2-ctl” and “media-ctl”  ?

    Apart from this as mentioned earlier as well, it would be nice to have the DTB file as well. So looking forward to this.

    Regards,

    Vaibhav

  • Hi Vaibhav,

    I apologize for the late response.  When testing the camera, the command "v4l2-ctl --list-devices"  doesn't actually show any connected devices when using the OV5647 dtbo file.  I know the sensor is functional, as we've tested it on a raspberry pi using libcamera, and it worked fine. 

    Regarding the dtb file, should I send you the code via DM, or should I email it to you?

    Best,

    Narayan

  • Hi Narayan,

    When testing the camera, the command "v4l2-ctl --list-devices"  doesn't actually show any connected devices when using the OV5647 dtbo file.

    Unexpected, lets see the DTB file.

    Please send me the file over direct message.

    Looking forward to it.

    Regards,

    Vaibhav

  • Hi Vaibhav,

    I have attached the zip file which contains the dts file for ov5647, its corresponding dtbo file, and compiled kernel object for ov5647 (ov5647.ko).

    ti_ov5647.zip

    After compiling the code, I did following steps to install kernel modules:

          $ sudo cp arch/arm64/boot/Image <path to root of file system>/boot

          $ sudo cp arch/arm64/boot/dts/ti/k3-am62a7-sk-csi2-ov5647.dtbo <path to root of file system>/boot

          $ sudo make INSTALL_MOD_PATH=<path to root of file system> modules_install

    I set the dts overlay in boot mode using following command:

         =>setenv name_overlays k3-am62a7-sk-csi2-ov5647.dtbo
         =>saveenv
          =>boot

    After executing above steps, when I issued command "v4l2-ctl --list-devices", I got following messages:

    root@am62axx-evm:/opt/edgeai-gst-apps# v4l2-ctl --list-devices
    TI-CSI2RX (platform:30102000.ticsi2rx):
    /dev/media0

    wave5-dec (platform:wave5-dec):
    /dev/video0

    wave5-enc (platform:wave5-enc):
    /dev/video1

     

    Also, "lsmod" didn't even list ov5647 module. Following is the response for "lsmod".

    I wonder if I have missed any steps which is preventing system from loading dts overlay and kernel module for ov5647.

    ov5647 camera we are using is bought from following link:

    https://www.amazon.com/dp/B07RXKZ1KN?ref=ppx_yo2ov_dt_b_product_details&th=1

    Thanks,

    Bipin

  • Hi Narayan,

    Many thanks for the detailed explanation.

    I think it would be helpful if I recreate this issue on my end with a sensor.

    Please allow me few business days to get back to you with updates.

    Regards,

    Vaibhav

  • Hi Narayan,

    Your device tree looks good. Can you do a "dmesg | grep ov5647" after you boot? Does it show some errors like below?

     ov5647 4-0036: write sensor_oe_enable_regs error
     ov5647: probe of 4-0036 failed with error -121

    I've tested this camera module of OV5647 and it worked fine on my board. Do you want to buy one and give it a try?

    Regards,

    Jianzhong 

  • Hello Jianzhong,

    I was able to get a video stream from ov5647 on AM62A7 by following your responses in following E2E thread:

    I was able to see video stream in the display successfully using following command:
    GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element+pipeline)" gst-launch-1.0 v4l2src io-mode=5 device=/dev/video2 ! video/x-bayer, width=1920, height=1080, format=bggr10, framerate=30/1 ! tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name=SENSOR_OV5647_RPI dcc-isp-file=/opt/imaging/ov5647/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/ov5647/dcc_2a.bin format-msb=9 ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! kmssink driver-name=tidss sync=false
    However, I tried to run edgeai-gst-apps using the ov5647 camera but the program exited immediately without displaying video.
    I modified input in rpiV2_cam_example.yaml and also added support for ov5647 in gstwrapper.py. I did the following modifications in gst_wrapper.py to support ov5647.
                if input.sen_id == "imx219":
                    sen_name = "SENSOR_SONY_IMX219_RPI"
                    format_msb = 7
                if input.sen_id == "ov5647":
                    sen_name = "SENSOR_OV5647_RPI"
                    format_msb = 9

                elif input.sen_id == "imx390":
                    sen_name = "SENSOR_SONY_IMX390_UB953_D3"
                    format_msb = 11
                elif input.sen_id == "ov2312":
                    sen_name = "SENSOR_OV2312_UB953_LI"
                    format_msb = 9
     
    Following modification is made  in rpiV2_cam_example.yaml:
       input0:
            source: /dev/video2
            width: 1920
            height: 1080
            format: rggb
            subdev-id: 2
            framerate: 30
            sen-id: ov5647
    Please find the attachment for modified files.
    Following is the output received in the terminal upon executing program:
    root@am62axx-evm:/opt/edgeai-gst-apps/apps_python# python3 app_edgeai.py ../configs/rpiV2_cam_example.yaml
    libtidl_onnxrt_EP loaded 0x87e9d00
    Final number of subgraphs created are : 1, - Offloaded Nodes - 289, Total Nodes - 289
    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=4) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
      6592.014590 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
      6592.014699 s:  VX_ZONE_INIT:Enabled
      6592.014711 s:  VX_ZONE_ERROR:Enabled
      6592.014720 s:  VX_ZONE_WARNING:Enabled
      6592.015395 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
      6592.015569 s:  VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
    ==========[INPUT PIPELINE(S)]==========

    [PIPE-0]

    v4l2src device=/dev/video2 io-mode=5 pixel-aspect-ratio=None ! queue leaky=2 ! capsfilter caps="video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb;" ! tiovxisp dcc-isp-file=/opt/imaging/ov5647/dcc_viss.bin sensor-name=SENSOR_OV5647_RPI format-msb=9 ! capsfilter caps="video/x-raw, format=(string)NV12;" ! tiovxmultiscaler name=split_01
    split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1280, height=(int)720;" ! tiovxdlcolorconvert out-pool-size=4 ! capsfilter caps="video/x-raw, format=(string)RGB;" ! appsink max-buffers=2 drop=True name=sen_0
    split_01. ! queue ! capsfilter caps="video/x-raw, width=(int)1168, height=(int)748;" ! tiovxmultiscaler target=1 ! capsfilter caps="video/x-raw, width=(int)416, height=(int)416;" ! tiovxdlpreproc out-pool-size=4 data-type=3 tensor-format=1 ! capsfilter caps="application/x-tensor-tiovx;" ! appsink max-buffers=2 drop=True name=pre_0


    ==========[OUTPUT PIPELINE]==========

    appsrc do-timestamp=True format=3 block=True name=post_0 ! tiovxdlcolorconvert ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720;" ! queue ! mosaic_0.sink_0

    tiovxmosaic target=1 background=/tmp/background_0 name=mosaic_0
    src::pool-size=3 sink_0::startx="<320>" sink_0::starty="<180>" sink_0::widths="<1280>" sink_0::heights="<720>"
    ! capsfilter caps="video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080;" ! queue ! tiperfoverlay title=RPi Camera ! kmssink sync=False max-lateness=5000000 qos=True processing-deadline=15000000 driver-name=tidss connector-id=39 plane-id=31 force-modesetting=True

      6597.850577 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:107] De-Initialization Done for HOST !!!
      6597.851877 s:  VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 30 alloc's of 64839132 bytes
    DDR_SHARED_MEM: Free's : 30 free's  of 64839132 bytes
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes
    DDR_SHARED_MEM: Total size: 184549376 bytes
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    root@am62axx-evm:/opt/edgeai-gst-apps/apps_python#
    Is there anything I missed to modify in order to execute an application using ov5647?
  • Hello Narayan,

    Glad that the camera is working.

    Following modification is made  in rpiV2_cam_example.yaml:
       input0:
            source: /dev/video2
            width: 1920
            height: 1080
            format: rggb

    The format should be bggr10.

    v4l2src device=/dev/video2 io-mode=5 pixel-aspect-ratio=None ! queue leaky=2 ! capsfilter caps="video/x-bayer, width=(int)1920, height=(int)1080, format=(string)rggb;" ! tiovxisp dcc-isp-file=/opt/imaging/ov5647/dcc_viss.bin sensor-name=SENSOR_OV5647_RPI format-msb=9 !

    The format in the pipeline should be bggr10.

    Can you try again with bggr10 and see if it works?

    Regards,

    Jianzhong

  • Thanks. It worked.

  • Hi Narayan,

    Thanks for the update. Glad that it's working for you now. I'll close this thread for now. If you need any other help, please open a new one.

    Regards,

    Jianzhong

  • HI Narayan,

    Please use the attached source code for OV5647 auto-exposure setting. 3632.gsttiovxisp.c

    Replace /opt/edgeai-gst-plugins/ext/tiovx/gsttiovxisp.c with the attached code. Then run /opt/edgeai-gst-apps/scripts/install_gst_plugins.sh.

    Regards,

    Jianzhong