• State TI Thinks Resolved
  • Locked Locked
  • Replies 18 replies
  • Answers 1 answer
  • Subscribers 102 subscribers
  • Views 1626 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: Some errors when using IMX390 from oToBrite on Linux

Jay Sun
Expert 1005 points
Part Number: TDA4VM
Other Parts Discussed in Thread: DS90UB960-Q1

Hi Experts,

I am trying to use IMX390 from oToBrite on Linux but there are some errors in the system log.

The cause is probably from the absence of IMX390 sensor on the I2C bus.

I am wondering is there a better way to fix this issue like skipping communication between IMX390 and driver?

ub953 is at 0x18 and ISP is at 0x6C.

Thanks

Michael

  • 0
    TI__Mastermind 41733 points

    Hi Michael,

    For some context, the exact camera module that we used for testing the V4L2 driver is the following on SK-TDA4VM board: https://www.d3engineering.com/product/designcore-d3rcm-imx390-953-rugged-camera-module/

    For the absence of the camera on the I2C bus, is there a dtbo overlay applied for the camera (and not just for the fusion board), similar to the following? https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-sk-tda4vm/08_04_00/exports/docs/getting_started.html#imx390-raw-sensor

    Regards,

    Takuma

  • 0 in reply to Takuma Fujiwara
    Expert 1005 points

    Hi Takuma,

    The reason that IMX390 sensor is absent from I2C bus is its original setting from oToBrite. So, I think  the way to make this sensor, IMX390 from oToBrite, work is to modify ds90ub960 driver and dts. As I understood, this camera works without any extra setting to ISP and sensor. It could work by just setting streamon on ds90ub960 in visionapps. Do you have recommended method to modify the driver and dts?

    Here is my overlay setting:

    name_overlays=k3-j721e-fpdlink-cpb-fusion.dtbo k3-j721e-fpdlink-imx390-rcm-0-0.dtbo

    Here is camera module structure:

    Tanks

    Michael

  • 0 in reply to Jay Sun
    TI__Guru* 82176 points

    Hi Michael,

    Jay Sun said:
    As I understood, this camera works without any extra setting to ISP and sensor

    Could you please clarify how the sensor is being configured in the oToCAM264ISP-S119T camera?
    Is the ISP and sensor's i2c addresses being accessed by the serializer?

    Wanted to know if you are trying to communicate to the IMX390 sensor via the backchannel i2c of the deserializer as the I2C address of the IMX390 is not available to the external.

    Have you modified the rcm dts file as per the sensor being used? If yes, could you share the modified dts file?


    You could find the dts file in the filepath ${PSDK_LINUX}/board-support/linux-5.10.120+gitAUTOINC+95b90aa828-g95b90aa828/arch/arm64/boot/dts/ti/k3-j721e-fpdlink-imx390-rcm-0-0.dts
    Once changes are made you could build the dtb file with the below command

    cd ${PSDK_LINUX}

    make linux-dtbs

    Regards,
    Nikhil

  • 0 in reply to Nikhil Dasan
    Expert 1005 points

    Hi Nikhil,

    Nikhil Dasan said:
    Could you please clarify how the sensor is being configured in the oToCAM264ISP-S119T camera?

    The sensor is configured automatically by the build-in ISP in the camera. 

    Nikhil Dasan said:
    Is the ISP and sensor's i2c addresses being accessed by the serializer?

    Only ISP can be access by the deserializer.

    Nikhil Dasan said:
    Wanted to know if you are trying to communicate to the IMX390 sensor via the backchannel i2c of the deserializer as the I2C address of the IMX390 is not available to the external.

    Yes, I was trying to do what you said. 

    I think there are few things need to be modified:

    1. Modify driver ti,ds90ub960-q1 with fixed frame desc so that it does need to qurry sensor anymore.

    2. Change input format from RAW to UYVY

    Here is the dts file I modified:

    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    // SPDX-License-Identifier: GPL-2.0
    /*
    * IMX390 FPD-Link 3 Camera Module
    *
    * Copyright (c) 2021 Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
    */
    /dts-v1/;
    /plugin/;
    #include <dt-bindings/gpio/gpio.h>
    / {
    fragment@101 {
    target-path = "/";
    __overlay__ {
    clk_cam_27M: fixed-clock-0-27M {
    compatible = "fixed-clock";
    #clock-cells = <0>;
    clock-frequency = <27000000>;
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

     

    Thanks 

    Michael

  • 0 in reply to Jay Sun
    TI__Genius 14345 points

    Hi.

    Correct what Michael said as the below.

    I think there are few things need to be modified:

    1. Modify driver ti,ds90ub960-q1 with fixed frame desc so that it does "NOT" need to qurry sensor anymore.

    2. Change input format from RAW to UYVY.

    Now, in short, Customer - Michael needs 4 cameras running concurrently, that's why they are posting this E2E.

    And now, the "Bottoleneck is Serdes - "UB953+960".".

  • 0 in reply to Rio Chan
    TI__Guru* 82176 points

    Hi Michael,

    Rio Chan said:

    Modify driver ti,ds90ub960-q1 with fixed frame desc so that it does "NOT" need to qurry sensor anymore.

    2. Change input format from RAW to UYVY.

    With these changes, could you please brief me the current status of your issue?

    Few questions from my end to get more clarity on your usecase:

    1. Have you written an ISP driver to where a streamON command could be sent from the user-space to start the IMX390 sensor?

    2. Have you set the format for UB953 and UB960 using the ./media-ctl -V command?

    3. Have you set the routes using the ./media-ctl -R command?

    4. Could you please share the current logs for the command ./media-ctl -p?

    5. What is the streaming command you are using? Could you please share the logs after sending the streaming command?

    Regards,
    Nikhil

  • 0 in reply to Nikhil Dasan
    Expert 1005 points

    Hi Nikhil,

    Nikhil Dasan said:
    1. Have you written an ISP driver to where a streamON command could be sent from the user-space to start the IMX390 sensor?

    The camera module does noe need streamON command. It sends out image data once the power is on.

    Nikhil Dasan said:
    2. Have you set the format for UB953 and UB960 using the ./media-ctl -V command?

    I am not able to set the format for UB953 and UB960 successfully. It gives invalid argumnet error.

    Nikhil Dasan said:
    3. Have you set the routes using the ./media-ctl -R command?

    I try to link them but it also gives invalid argument error.

    Nikhil Dasan said:
    4. Could you please share the current logs for the command ./media-ctl -p?

    Something is wrong that video devices do not show up.

    Nikhil Dasan said:
    5. What is the streaming command you are using? Could you please share the logs after sending the streaming command?

    Thanks 

    Michael

  • 0 in reply to Jay Sun
    TI__Guru* 82176 points

    Hi Michael,

    Jay Sun said:
    I am not able to set the format for UB953 and UB960 successfully. It gives invalid argumnet error

    Here the syntax is wrong. You shouldn't include @1/30 here.
    Please refer below format (Please change the name and format accordingly)

    media-ctl -V '"ds90ub953 9-0044":0 [fmt:SRGGB12_1X12/1936x1100 field: none]'

    Jay Sun said:
    I try to link them but it also gives invalid argument error.

    Could you try to set the route here instead.
    Please refer below format (Please change the name and format accordingly)

    ./media-ctl -R '"4500000.ticsi2rx" [0/0 -> 1/0 [1]]'

    Jay Sun said:
    Something is wrong that video devices do not show up.

    yes, here the video devices did not show up and also the connection between the bridge to csi2rx is missing.

    Could you share the full linux kernel logs in order to check if some probe error occured?

    Regards,
    Nikhil

  • 0 in reply to Nikhil Dasan
    Expert 1005 points

    Hi Nikhil, 

    Nikhil Dasan said:

    Here the syntax is wrong. You shouldn't include @1/30 here.
    Please refer below format (Please change the name and format accordingly)

    media-ctl -V '"ds90ub953 9-0044":0 [fmt:SRGGB12_1X12/1936x1100 field: none]'

    I used the command accordingly, but it still gave some error.

    ./media-ctl --set-v4l2 '"ds90ub953 9-0044":0 [fmt:UYVY8_2X8/1920x1080 field: none]'

    --> Unable to setup fornats: No such file or directory (2)

    ./media-ctl -R '"4500000.ticsi2rx" [0/0 -> 1/0 [1]]'

    --> VIDIOC_SUBDEV_S_ROUTING failed: -2

         unable to setup routes: No such file or directory (2)

    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd080]
    [ 0.000000] Linux version 5.10.120-g95b90aa828 (oe-user@oe-host) (aarch64-none-linux-gnu-gcc (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 9.2.1 20191025, GNU ld (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 2.33.1.20191209) #1 SMP PREEMPT Tue Aug 30 09:21:40 UTC 2022
    [ 0.000000] Machine model: Texas Instruments K3 J721E SoC
    [ 0.000000] earlycon: ns16550a0 at MMIO32 0x0000000002800000 (options '')
    [ 0.000000] printk: bootconsole [ns16550a0] enabled
    [ 0.000000] efi: UEFI not found.
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a0000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a0000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a0100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a0100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a1000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a1000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a1100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a1100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a2000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a2000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a2100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a2100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a3000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a3000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a3100000, size 15 MiB
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Thanks 

    Michael

  • 0 in reply to Jay Sun
    TI__Guru* 82176 points

    Hi Michael,

    Thank you for providing the logs. 
    I did a review of your linux logs and didn't find any specific error that could cause this.

    Jay Sun said:
    Could you please share the current logs for the command ./media-ctl -p

    Could you share the logs of media-ctl -p? (Not the image). Here we could get more information from the logs.

    Also, is it possible for you to put logs in the linux driver and check? If yes, could you put few logs in the function ti_csi2rx_probe() in the file ${PSDKLA}/board_support/linux-xx/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c and check whether this function fails? (i.e., checking its return value at each stage)

    Regards,
    Nikhil

  • 0 in reply to Nikhil Dasan
    TI__Genius 14345 points

    Hi Nikhil:

    Now, the Customer - Michael confirmed:

    Since the TI native code will bring up the ISP driver, but, this is not what he needs.

    May you help to let him know about : How to NOT use the Native ISP camera driver?

    Thanks.

    BR Rio

  • 0 in reply to Rio Chan
    TI__Guru* 82176 points

    Hi Rio,

    I don't think we have a Native ISP driver on the Linux side.

    Nikhil Dasan said:

    Could you share the logs of media-ctl -p? (Not the image). Here we could get more information from the logs.

    Also, is it possible for you to put logs in the linux driver and check? If yes, could you put few logs in the function ti_csi2rx_probe() in the file ${PSDKLA}/board_support/linux-xx/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c and check whether this function fails? (i.e., checking its return value at each stage)

    Could you please get the above information? This could tell us more about the issue.

    Regards,
    Nikhil

  • 0 in reply to Nikhil Dasan
    Expert 1005 points

    Hi Nikhil,

    • logs of media-ctl -p
      Fullscreen
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      root@j7-evm:~# media-ctl -p
      Media controller API version 5.10.120
      Media device information
      ------------------------
      driver j721e-csi2rx
      model TI-CSI2RX
      serial
      bus info platform:4500000.ticsi2rx
      hw revision 0x1
      driver version 5.10.120
      Device topology
      - entity 1: 4500000.ticsi2rx (17 pads, 0 link, 0 route)
      type V4L2 subdev subtype Unknown flags 0
      pad0: Sink
      pad1: Source
      pad2: Source
      pad3: Source
      pad4: Source
      pad5: Source
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    • debug info of ti_csi2rx_probe()
      Fullscreen
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      t@j7-evm:~# dmesg | grep ti_csi2rx_probe
      [ 9.270659] ti_csi2rx_probe->devm_kzalloc: (____ptrval____)
      [ 9.284556] ti_csi2rx_probe->devm_ioremap_resource: (____ptrval____)
      [ 9.296572] ti_csi2rx_probe->of_property_count_strings: 16
      [ 9.310444] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.331584] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.355017] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.376631] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.467411] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.586120] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.794908] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.860305] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.975587] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 9.999337] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 10.007181] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 10.030327] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 10.061256] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 10.181721] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 10.251913] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 10.308605] ti_csi2rx_probe->ti_csi2rx_init_ctx: 0
      [ 10.324729] ti_csi2rx_probe->ti_csi2rx_init_subdev: 0
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    • source code modified
      Fullscreen
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      static int ti_csi2rx_probe(struct platform_device *pdev)
      {
      struct device_node *np = pdev->dev.of_node;
      struct ti_csi2rx_dev *csi;
      struct resource *res;
      int ret, i, count;
      unsigned int reg;
      csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
      pr_info("ti_csi2rx_probe->devm_kzalloc: %p\n", csi);
      if (!csi)
      return -ENOMEM;
      csi->dev = &pdev->dev;
      platform_set_drvdata(pdev, csi);
      res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
      csi->shim = devm_ioremap_resource(&pdev->dev, res);
      pr_info("ti_csi2rx_probe->devm_ioremap_resource: %p\n", csi->shim);
      if (IS_ERR(csi->shim))
      return PTR_ERR(csi->shim);
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Thanks

    Michael

  • 0 in reply to Jay Sun
    TI__Genius 14345 points

    Hi Nikhil:

    Any feedback of this ticket?

    BR Rio

  • 0 in reply to Rio Chan
    TI__Guru* 82176 points

    Hi Michael,

    Sorry for the delay in response.
    We were trying to simulate this scenario at our end by not connecting the camera and doing your dts changes.

    We were able to see that the context/video devices did not show up.
    After further debugging, we saw that the UB953 serializer driver waits for a notification from the sensor in order to propogate further.

    Hence, we included a dummy IMX390 driver which only notifies to the serializer. Please find the patch below for the same.


    /cfs-file/__key/communityserver-discussions-components-files/791/0001_2D00_media_2D00_i2c_2D00_imx390_2D00_dummy_2D00_test_2D00_driver.patch

    Further you could make changes in your dts file by referring the below changes

    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    diff --git a/arch/arm64/boot/dts/ti/k3-j721e-fpdlink-imx390-cm-0-0.dts b/arch/arm64/boot/dts/ti/k3-j721e-fpdlink-imx390-cm-0-0.dts
    index fff3678ab505..8ad93ff2affd 100644
    --- a/arch/arm64/boot/dts/ti/k3-j721e-fpdlink-imx390-cm-0-0.dts
    +++ b/arch/arm64/boot/dts/ti/k3-j721e-fpdlink-imx390-cm-0-0.dts
    @@ -78,17 +78,9 @@
    #size-cells = <0>;
    sensor@21 {
    - compatible = "sony,imx390";
    + compatible = "ottobrite,imx390dummy";
    reg = <0x21>;
    - clocks = <&clk_cam_27M>;
    - clock-names = "inck";
    -
    - xclr-gpios = <&serializer 0 GPIO_ACTIVE_LOW>;
    - error0-gpios = <&serializer 1 GPIO_ACTIVE_HIGH>;
    - error1-gpios = <&serializer 2 GPIO_ACTIVE_HIGH>;
    - comready-gpios = <&serializer 3 GPIO_ACTIVE_HIGH>;
    -
    port {
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Please try out these changes at your end and let me know if this resolves your issue.

    Regards,
    Nikhil

  • 0 in reply to Nikhil Dasan
    Expert 1005 points

    Hi Nikhil,

    Thank you so much for giving us great help.

    There is some minor problem got to be resolved.

    1. The log of "media-ctl -p" seems fine.
      Fullscreen
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      Media controller API version 5.10.120
      Media device information
      ------------------------
      driver j721e-csi2rx
      model TI-CSI2RX
      serial
      bus info platform:4500000.ticsi2rx
      hw revision 0x1
      driver version 5.10.120
      Device topology
      - entity 1: 4500000.ticsi2rx (17 pads, 17 links, 1 route)
      type V4L2 subdev subtype Unknown flags 0
      device node name /dev/v4l-subdev0
      routes:
      0/0 -> 1/0 [ACTIVE]
      pad0: Sink
      <- "cdns_csi2rx.4504000.csi-bridge":1 [ENABLED,IMMUTABLE]
      pad1: Source
      -> "4500000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    2. It gives broke pipe error after requesting it to start capturing.
      Fullscreen
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      root@j7-evm:~# yavta -c -Fcapture -s 1920x1080 -f UYVY /dev/video3
      Device /dev/video3 opened.
      Device `j721e-csi2rx' on `platform:4500000.ticsi2rx' is a video output (without mplanes) device.
      Video format set: UYVY (59565955) 1920x1080 (stride 3840) field none buffer size 4147200
      Video format: UYVY (59565955) 1920x1080 (stride 3840) field none buffer size 4147200
      8 buffers requested.
      length: 4147200 offset: 0 timestamp type/source: mono/EoF
      Buffer 0/0 mapped at address 0xffff843b4000.
      length: 4147200 offset: 4149248 timestamp type/source: mono/EoF
      Buffer 1/0 mapped at address 0xffff83fbf000.
      length: 4147200 offset: 8298496 timestamp type/source: mono/EoF
      Buffer 2/0 mapped at address 0xffff83bca000.
      length: 4147200 offset: 12447744 timestamp type/source: mono/EoF
      Buffer 3/0 mapped at address 0xffff837d5000.
      length: 4147200 offset: 16596992 timestamp type/source: mono/EoF
      Buffer 4/0 mapped at address 0xffff833e0000.
      length: 4147200 offset: 20746240 timestamp type/source: mono/EoF
      Buffer 5/0 mapped at address 0xffff82feb000.
      length: 4147200 offset: 24895488 timestamp type/source: mono/EoF
      Buffer 6/0 mapped at address 0xffff82bf6000.
      length: 4147200 offset: 29044736 timestamp type/source: mono/EoF
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Thanks 

    Michael

  • 0 in reply to Jay Sun
    TI__Guru* 82176 points

    Hi Michael,

    2 things I noticed here

    1. The context is mapped to /dev/video2 and you are trying to stream video3 in the yavta command.
        Could you try to stream in /dev/video2?

    2. The format of the imx390dummy 10-0021 is YUYV8_2X8 whereas others are in UYVY8_2X8. This mostly shouldn't be an issue, but can you set the same format in the dummy driver too if the issue doesn't resolve with the first step.

    Regards,
    Nikhil

  • 0 in reply to Nikhil Dasan
    Expert 1005 points

    Appreciate for your great work, Nikhil.

    It works like a charm.