• State Resolved
  • Replies 4 replies
  • Subscribers 98 subscribers
  • Views 83 views
  • Users 0 members are here
Support feedback
Options
Options
Related

J784S4XEVM: Camera support for Linux SDK

Sivadharshini Palanivelrajan
Prodigy 10 points

Part Number: J784S4XEVM
Other Parts Discussed in Thread: DS90UB9702-Q1

We are working with a J784S4 EVM + Fusion2 board (DS90UB9702) and a Leopard Imaging LI-AR0233-GW5400-FPDLINKIII-060H camera.

We have successfully brought up this camera using Processor SDK RTOS / Vision Apps with the TI Imaging sensor driver GW_AR0233_UYVY.

We are now evaluating whether the same camera can be enabled using the native Linux V4L2 / Media Controller / CSI2RX stack, without using the RTOS Imaging camera stack.

 

On our current J784S4 Linux image we already have:

  • Linux SDK 11.02.01.03
  • j721e-csi2rx.ko
  • cdns-csi2rx.ko
  • cdns-dphy-rx.ko
  • ds90ub953.ko
  • ds90ub960.ko, including ti,ds90ub9702-q1 support
  • k3-j784s4-evm-fpdlink-iv-fusion.dtbo
  • existing FPD-Link camera overlays for IMX390, OV2312 and IMX728

We do not currently see a Linux overlay or V4L2 sub device implementation for the GW5400/AR0233 camera.

 

We would like to understand some points:

  1. Is LI-AR0233-GW5400-FPDLINKIII supported through the J784S4 Linux V4L2 stack?
  2. Is there an already existing Linux V4L2 sub device driver or some overlay for this camera?
  3. Are the standard Linux ds90ub953 + ds90ub960/UB9702 drivers the recommended FPD-Link implementation for this setup?
  4. If we do get the driver from leopard imaging, in that sense do we have any documents or notes on how could we build the driver to the board etc.,?

 

Our goal is to understand whether native Linux capture is a supported so that we could enable the Camera directly with Linux, because the perception stack - vision pilot is running on linux on the board and we need the camera frames for the stack for further processing.

  • 0
    TI__Mastermind 28810 points

    Hi ,

    It does not appear that there is a Linux driver for the AR0234AT. To support the sensor within our Linux SDK, you will have to create a driver for the sensor (similar to the imx219 and other i2c sensors).

    You can look at other sensors within: drivers/media/i2c

    If there is a working Linux kernel driver, you can add it to the device tree and it should enumerate as v4l2 device.

    Best,
    Jared

  • 0 in reply to Jared McArthur
    Prodigy 10 points

    Hi Jared,

    Thanks for the clarification.

    Since our LI-AR0233-GW5400-FPDLINKIII camera uses a DS90UB953 serializer and a Fusion2 / DS90UB9702 deserializer path, would the existing IMX390 Linux V4L2/media-controller implementation be the recommended reference for adding AR0233/GW5400 support on J784S4?

    My understanding is that the IMX390 FPD-Link camera path is already supported/tested on the Linux side and also through the RTOS/Vision Apps camera framework. If so, would it make sense to reuse the existing FPD-Link / CSI2RX / media-controller topology and mainly add the camera-module-specific driver and device-tree configuration for the AR0233 + GW5400 module?

    As a side note, I have already been able to bring up this camera successfully using the Vision Apps / RTOS SDK path. The module outputs processed UYVY through the GW5400 ISP, and capture/display is working through TIOVX.

    Our perception stack currently expects camera input through a standard Linux V4L2 device such as /dev/videoX, so we are evaluating both integration approaches.

    Do you have a recommendation on which approach would be more suitable for a production perception application on J784S4?

    1. Add Linux V4L2/media-controller support for the AR0233/GW5400 FPD-Link camera and use /dev/videoX, or
    2. Use the Vision Apps/TIOVX capture path directly from the application?

    From a performance and latency perspective, would the TIOVX capture path provide a significant advantage, especially if image preprocessing and TIDL inference are also done using TI accelerators? Or can the Linux V4L2 path achieve comparable performance ?

    Best,

    Siva

  • +1 in reply to Sivadharshini Palanivelrajan
    TI__Mastermind 28810 points

    Hi ,

    Sivadharshini Palanivelrajan said:
    My understanding is that the IMX390 FPD-Link camera path is already supported/tested on the Linux side and also through the RTOS/Vision Apps camera framework. If so, would it make sense to reuse the existing FPD-Link / CSI2RX / media-controller topology and mainly add the camera-module-specific driver and device-tree configuration for the AR0233 + GW5400 module?

    This is what I was suggesting within the original comment.

    Sivadharshini Palanivelrajan said:
    • Add Linux V4L2/media-controller support for the AR0233/GW5400 FPD-Link camera and use /dev/videoX, or
    • Use the Vision Apps/TIOVX capture path directly from the application?

    I would recommend the RTOS stack. The RTOS drivers support hardware accelerators while Linux does not. The preprocessing and TIDL inference will greatly accelerate the pipeline than having those applications run on Linux.

    Best,
    Jared

  • 0 in reply to Jared McArthur
    Prodigy 10 points

    Hi Jared,

    Thank you very much! That helps!

    best,

    Siva