TDA4VM: sdk9.2 with imx390 (built-in isp)

zhiyong liu

Part Number: TDA4VM

Tool/software:

hi ti

I have a problem, my current environment is J721EVM
And sub-board Fusion Application Board, the compilation environment is linux sdk 9.2, we're using an IMX390 with in-built ISP（imx390+5200+ub953）All ends up being yuv output, but the following question arises when configuring whether the environment now supports yuv output?

root@j721e-evm:/opt/edgeai-gst-apps# dmesg | grep "390"
[    3.753903] systemd[1]: Kernel Trace File System was skipped because of a failed condition check (ConditionPathExists=/sys/kernel/tracing).
[    9.390948] cdns-ufshcd 4e84000.ufs: ufshcd_populate_vreg: Unable to find vcc-supply regulator, assuming enabled
[   10.054370] imx390 7-001a: inck rate: 27027027 Hz
[   10.134815] imx390 7-001a: imx390_read: failed to read reg 0x0330: -121
[   10.144655] imx390: probe of 7-001a failed with error -121

over 1 year ago

0 zhiyong liu over 1 year ago

Prodigy 60 points

If the output of senser is whether yuv 8bit is supported or not

0 Praveen Rao over 1 year ago in reply to zhiyong liu

TI__Mastermind 47963 points

Hello Zhiyong,

Our expert handling this topic is currently out of the office; please expect a 1-2 day delay in response.

Thanks.

0 Fabiana Jaimes over 1 year ago in reply to zhiyong liu

TI__Mastermind 19680 points

Hello Zhiyong,

YUV format is supported. Please keep in mind that we currently do not support IMX390 with built-in ISP. For more information on what will be required to enable IMX390 with built-in ISP, see the following thread: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1233573/sk-am62a-lp-debug-about-csi2rx

Thank you,

Fabiana

0 zhiyong liu over 1 year ago in reply to Fabiana Jaimes

Prodigy 60 points

Fabiana Jaimes said:
e2e.ti.com/.../quote]

Thank you very much, I have changed the connection and used gst-launch-1.0v4l2src device="/dev/video18"! video/x-raw, width=1920, height=1080, format=UYVY ! autovideosink command, it can be successfully displayed on the dp screen, but I do not understand this command, media-ctl-R '" 45000000.ticsi2rx "[0/0 -> 1/0 [1]]' and media-ctl-R '"ds90ub960 9-003d" [0/0 -> 4/0 [1]]' if I'm on route 4 How should this command be set, and what do the parameters mean? Also, because it is my own isp, I configured a fake 390. Do you have a demo code that allows me to get buffer and save it

0 Fabiana Jaimes over 1 year ago in reply to zhiyong liu

TI__Mastermind 19680 points

Hi Zhiyong,

Could you start by sharing the output of media-ctl -p?

Thank you,

Fabiana

0 zhiyong liu over 1 year ago in reply to Fabiana Jaimes

Prodigy 60 points

C
root@j7-evm:/opt/edgeai-gst-apps# ^C
root@j7-evm:/opt/edgeai-gst-apps#
root@j7-evm:/opt/edgeai-gst-apps#
root@j7-evm:/opt/edgeai-gst-apps# media-ctl -p
Media controller API version 5.10.162

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial
bus info        platform:4500000.ticsi2rx
hw revision     0x1
driver version  5.10.162

Device topology
- entity 1: 4500000.ticsi2rx (17 pads, 17 links, 1 route)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev3
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "cdns_csi2rx.4504000.csi-bridge":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "4500000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
        pad2: Source
                -> "4500000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
        pad3: Source
                -> "4500000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
        pad4: Source
                -> "4500000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
        pad5: Source
                -> "4500000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
        pad6: Source
                -> "4500000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]
        pad7: Source
                -> "4500000.ticsi2rx context 6":0 [ENABLED,IMMUTABLE]
        pad8: Source
                -> "4500000.ticsi2rx context 7":0 [ENABLED,IMMUTABLE]
        pad9: Source
                -> "4500000.ticsi2rx context 8":0 [ENABLED,IMMUTABLE]
        pad10: Source
                -> "4500000.ticsi2rx context 9":0 [ENABLED,IMMUTABLE]
        pad11: Source
                -> "4500000.ticsi2rx context 10":0 [ENABLED,IMMUTABLE]
        pad12: Source
                -> "4500000.ticsi2rx context 11":0 [ENABLED,IMMUTABLE]
        pad13: Source
                -> "4500000.ticsi2rx context 12":0 [ENABLED,IMMUTABLE]
        pad14: Source
                -> "4500000.ticsi2rx context 13":0 [ENABLED,IMMUTABLE]
        pad15: Source
                -> "4500000.ticsi2rx context 14":0 [ENABLED,IMMUTABLE]
        pad16: Source
                -> "4500000.ticsi2rx context 15":0 [ENABLED,IMMUTABLE]

- entity 19: cdns_csi2rx.4504000.csi-bridge (5 pads, 2 links, 0 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev4
        pad0: Sink
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "ds90ub960 9-003d":4 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "4500000.ticsi2rx":0 [ENABLED,IMMUTABLE]
        pad2: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
        pad3: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
        pad4: Source
                [stream:0 fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]

- entity 25: ds90ub960 9-003d (6 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
        routes:
                0/0 -> 4/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_2X8/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "ds90ub953 9-0044":1 [ENABLED,IMMUTABLE]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Source
                [stream:0 fmt:UYVY8_2X8/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "cdns_csi2rx.4504000.csi-bridge":0 [ENABLED,IMMUTABLE]
        pad5: Source

- entity 34: ds90ub953 9-0044 (2 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: Sink
                [stream:0 fmt:UYVY8_2X8/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "imx390dummy 10-0021":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:UYVY8_2X8/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "ds90ub960 9-003d":0 [ENABLED,IMMUTABLE]

- entity 39: imx390dummy 10-0021 (1 pad, 1 link, 2 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
        routes:
                0/0 -> 0/0 [ACTIVE, IMMUTABLE, SOURCE]
                0/0 -> 0/1 [INACTIVE, IMMUTABLE, SOURCE]
        pad0: Source
                [stream:0 fmt:UYVY8_2X8/1920x1080@1/30 field:none colorspace:smpte170m]
                -> "ds90ub953 9-0044":0 [ENABLED,IMMUTABLE]

- entity 45: 4500000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video18
        pad0: Sink
                <- "4500000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 51: 4500000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video19
        pad0: Sink
                <- "4500000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 57: 4500000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video20
        pad0: Sink
                <- "4500000.ticsi2rx":3 [ENABLED,IMMUTABLE]

- entity 63: 4500000.ticsi2rx context 3 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video21
        pad0: Sink
                <- "4500000.ticsi2rx":4 [ENABLED,IMMUTABLE]

- entity 69: 4500000.ticsi2rx context 4 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video22
        pad0: Sink
                <- "4500000.ticsi2rx":5 [ENABLED,IMMUTABLE]

- entity 75: 4500000.ticsi2rx context 5 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video23
        pad0: Sink
                <- "4500000.ticsi2rx":6 [ENABLED,IMMUTABLE]

- entity 81: 4500000.ticsi2rx context 6 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video24
        pad0: Sink
                <- "4500000.ticsi2rx":7 [ENABLED,IMMUTABLE]

- entity 87: 4500000.ticsi2rx context 7 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video25
        pad0: Sink
                <- "4500000.ticsi2rx":8 [ENABLED,IMMUTABLE]

- entity 93: 4500000.ticsi2rx context 8 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video26
        pad0: Sink
                <- "4500000.ticsi2rx":9 [ENABLED,IMMUTABLE]

- entity 99: 4500000.ticsi2rx context 9 (1 pad, 1 link, 0 route)
             type Node subtype V4L flags 0
             device node name /dev/video27
        pad0: Sink
                <- "4500000.ticsi2rx":10 [ENABLED,IMMUTABLE]

- entity 105: 4500000.ticsi2rx context 10 (1 pad, 1 link, 0 route)
              type Node subtype V4L flags 0
              device node name /dev/video28
        pad0: Sink
                <- "4500000.ticsi2rx":11 [ENABLED,IMMUTABLE]

- entity 111: 4500000.ticsi2rx context 11 (1 pad, 1 link, 0 route)
              type Node subtype V4L flags 0
              device node name /dev/video29
        pad0: Sink
                <- "4500000.ticsi2rx":12 [ENABLED,IMMUTABLE]

- entity 117: 4500000.ticsi2rx context 12 (1 pad, 1 link, 0 route)
              type Node subtype V4L flags 0
              device node name /dev/video30
        pad0: Sink
                <- "4500000.ticsi2rx":13 [ENABLED,IMMUTABLE]

- entity 123: 4500000.ticsi2rx context 13 (1 pad, 1 link, 0 route)
              type Node subtype V4L flags 0
              device node name /dev/video31
        pad0: Sink
                <- "4500000.ticsi2rx":14 [ENABLED,IMMUTABLE]

- entity 129: 4500000.ticsi2rx context 14 (1 pad, 1 link, 0 route)
              type Node subtype V4L flags 0
              device node name /dev/video32
        pad0: Sink
                <- "4500000.ticsi2rx":15 [ENABLED,IMMUTABLE]

- entity 135: 4500000.ticsi2rx context 15 (1 pad, 1 link, 0 route)
              type Node subtype V4L flags 0
              device node name /dev/video33
        pad0: Sink
                <- "4500000.ticsi2rx":16 [ENABLED,IMMUTABLE]

root@j7-evm:/opt/edgeai-gst-apps#
root@j7-evm:/opt/edgeai-gst-apps# gst-launch-1.0 v4l2src device="/dev/video18" ! video/x-raw, width=1920, height=1080, format=UYVY ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
../gst-plugins-good-1.16.3/sys/v4l2/gstv4l2src.c(660): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.007780540
Setting pipeline to NULL ...
Freeing pipeline ...
root@j7-evm:/opt/edgeai-gst-apps#
root@j7-evm:/opt/edgeai-gst-apps# media-ctl -V '"ds90ub953 9-0044":0 [fmt:UYVY8_2X8/1920x1080 field: none]'
csi2rx" [0/0 -> 1/0 [1]]'root@j7-evm:/opt/edgeai-gst-apps# media-ctl -R '"ds90ub960 9-003d" [0/0 -> 4/0 [1]]'
root@j7-evm:/opt/edgeai-gst-apps# media-ctl -V '"ds90ub960 9-003d":0 [fmt:UYVY8_2X8/1920x1080 field: none]'
root@j7-evm:/opt/edgeai-gst-apps# media-ctl -R '"4500000.ticsi2rx" [0/0 -> 1/0 [1]]'
root@j7-evm:/opt/edgeai-gst-apps#
root@j7-evm:/opt/edgeai-gst-apps# gst-launch-1.0 v4l2src device="/dev/video18" ! video/x-raw, width=1920, height=1080, format=UYVY ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock



^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:03.768887860
Setting pipeline to NULL ...
Freeing pipeline ...
root@j7-evm:/opt/edgeai-gst-apps#

0 zhiyong liu over 1 year ago in reply to Fabiana Jaimes

Prodigy 60 points

hi jaimes；

I have another problem, I tried to write an application layer program, get buffer code as follows

struct v4l2_format vfmt;

vfmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; // Camera capture
vfmt.fmt.pix.width = 1920; // Set camera acquisition parameters, not arbitrary Settings
vfmt.fmt.pix.height = 1080;

vfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
perror("2\n");
int ret = ioctl(fd, VIDIOC_S_FMT, &vfmt);
if (ret < 0) {
perror(" Formatting failed 1");
return -1;
}

The PROBLEM is always in VIDIOC_S_FMT error, I check the format size and format because of the problem

#include <fcntl.h>
#include <linux/videodev2.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>


int main(void) {
  int fd = open("/dev/video18", O_RDWR);
  if (fd < 0) {
    perror("打开设备失败");
    return -1;
  }
  perror("1\n");
  // 获取摄像头支持格式 ioctl(文件描述符,命令，与命令对应的结构体)
  struct v4l2_format vfmt;

  vfmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; // 摄像头采集
  vfmt.fmt.pix.width = 1920; // 设置摄像头采集参数，不可以任意设置
  vfmt.fmt.pix.height = 1080;

  vfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
  perror("2\n");
  int ret = ioctl(fd, VIDIOC_S_FMT, &vfmt);
  if (ret < 0) {
    perror("设置格式失败1");
    return -1;
  }
  perror("3\n");
  // 申请内核空间
  struct v4l2_requestbuffers reqbuffer;
  reqbuffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
  reqbuffer.count = 4;                 // 申请4个缓冲区
  reqbuffer.memory = V4L2_MEMORY_MMAP; // 映射方式

  ret = ioctl(fd, VIDIOC_REQBUFS, &reqbuffer);
  if (ret < 0) {
    perror("申请空间失败");
    return -1;
  }
  perror("4\n");
  // 映射
  unsigned char *mptr[4]; // 保存映射后用户空间的首地址
  unsigned int size[4];
  struct v4l2_buffer mapbuffer;
  // 初始化type和index
  mapbuffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

  for (int i = 0; i < 4; i++) {
    perror("5\n");
    mapbuffer.index = i;
    ret = ioctl(fd, VIDIOC_QUERYBUF,
                &mapbuffer); // 从内核空间中查询一个空间作映射
    if (ret < 0) {
      perror("查询内核空间失败");
    return -1;
    }
    // 映射到用户空间
    mptr[i] =
        (unsigned char *)mmap(NULL, mapbuffer.length, PROT_READ | PROT_WRITE,
                              MAP_SHARED, fd, mapbuffer.m.offset);
    size[i] = mapbuffer.length; // 保存映射长度用于后期释放
    // 查询后通知内核已经放回
    ret = ioctl(fd, VIDIOC_QBUF, &mapbuffer);
    if (ret < 0) {
      perror("放回失败");
    return -1;
    }
  }
  // 开始采集
  int type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
  ret = ioctl(fd, VIDIOC_STREAMON, &type);
  if (ret < 0) {
    perror("开启失败");
    return -1;
  }
  // 从队列中提取一帧数据
  struct v4l2_buffer readbuffer;
  readbuffer.type =
      V4L2_BUF_TYPE_VIDEO_CAPTURE; // 每个结构体都需要设置type为这个参赛要记住
  ret = ioctl(fd, VIDIOC_DQBUF, &readbuffer);
  if (ret < 0) {
    perror("读取数据失败");
    return -1;
  }

  FILE *file = fopen("my.jpg", "w+"); // 打开一个文件
  fwrite(mptr[readbuffer.index], readbuffer.length, 1, file); // 写入文件
  fclose(file); // 写入完成，关闭文件
  perror("6\n");
  // 通知内核使用完毕
  ret = ioctl(fd, VIDIOC_QBUF, &readbuffer);
  if (ret < 0) {
    perror("放回队列失败");
    return -1;
  }

  // 停止采集
  ret = ioctl(fd, VIDIOC_STREAMOFF, &type);
  perror("7\n");
  // 释放映射
  for (int i = 0; i < 4; i++)
    munmap(mptr[i], size[i]);
  perror("8\n");
  close(fd); // 关闭文件

  return 0;
}

0 Fabiana Jaimes over 1 year ago in reply to zhiyong liu

TI__Mastermind 19680 points

Hello,

I recommend including your new sensor to /opt/edgeai-gst-apps/scripts/setup_cameras.sh to setup the camera for you upon boot. setup_ov5640() would be a good example to look at since OV5640 also has built-in ISP.

We do not offer application development support. I recommend looking into external online resources for this.

Thank you,

Fabiana

Processors

Processors forum

TDA4VM: sdk9.2 with imx390 (built-in isp)