SK-AM68: 4K UYVY capture/encode

Hello Brad,

The expert handling this topic is out-of-office for a week, so please expect some delay responding to this query. We request your patience.

Thanks.

Hi Brad,

What camera sensor and which SDK and version are you using? Also, please ensure that nothing else is running in parallel as this can affect streaming performance.

Thank you,

Fabiana Jaimes

We built an HDMI to MIPI converter board for the dual CSI header on the EVM. The part is capable of capturing up to 4k60 which is what we are currently targeting. The SDK version is 8.06.01.03.

Hi Brad,

I would like to reproduce this on my end. Please allow me some time to gather the necessary equipment.

Thank you for your patience,

Fabiana Jaimes

Hi Brad,

Apologies on the long delay on this issue. Could you share how you are obtaining the fps information?

Thank you,

Fabiana Jaimes

We drew a timer on the HDMI output of a device and recorded with the above pipeline. Single stepping through the encoded stream shows the number of frames acquired in a second. I just reran the experiment and observed right at 13 fps.

Hi Brad,

I am still awaiting a response from my team. I apologize for the delay on this.

Thank you for your patience.

- Fabiana

In the meantime, could you please provide the outputs for the commands in the attached image? The sensor used for this example is OV5640.

Thank you,

Fabiana

root@am68a-sk:/opt/edgeai-gst-apps# media-ctl -d /dev/media0 -p
Media controller API version 5.10.162

- entity 17: dp720 5-002b (1 pad, 1 link, 0 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
        pad0: Source
                [stream:0 fmt:UYVY8_1X16/0x0 field:none colorspace:smpte170m]
                [dv.caps:BT.656/1120 min:640x350@13000000 max:3840x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 3840x2160p30 (4400x2250) stds: flags:]
                [dv.current:BT.656/1120 0x0p0 (0x0) stds: flags:]
                -> "cdns_csi2rx.4504000.csi-bridge":0 [ENABLED,IMMUTABLE]


root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes pad=0
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0,stream=0)
        0x100a: MEDIA_BUS_FMT_RGB888_1X24
        0x200f: MEDIA_BUS_FMT_UYVY8_1X16

root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x200f
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0,stream=0)


root@am68a-sk:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-frameintervals pad=0,width=640,height=480,code=0x200f
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0,stream=0)

Hi Brad, 

Thank you for the details. Our team is still running a few tests and will get back to you soon.

Thank you,

Fabiana

Hi Brad,

We are still awaiting a shipment with the HDMI to MIPI converter board to run some tests. In the meantime, could you attach a CSI camera to if you can reproduce the mismatch in framerate?

Thank you,

Fabiana

Can you recommend a 4k30 yuv 4:2:2 colorspace sensor that would possibly recreate this issue? The CSI devices we have are 1080p and work fine.

Hi Brad,

Before we test another sensor, could you share the output from running the parse GST tracers script to help identify where the bottleneck is?

This will help identify where the bottleneck is.

Thank you,

Fabiana

Hi Brad,

What was the GStreamer command you prepended the parse GST tracers string to?

Thank you,

Fabiana

Hi Brad,

We are still awaiting the shipment.

Apologies for the delay and thank you for your patience.

-Fabiana

Hi Brad,

The part has been delivered to us today and we will test tomorrow. As far as the latency seen with tiovxcolorconvert, we are waiting for our experts to comment.

Thank you,

Fabiana

Hi Brad,

you can use tiovxldc to do color conversion from UYVY to NV12
This will give much better latency, since it runs on HW accelerator.

Regards
Rahul T R

Running the following:

gst-launch-1.0 v4l2src device=/dev/video2 io-mode=dmabuf ! video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY ! tiovxldc ! v4l2h264enc ! filesink location=./cap_ldc.264

Exits with the following warning

WARNING: erroneous pipeline: could not link tiovxldc0 to v4l2h264enc0

can you please run it with -v and share the log
gst-launch-1.0 -v v4l2src device=/dev/video2 io-mode=dmabuf ! video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY ! tiovxldc ! v4l2h264enc ! filesink location=./cap_ldc.264

root@am68a-sk:~/test# gst-launch-1.0 -v v4l2src device=/dev/video2 io-mode=dmabuf ! video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY ! tiovxldc ! v4l2h264enc ! filesink location=./cap_ldc.264
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
 35314.366157 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
 35314.366227 s:  VX_ZONE_INIT:Enabled
 35314.366234 s:  VX_ZONE_ERROR:Enabled
 35314.366241 s:  VX_ZONE_WARNING:Enabled
 35314.366975 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
 35314.367876 s:  VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
WARNING: erroneous pipeline: could not link tiovxldc0 to v4l2h264enc0

Would love to know the details of this hardware that you are testing. Is this a commercially available item or internal to TI. Either our team here is ready to test.

Hi Brad,

Which SDK version you are on
Can you please try with 9.0 SDK

PROCESSOR-SDK-LINUX-AM68A Software development kit (SDK) | TI.com

Regards
Rahul T R

Rahul,

My current test setup is with the 08.06.01.03. Moving to 9.0 will require porting the driver for our capture hardware to the 6.1 kernel. Before I do that, do you have any indication that the colorspace convert pipeline module has been improved?

Also, what is your analysis of the tiovxldc module? Did you expect it would work?

Fabiana,

You claimed that you were testing last Tuesday. Do you have results? I would also like any technical detail you can provide on the hardware you are using.

Hi Brad,

LDC is a HWA present on TI SOCs for lens distortion correction
It has some additional capabilities to do some color space conversion

We have added this support to plugin here post 8.6
ext: ldc: Add support for colorconversion UYVY -> NV12 · TexasInstruments/edgeai-gst-plugins@2de7a65 (github.com)

You can cherry pick this patch to /opt/edgeai-gst-plugins and try the pipeline
But it might not be straight forward, since there might be some additional dependencies.

Alternatively you can use tiovxdlcolorconvert which runs on ARM with neon optimization

Regards
Rahul T R