TDA4VH-Q1: Multicamera stream freezing Issue

Rithika G

Part Number: TDA4VH-Q1
Other Parts Discussed in Thread: TDA4VH

Tool/software:

Hello all ,

We are using three ar0234 sensors and one ar0821 in TDA4VH (sdk version - 10_01_00_05 ) with the native serializer (ds90ub953.c) and deserializer (ds90ub960.c) drivers .

While testing parallel streaming with three AR0234 sensors (each at 1920x1080 resolution) and one AR0821 sensor (at 3840x2160 resolution), the stream freezes when pushed to the display. The parallel stream freezes even for one ar0234 and one ar0821 sensor.Could this issue be related to memory buffer limitations?

Gstreamer pipeline used for streaming of 3xar0234 at 1080p and 1xar0821 at 4k resolution :

gst-launch-1.0 v4l2src device=/dev/video6 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_0 v4l2src device=/dev/video7 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_1 v4l2src device=/dev/video8 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_2 v4l2src device=/dev/video9 ! video/x-raw,width=3840,height=2160 ! videoscale ! video/x-raw,width=1920,height=1080 ! tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue ! mosaic.sink_3 tiovxmosaic name=mosaic sink_0::startx="<0>" sink_0::starty="<0>" sink_1::startx="<1920>" sink_1::starty="<0>" sink_2::startx="<0>" sink_2::starty="<1080>" sink_3::startx="<1920>" sink_3::starty="<1080>" ! kmssink sync=false driver-name=tidss -e -v

4 months ago

0 Fabiana Jaimes 4 months ago

TI__Mastermind 19680 points

Hi Rithika,

Are you able to successfully stream from a single camera to your connected display? What is your display's resolution? Could you share the output of media-ctl -p?

Thank you,

Fabiana

0 Rithika G 4 months ago in reply to Fabiana Jaimes

Prodigy 20 points

Hi Fabiana,

1. Yes ,we can get stream from a single camera.

2. Display resolution : 800x600/60Hz

3. Media logs after setting resolution for 1xar0821 (at 3840x2160) and 3xar0234 (at 1920x1080).

Media controller API version 6.6.44

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial
bus info        platform:4520000.ticsi2rx
hw revision     0x1
driver version 6.6.44

Device topology
- entity 1: 4520000.ticsi2rx (9 pads, 9 links, 4 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
    routes:
        0/2 -> 4/0 [ACTIVE]
        0/3 -> 5/0 [ACTIVE]
        0/4 -> 6/0 [ACTIVE]
        0/5 -> 7/0 [ACTIVE]
    pad0: Sink
        [stream:2 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:3 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:4 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:5 fmt:UYVY8_1X16/3840x2160 field:none]
        <- "cdns_csi2rx.4524000.csi-bridge":1 [ENABLED,IMMUTABLE]
    pad1: Source
        -> "4520000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
    pad2: Source
        -> "4520000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
    pad3: Source
        -> "4520000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
    pad4: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none]
        -> "4520000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
    pad5: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none]
        -> "4520000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
    pad6: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none]
        -> "4520000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]
    pad7: Source
        [stream:0 fmt:UYVY8_1X16/3840x2160 field:none]
        -> "4520000.ticsi2rx context 6":0 [ENABLED,IMMUTABLE]
    pad8: Source
        -> "4520000.ticsi2rx context 7":0 [ENABLED,IMMUTABLE]

- entity 11: cdns_csi2rx.4524000.csi-bridge (5 pads, 2 links, 4 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
    routes:
        0/2 -> 1/2 [ACTIVE]
        0/3 -> 1/3 [ACTIVE]
        0/4 -> 1/4 [ACTIVE]
        0/5 -> 1/5 [ACTIVE]
    pad0: Sink
        [stream:2 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:3 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:4 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:5 fmt:UYVY8_1X16/3840x2160 field:none]
        <- "ds90ub960 6-0030":4 [ENABLED,IMMUTABLE]
    pad1: Source
        [stream:2 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:3 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:4 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:5 fmt:UYVY8_1X16/3840x2160 field:none]
        -> "4520000.ticsi2rx":0 [ENABLED,IMMUTABLE]
    pad2: Source
    pad3: Source
    pad4: Source

- entity 17: ds90ub960 6-0030 (6 pads, 5 links, 4 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
    routes:
        2/0 -> 4/2 [ACTIVE]
        3/0 -> 4/3 [ACTIVE]
        1/0 -> 4/4 [ACTIVE]
        0/0 -> 4/5 [ACTIVE]
    pad0: Sink
        [stream:0 fmt:UYVY8_1X16/3840x2160 field:none]
        <- "ds90ub953 6-0044":1 [ENABLED,IMMUTABLE]
    pad1: Sink
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none]
        <- "ds90ub953 6-0045":1 [ENABLED,IMMUTABLE]
    pad2: Sink
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none]
        <- "ds90ub953 6-0046":1 [ENABLED,IMMUTABLE]
    pad3: Sink
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none]
        <- "ds90ub953 6-0047":1 [ENABLED,IMMUTABLE]
    pad4: Source
        [stream:2 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:3 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:4 fmt:UYVY8_1X16/1920x1080 field:none]
        [stream:5 fmt:UYVY8_1X16/3840x2160 field:none]
        -> "cdns_csi2rx.4524000.csi-bridge":0 [ENABLED,IMMUTABLE]
    pad5: Source

- entity 26: ds90ub953 6-0045 (2 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev3
    routes:
        0/0 -> 1/0 [ACTIVE]
    pad0: Sink
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
        <- "ar0234 12-0042":0 [ENABLED,IMMUTABLE]
    pad1: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
        -> "ds90ub960 6-0030":1 [ENABLED,IMMUTABLE]

- entity 31: ds90ub953 6-0047 (2 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev4
    routes:
        0/0 -> 1/0 [ACTIVE]
    pad0: Sink
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
        <- "ar0234 11-0042":0 [ENABLED,IMMUTABLE]
    pad1: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
        -> "ds90ub960 6-0030":3 [ENABLED,IMMUTABLE]

- entity 36: ds90ub953 6-0044 (2 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
    routes:
        0/0 -> 1/0 [ACTIVE]
    pad0: Sink
        [stream:0 fmt:UYVY8_1X16/3840x2160 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
        <- "ar0821 8-0042":0 [ENABLED,IMMUTABLE]
    pad1: Source
        [stream:0 fmt:UYVY8_1X16/3840x2160 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
        -> "ds90ub960 6-0030":0 [ENABLED,IMMUTABLE]

- entity 41: ds90ub953 6-0046 (2 pads, 2 links, 1 route)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
    routes:
        0/0 -> 1/0 [ACTIVE]
    pad0: Sink
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
        <- "ar0234 7-0042":0 [ENABLED,IMMUTABLE]
    pad1: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
        -> "ds90ub960 6-0030":2 [ENABLED,IMMUTABLE]

- entity 46: ar0821 8-0042 (1 pad, 1 link, 1 route)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
    routes:
        0/0 -> 0/0 [ACTIVE]
    pad0: Source
        [stream:0 fmt:UYVY8_1X16/3840x2160@1/0 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
        -> "ds90ub953 6-0044":0 [ENABLED,IMMUTABLE]

- entity 50: ar0234 7-0042 (1 pad, 1 link, 2 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
    routes:
        0/0 -> 0/0 [ACTIVE]
        0/0 -> 0/1 [INACTIVE]
    pad0: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080@1/0 field:none colorspace:srgb]
        -> "ds90ub953 6-0046":0 [ENABLED,IMMUTABLE]

- entity 54: ar0234 11-0042 (1 pad, 1 link, 2 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev9
    routes:
        0/0 -> 0/0 [ACTIVE]
        0/0 -> 0/1 [INACTIVE]
    pad0: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080@1/0 field:none colorspace:srgb]
        -> "ds90ub953 6-0047":0 [ENABLED,IMMUTABLE]

- entity 58: ar0234 12-0042 (1 pad, 1 link, 2 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev10
    routes:
        0/0 -> 0/0 [ACTIVE]
        0/0 -> 0/1 [INACTIVE]
    pad0: Source
        [stream:0 fmt:UYVY8_1X16/1920x1080@1/0 field:none colorspace:srgb]
        -> "ds90ub953 6-0045":0 [ENABLED,IMMUTABLE]

- entity 64: 4520000.ticsi2rx context 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
    pad0: Sink
        <- "4520000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 70: 4520000.ticsi2rx context 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
    pad0: Sink
        <- "4520000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 76: 4520000.ticsi2rx context 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
    pad0: Sink
        <- "4520000.ticsi2rx":3 [ENABLED,IMMUTABLE]

- entity 82: 4520000.ticsi2rx context 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video7
    pad0: Sink
        <- "4520000.ticsi2rx":4 [ENABLED,IMMUTABLE]

- entity 88: 4520000.ticsi2rx context 4 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video8
    pad0: Sink
        <- "4520000.ticsi2rx":5 [ENABLED,IMMUTABLE]

- entity 94: 4520000.ticsi2rx context 5 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video9
    pad0: Sink
        <- "4520000.ticsi2rx":6 [ENABLED,IMMUTABLE]

- entity 100: 4520000.ticsi2rx context 6 (1 pad, 1 link)
              type Node subtype V4L flags 0
              device node name /dev/video10
    pad0: Sink
        <- "4520000.ticsi2rx":7 [ENABLED,IMMUTABLE]

- entity 106: 4520000.ticsi2rx context 7 (1 pad, 1 link)
              type Node subtype V4L flags 0
              device node name /dev/video11
    pad0: Sink
        <- "4520000.ticsi2rx":8 [ENABLED,IMMUTABLE]

Thanks,

Rithika

0 Fabiana Jaimes 4 months ago in reply to Rithika G

TI__Mastermind 19680 points

Hi Rithika,

Please take a look at the following GStreamer pipeline examples in this FAQ: [FAQ] SK-AM69: How to stream from CSI sensor and USB camera to display using GStreamer?

Be sure to define your display resolution to 800 width & 600 height in the output portion of your pipeline. Let me know if you have any further questions.

Thank you,

Fabiana

0 Rithika G 4 months ago in reply to Fabiana Jaimes

Prodigy 20 points

Hi Fabiana,

I have used the gstreamer pipeline which you have mentioned .I have changed the output portion to 800x600 .Still the issue persists.

Gstreamer pipeline used for streaming of 3xar0234 at 1080p and 1xar0821 at 4k resolution :

The gstreamer pipeline above has two different resolutions . For video nodes /dev/video6,/dev/video7,/dev/video8 at 1920x1080 resolution and dev/video9 at 3840x2160 resolution.

In othere cases, where we are using 1920x1080 or 1280x720 for /dev/video9 , the parallel streaming is proper.It freezes only when we are using 3840x2160

Thank you,

Rithika

0 Fabiana Jaimes 4 months ago in reply to Rithika G

TI__Mastermind 19680 points

Hi Rithika,

The pipeline you shared in your last message does not have the output resolution defined at 800x600 like so.

video/x-raw, width=800, height=600 ! kmssink driver-name=tidss sync=false force-modesetting=true

Please use one of the 4 camera pipelines in the FAQ as a base and make changes to fit your usecase.

Thank you,

Fabiana

0 Rithika G 4 months ago in reply to Fabiana Jaimes

Prodigy 20 points

Hi Fabiana,

Below is the Gstreamer pipeline for 4 camera setup by formatting output resolution to 800x600 as you mentioned:

gst-launch-1.0 v4l2src device=/dev/video6 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_0 v4l2src device=/dev/video7 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_1 v4l2src device=/dev/video8 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_2 v4l2src device=/dev/video9 ! video/x-raw,width=3840,height=2160 ! videoscale ! video/x-raw,width=1920,height=1080 ! tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue ! mosaic.sink_3 tiovxmosaic name=mosaic sink_0::startx="<0>" sink_0::starty="<0>" sink_1::startx="<1920>" sink_1::starty="<0>" sink_2::startx="<0>" sink_2::starty="<1080>" sink_3::startx="<1920>" sink_3::starty="<1080>" ! video/x-raw, width=800, height=600 ! kmssink driver-name=tidss sync=false force-modesetting=true -e -v

The above pipeline is not even initiating the stream. Let me know if we have to do any changes for this pipeline.

Below is the Working pipeline :

gst-launch-1.0 v4l2src device=/dev/video6 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_0 v4l2src device=/dev/video7 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_1 v4l2src device=/dev/video8 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_2 v4l2src device=/dev/video9 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_3 tiovxmosaic name=mosaic sink_0::startx="<0>" sink_0::starty="<0>" sink_1::startx="<1920>" sink_1::starty="<0>" sink_2::startx="<0>" sink_2::starty="<1080>" sink_3::startx="<1920>" sink_3::starty="<1080>" ! kmssink sync=false driver-name=tidss -e -v

In the above pipeline, the resolution for all four video devices (video6,video7,video8,video9)are same.

Non-working pipeline (stream freezing) :

gst-launch-1.0 v4l2src device=/dev/video6 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_0 v4l2src device=/dev/video7 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_1 v4l2src device=/dev/video8 ! video/x-raw,width=1920, height=1080 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_2 v4l2src device=/dev/video9 ! video/x-raw,width=3840, height=2160 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_3 tiovxmosaic name=mosaic sink_0::startx="<0>" sink_0::starty="<0>" sink_1::startx="<1920>" sink_1::starty="<0>" sink_2::startx="<0>" sink_2::starty="<1080>" sink_3::startx="<1920>" sink_3::starty="<1080>" ! kmssink sync=false driver-name=tidss -e -v

In the above pipeline , the resolution for /dev/video9 is different resolution and resolution is same for every other video devices.

In case of same resolution with all the four video devices, the parallel stream for 4 cameras is proper . In case of different resolution for any of the video devices, in our case /dev/video9 , the parallel stream is getting freezed.

Thank you ,

Rithika

0 Takuma Fujiwara 4 months ago in reply to Rithika G

TI__Guru 51353 points

Hi Rithika,

Fabiana is currently out, so I will fill in for her in the meantime.

I saw you have experimented with 3xar0234 + 1xar0821, and also experimented with 1xar0234 + 1xar0821.

Could you do an experiment with only 1xar0821 and see if the issue still persists? And please share the pipeline used in the experiment.

I suspect the 4K resolution is what is causing the issue, but would like to simplify the problem statement by taking out the other cameras. If the high resolution is the cause of issue, I have a theory of what could be happening.

Regards,

Takuma

0 Rithika G 4 months ago in reply to Takuma Fujiwara

Prodigy 20 points

Hi Takuma,

Stream is proper for 1xar0821.

Below is the gstreamer pipeline used:

gst-launch-1.0 v4l2src device=/dev/video5 ! video/x-raw,width=3840, height=2160 ! tiovxldc out-block-width=128 ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_0 tiovxmosaic name=mosaic sink_0::startx="<0>" sink_0::starty="<0>" ! kmssink sync=false driver-name=tidss -e -v

Thank you,

Rithika

0 Takuma Fujiwara 4 months ago in reply to Rithika G

TI__Guru 51353 points

Hi Rithika,

I see the mosaic startx and starty is different between the "Non-working pipeline" and the new working pipeline for 1xar0821.

Specifically, in the new pipeline:

Rithika G said:
mosaic.sink_0 tiovxmosaic name=mosaic sink_0::startx="<0>" sink_0::starty="<0>"

And compared to the old non-working pipeline:

Rithika G said:
sink_3::startx="<1920>" sink_3::starty="<1080>"

Maybe changing the size of the camera input is making mosaic try to write to a location that is not on the screen?

Regards,

Takuma

Processors

Processors forum

TDA4VH-Q1: Multicamera stream freezing Issue