SK-AM62-LP: In the MIPI-CSI2 camera pipeline, camera frames are dropped at higher frame rates.

Hi Gopal,

Videoconvert is a Software based plugin and would definitely be taking a lot of CPU on AM62x. Does adding a queue element in the pipeline help improve the FPS?

Also can you share what camera are you testing with and the media-ctl -p /dev/media0 output when camera is connected on to the EVM.

Best Regards,

Suren

Hi Suren,

Suren Porwar said:

Does adding a queue element in the pipeline help improve the FPS?

No

Pipeline with queue:-  

root@am62xx-lp-evm:~# gst-launch-1.0 v4l2src device=/dev/video0 ! \
>     queue max-size-buffers=4 ! \
>     video/x-raw,format=GRAY8,width=1120,height=1360,framerate=60/1 ! \
>     queue max-size-buffers=4 ! \
>     videoconvert ! \
>     queue max-size-buffers=4 ! \
>     kmssink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
There may be a timestamping problem, or this computer is too slow.
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:05.738267740
Setting pipeline to NULL ...
Freeing pipeline ...

Suren Porwar said:

Also can you share what camera are you testing with and the media-ctl -p /dev/media0 output when camera is connected on to the EVM.

media-ctl -p /dev/media0
Media controller API version 6.12.17

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial          
bus info        platform:30102000.ticsi2rx
hw revision     0x1
driver version  6.12.17

Device topology
- entity 1: 30102000.ticsi2rx (5 pads, 5 links, 1 route)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: SINK
                [stream:0 fmt:Y8_1X8/1120x1360 field:none]
                <- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
        pad1: SOURCE
                [stream:0 fmt:Y8_1X8/1120x1360 field:none]
                -> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
        pad2: SOURCE
                -> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
        pad3: SOURCE
                -> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
        pad4: SOURCE
                -> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]

- entity 7: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links, 1 route)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        routes:
                0/0 -> 1/0 [ACTIVE]
        pad0: SINK
                [stream:0 fmt:Y8_1X8/1120x1360 field:none]
                <- "vd56g3_iil 4-0010":0 [ENABLED,IMMUTABLE]
        pad1: SOURCE
                [stream:0 fmt:Y8_1X8/1120x1360 field:none]
                -> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
        pad2: SOURCE
        pad3: SOURCE
        pad4: SOURCE

- entity 13: vd56g3_iil 4-0010 (1 pad, 1 link, 0 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev2
        pad0: SOURCE
                [stream:0 fmt:Y8_1X8/1120x1360 field:none colorspace:raw xfer:none quantization:full-range
                 crop.bounds:(0,0)/1124x1364
                 crop:(2,2)/1120x1360]
                -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]

- entity 19: 30102000.ticsi2rx context 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
        pad0: SINK
                <- "30102000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 25: 30102000.ticsi2rx context 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
        pad0: SINK
                <- "30102000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 31: 30102000.ticsi2rx context 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
        pad0: SINK
                <- "30102000.ticsi2rx":3 [ENABLED,IMMUTABLE]

- entity 37: 30102000.ticsi2rx context 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
        pad0: SINK
                <- "30102000.ticsi2rx":4 [ENABLED,IMMUTABLE]

Regards,
Gopal Sharma

Hi Gopal,

I have assigned your query to our camera expert for further support. Expect a response soon.

Also can you add kmssink driver-name=tidss sync=false on the kmssink element.

What framerates are supported with your camera sensor? 

You can refer the AM62 Academy on dumping the framesizes  and frameintervals supported with your camera

https://dev.ti.com/tirex/explore/node?node=A__Afvqyi8mUm05676JZJ-UlQ__AM62-ACADEMY__uiYMDcq__LATEST

Linux->Develop Linux on TI EVM ->Use Camera section in the above link.

Best Regards,

Suren