AM62A7: Gstreamer - low framerate

Hello Stuart,

Your pipeline is most likely slowed down by the videoconvertscale plugin. Can you try to use "ticolorconvert" instead?

Regards,

Jianzhong

Jianzhong,

I tried using TIcolorconvert.  GstFPSDisplaySink was reporting an increase of about +2 FPS (from 6-7 to 9ish), and the frame rate is still quite low visually.  Additionally, there is about 0.5 seconds of latency.  I'm assuming that's because of the # of queue's.

Are there any other suggestions? I would ideally like to get close to 30fps with relatively low latency.

Hi Stuart,

If you remove colorconvert and multiscaler, your pipeline should still work. Can you try that?

gst-launch-1.0 -v v4l2src device=/dev/video4 ! queue ! video/x-raw, width=1920, height=1536, format=UYVY, framerate=30/1 ! fpsdisplaysink fps-update-interval=5000 name=rgb text-overlay=false sync=false video-sink="kmssink driver-name=tidss force-modesetting=true sync=false

Regards,

Jianzhong

Hi Jianzhong,

It looks like it needs to be 1920 x 1200 to be pushed to the display.

root@am62xx-evm:/opt/edgeai-gst-apps# gst-launch-1.0 -v v4l2src device=/dev/video4 ! queue ! video/x-raw, width=1920, height=1536, format=UYVY, framerate=30/1 ! fpsdisplaysink fps-update-interval=5000 name=rgb text-overlay=false sync=false video-sink="kmssink driver-name=tidss force-modesetting=true sync=false"
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb/GstKMSSink:kmssink0: display-width = 1920
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb/GstKMSSink:kmssink0: display-height = 1200
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb/GstKMSSink:kmssink0: sync = false
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesrc.c(3134): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Execution ended after 0:00:00.000468980
Setting pipeline to NULL ...
Freeing pipeline ...
root@am62xx-evm:/opt/edgeai-gst-apps# 

Hi Stuart,

If you are using the latest SDK 10.1 released code:

Can you try to modify the video-sink="kmssink driver-name=tidss sync=false force-modesetting=true" to "kmssink driver-name=tidss sync=false skip-vsync=true plane_id=31" and see if it helps?

Best Regards,

Suren

Suren,

I am using sdk `10.01.10.04`.  I tried the command:

root@am62xx-evm:/opt/edgeai-gst-apps# gst-launch-1.0 -v v4l2src device=/dev/video4 ! queue ! video/x-raw, width=1920, height=1200, format=UYVY, framerate=30/1 ! fpsdisplaysink fps-update-interval=5000 name=rgb text-overlay=false sync=false video-sink="kmssink driver-name=tidss sync=false skip-vsync=true plane_id=31"
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb/GstKMSSink:kmssink0: display-width = 1920
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb/GstKMSSink:kmssink0: display-height = 1200
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb/GstKMSSink:kmssink0: sync = false
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstFPSDisplaySink:rgb.GstGhostPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1200, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
/usr/src/debug/gstreamer1.0-plugins-good/1.22.12/sys/v4l2/gstv4l2src.c(950): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.020923520
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesrc.c(3134): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

The monitor in question is only a 1080p monitor - so I think it refuses to display anything that isn't the proper resolution.  If I grab a 4k monitor, will it work?

Hi Stuart,

Maybe try a different monitor. I'm able to stream 1936x1100 to a 1920x1080 monitor without scaling down:

gst-launch-1.0 \
v4l2src device=/dev/video-imx390-cam0 io-mode=dmabuf-import ! video/x-bayer, width=1936, height=1100, format=rggb12 ! \
tiovxisp sink_0::device=/dev/v4l-imx390-subdev0 sensor-name=SENSOR_SONY_IMX390_UB953_D3 \
dcc-isp-file=/opt/imaging/imx390/linear/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/imx390/linear/dcc_2a.bin format-msb=11 ! \
video/x-raw, format=NV12, width=1936, height=1100 ! kmssink driver-name=tidss sync=false

Regards,

Jianzhong

Hi jianzhong,

Sorry for the delay in a response.  I am still having trouble with this system.  most gstreamer pipelines do not seem to work - I must convert the color, and rescale the video.

I also noticed that you use `gst-launch-1.0 v4l2src device=/dev/video-imx390-cam0 io-mode=dmabuf-import`, which i cannot use because it seems to break any pipeline I create.  Am I missing software on the device, or is it because I am not receiving raw frames but instead UYVY frames?

Here's the latest attempt - using this command:
`gst-launch-1.0 -v v4l2src device=/dev/video4 ! queue ! video/x-raw, width=1920, height=1536, format=UYVY, framerate=30/1 ! kmssink driver-name=tidss sync=false`

root@am62xx-evm:/opt/edgeai-gst-apps# gst-launch-1.0 -v v4l2src device=/dev/video4 ! queue ! video/x-raw, width=
1920, height=1536, format=UYVY, framerate=30/1 ! kmssink driver-name=tidss sync=false
[ 24.144934] ds90ub960 5-0030: rx0 CSI error: 0x4
[ 24.149599] ds90ub960 5-0030: rx0 CSI checksum error

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 1920
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 1[ 24.549071] j721e-csi2rx 30102000.ticsi2rx: Width does not match (source 1920, sink 640)
200
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1536, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1536, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1536, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1536, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1536, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)[ 24.656883] ds90ub960 5-0030: rx0 CSI error: 0x4
progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/G[ 24.663885] ds90ub960 5-0030: rx0 CSI checksum error
stCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1536, format=(string)UYVY, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
/usr/src/debug/gstreamer1.0-plugins-good/1.22.12/sys/v4l2/gstv4l2src.c(950): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.023580170
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesrc.c(3134): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

Hi Stuart,

I saw the following error in your log:

j721e-csi2rx 30102000.ticsi2rx: Width does not match (source 1920, sink 640)

Can you share the output of "media-ctl -p"?

Thanks,

Jianzhong

Jianzhong,

After multiple attempts of trying to reproduce the error, I ended up getting the system to work with the command I previously mentioned.  It's unclear why, and I'm not certain it will work again tomorrow - but it may have to do with specifics of the camera (it tends to stream upon being turned on, instead of waiting for a command).

When streaming without color convert or the msc node - I get a reasonable framerate / latency - which is good.  Is there any way we can get the system to operate at a higher framerate with the rescale node - or is that not possible?

Either way - thank you for your help