Tool/software: Linux
I am using latest prebuild sdk(am57xx-evm-linux-sdk-bin-04.02.00.09.tar.xz) on evaluation board. We have problem that hardware h264 decoder causes unacceptably big latency. Sofware decoder displays same stream with latency under 100ms(but 80% cpu usage). Hardware accelerated video has delay of around three second if we don't use latency parameter in rtsp source (with or without jitterbuffer)
If we use latency parameter it needs 400ms for camera resolution 1024x768 to work properly. But smaller resolutions are worse, decoding 320x200 or 640x480 or 800x600 take minimum of 600ms latency.
With smaller latency gstreamer renders only one frame per second and displays following warning
WARNING: from element /GstPipeline:pipeline0/GstWaylandSink:waylandsink0: A lot of buffers are being dropped. Additional debug info: ../../../../gstreamer-1.8.3/libs/gst/base/gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstWaylandSink:waylandsink0: There may be a timestamping problem, or this computer is too slow.
Pipeline for software decoding that works properly is following:
CX=1024
CY=768
X=1024
Y=768
LATENCY=100
gst-launch-1.0 rtspsrc latency=$LATENCY location="rtsp://192.168.33.46/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=${CX}x${CY}&fps=25" ! \
rtpjitterbuffer ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! fpsdisplaysink video-sink=kmssink
Hardware accelerated one is following pipeline (problem is same with kmssink and waylandsink):
LATENCY=500
CX=320
CY=200
X=320
Y=200
gst-launch-1.0 rtspsrc latency=$LATENCY location="rtsp://192.168.33.46/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=${CX}x${CY}&fps=25" !\
rtpjitterbuffer ! rtph264depay ! h264parse ! ducatih264dec ! vpe ! 'video/x-raw, width='${X}', height='${Y} ! waylandsink