• State TI Thinks Resolved
  • Locked Locked
  • Replies 12 replies
  • Answers 1 answer
  • Subscribers 100 subscribers
  • Views 1381 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5728: GStreamer H.264 latency issue

Ali Ansari
Expert 1260 points
Part Number: AM5728
Other Parts Discussed in Thread: TMDSEVM572X

hello

I have a problem with TMDSEVM572x in H264 Encoding/Decoding.

when I run command bellow for MPEG4 encoding/decoding :

gst-launch-1.0 -e v4l2src device=/dev/video1 num-buffers=-1 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! vpe num-input-buffers=8 ! ducatimpeg4enc bitrate=4000 ! mpeg4videoparse ! ducatimpeg4dec ! waylandsink

it is fine and latency between input capture video and output display is about 300ms (acceptable a little).

but when I run similar command for H264 encoding/decoding :

gst-launch-1.0 -e v4l2src device=/dev/video1 num-buffers=1000 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! vpe num-input-buffers=4 ! ducatih264enc bitrate=4000 ! h264parse ! ducatih264dec ! waylandsink

it's latency is very high (about 1 second) and only one frame is shown and others are dropped.

the warning messages is shown as follows:

WARNING: from element /GstPipeline:pipeline0/GstWaylandSink:waylandsink0: A lot of buffers are being dropped.
Additional debug info:
/home/gtbldadm/processor-sdk-linux-fido-build/build-CORTEX_1/arago-tmp-external-linaro-toolchain/work/cortexa15hf-vfp-neon-linux-gnueabi/gstreamer1.0/1.2.3-r0/gstreamer-1.2.3/libs/gst/base/gstbasesink.c(2791): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstWaylandSink:waylandsink0:
There may be a timestamping problem, or this computer is too slow.

for preventing dropped frames, I added ts-offset parameter and run command :

gst-launch-1.0 -e v4l2src device=/dev/video1 num-buffers=1000 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! vpe num-input-buffers=1 ! ducatih264enc bitrate=4000 ! queue ! h264parse ! ducatih264dec ! waylandsink ts-offset=1000000000

this way, no frames are dropped and all frames is shown correctly but with high latency about 1 second.

I wonder why this is happened? why is there such a high latency when H264 encoding and low latency when MPEG4 encoding?

is there any solution for preventing this hig latency?

thanks for any help

Ali

  • 0
    Expert 1260 points

    hello All

    is there Any help?

  • 0 in reply to Ali Ansari
    Expert 1260 points

    hello again

    I add "sync=false" at the end of command, it seems to be resolved.

    no frames are dropped and all frames are displayed correctly. but the latency is a little high yet (about 0.5 second).

    gst-launch-1.0 -e v4l2src device=/dev/video1 num-buffers=-1 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! vpe num-input-buffers=1 ! ducatih264enc bitrate=4000 ! h264parse ! ducatih264dec ! waylandsink sync=false

    how can I reduce the latency more? I think it can be less than 0.2 second because of using hardware accellerated video encode/decode (IVAHD)

    I'm waiting for any kindly response

    Ali

  • 0 in reply to Ali Ansari
    TI__Mastermind 43735 points

    Hello Ali,

    Could you please share which PSDK you are using?

    Regards,
    Krunal

  • 0 in reply to Krunal Bhargav34
    Expert 1260 points

    Hello Krunal

    thank you for reply.

    I use the default kernel image stored in SD Card sent with the EVM. 

    I think the linux version of it is 4.1.13  ( Linux am57xx-evm 4.1.13 ) . this is very old version I think.

    Regards,

    Ali

  • 0 in reply to Ali Ansari
    TI__Mastermind 43735 points

    Hello Ali,

    I have been able to replicate the issue on my setup and currently, I am internally debugging with our driver experts. I will keep you posted on any updates.

    Regards,
    Krunal

  • 0 in reply to Krunal Bhargav34
    TI__Mastermind 43735 points

    Hello Ali,

    Based on my discussion with our developer, the default number of B frames setting is 3. Please set inter-interval =1 for ducatih264enc: " ! ducatih264enc bitrate=4000 inter-interval =1 ! ...."

    Regards,
    Krunal

  • 0 in reply to Krunal Bhargav34
    Expert 1260 points

    Hello Krunal

    very thanks for response.

    I tested :

    gst-launch-1.0 -e v4l2src device=/dev/video1 num-buffers=-1 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! vpe num-input-buffers=1 ! ducatih264enc inter-interval=1 bitrate=4000 ! h264parse ! ducatih264dec ! waylandsink sync=false

    the latency is about 0.3 second (a little high yet).

    is it possible to reduse it more until reaching to 0.2 second (or less for example 0.1 second)?

    the need for this low latency is because I want to send H264 frames to other client side through network.

    I think this may increase the latency more and I should decrease the encode/decode latency as more as possible.

    thanks and regards,

    Ali

  • 0 in reply to Ali Ansari
    TI__Mastermind 43735 points

    Hello Ali,

    I recommend you to run the command "gst-inspect-1.0 ducatih264enc" and inspect various properties such as hrd-buffer-size, rate-preset and etc.

    Regards,
    Krunal

  • 0 in reply to Krunal Bhargav34
    Expert 1260 points

    Hi Krunal

    thank you for reply.

    excuse me for large delay.

    I tested above command with setting parametere hrd-buffer-size=4096 rate-preset=1 . but the result is the same : 

    gst-launch-1.0 -e v4l2src device=/dev/video1 num-buffers=-1 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! vpe num-input-buffers=1 ! ducatih264enc inter-interval=1 bitrate=4000 hrd-buffer-size=4096 rate-preset=1 ! h264parse ! ducatih264dec ! waylandsink sync=false

    also I added parameter rate-control-algo=1 and it give the result as same as previous:

    gst-launch-1.0 -e v4l2src device=/dev/video1 num-buffers=-1 io-mode=4 ! 'video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! vpe num-input-buffers=1 ! ducatih264enc inter-interval=1 bitrate=4000 hrd-buffer-size=4096 rate-preset=1 rate-control-algo=1 ! h264parse ! ducatih264dec ! waylandsink sync=false

    I would guess that the latency's main part is from display section. is that right?

    thanks,

    Ali

  • 0 in reply to Ali Ansari
    TI__Mastermind 43735 points

    Hi Ali,

    I am wondering if your camera sensor supports NV12 format because VPE could be removed from your pipeline and the captured output directly goes to ducati-encoder. Unfortunately, I don’t think any more optimization or any property setting is available to reduce latency.

    Regards,
    Krunal

  • 0 in reply to Krunal Bhargav34
    Expert 1260 points

    Hi Krunal

    very very thanks for reply.

    BR, 

    Ali

  • 0 in reply to Krunal Bhargav34
    Expert 1260 points

    Hi Krunal.


    thanks for your response.

    OK, I will try other things, perhaps find the solution to reduce latency.

    BR,

    Ali