AM5728: TMDSEVM572x GStreamer pipeline : appsrc h264parse ducatih264dec waylandsync : latency is high about 12 frames

Ali Ansari

Part Number: AM5728

hello
I have written a C code for running a h264 decoder pipeline. the gstreamer pipeline I used is as follows :

appsrc -> h264parse -> ducatih264dec -> waylandsync

also I have inited and started a UDP socket that receives H264 frames from PC.
a thread has been started in EVM for receiving UDP packets and exctracting H264 frames and then pushing it in appsrc buffer.
In PC side, a software reads H264 frames from a file and send it to EVM frame by frame.
when I debug the PC software and run it step by step, it is seen that after sending about 11 or 12 frames to EVM, the output display of EVM starts to show the frames.
why does this high latency exist?
I also set the parameter "is-live" of appsrc to TRUE and parameter "max-reorder-frames" of ducatih264dec to 0. but still the high latency appears.
what can I do for reducing the latency to a value smaller than 2 frames?

I must declare that a similar pipeline exactly as above pipeline ran in PC side (in C code) and the parameter "is-live" of appsrc is set to TRUE. the latency between pushing h264 frames to appsrc and display was 2 frames. note that when setting parameter "is-live" of appsrc to FALSE, the latency was very high same as what is seen in EVM (about 12 frames).

now, I wonder that what is the difference between PC side C code and EVM C code? although all elements are the same and their parameters are same as each other.

any help will be appreciated.

Best Regards,

Ali

over 5 years ago

0 Krunal Bhargav34 over 5 years ago

TI__Mastermind 44755 points

Hi Ali,

Is the current thread related to the previous gstreamer latency thread? Are you using the same gstreamer pipeline as your previous thread?

Regards,
Krunal

0 Ali Ansari over 5 years ago in reply to Krunal Bhargav34

Expert 1260 points

Hi Krunal

no, this question is different than that thread. in that thread, I had been testing the full pipeline from camera to display and all elements had run and gotten data in EVM internal.

but in this thread, I have tested other pipeline in which the data (H264 encoded frames) is received from PC via a UDP socket.

I saw that the latency between appsrc's input pad and display is very high in EVM (about 12 frames).

on the other hand, the same pipeline was ran in PC and the same H264 frames was sent to appsrc's input pad. the latency between appsrc's input and display was low about 2 frames.

an other note is that on previous thread the pipeline was run by a gst-launch-1.0 command via terminal. but in this question, a C code was written and a pipeline was implemented by software.

Regards,

Ali

0 Krunal Bhargav34 over 5 years ago in reply to Ali Ansari

TI__Mastermind 44755 points

Hello Ali,

Based on my internal discussions, any delays in seeing the frame on display could be related to the "display-delay" setting of ducati-h264 decoder. If possible, please set the "display-delay" to 0 and the decoder will output a buffer for display immediately after decoding. Also, ensure that the input stream does not have B frames, otherwise the frames will look out of order.

Regards,
Krunal

0 Ali Ansari over 5 years ago in reply to Krunal Bhargav34

Expert 1260 points

Hello Krunal

thanks a lot for response.

when I run command gst-inspect-1.0 ducatih264dec , I see no property with name "display-delay".

only properties "name" , "parent' , "version" , "max-reorder-frames" and "codec-debug-info" are shown.

BR,

Ali

0 Krunal Bhargav34 over 5 years ago in reply to Ali Ansari

TI__Mastermind 44755 points

Hi Ali,

I am wondering if you could please try using our latest PSDK. Here is my output log:

root@am57xx-evm:~# uname -a                                                                                                                                                                                 
Linux am57xx-evm 4.19.59-g5f8c1c6121 #1 SMP PREEMPT Sat Oct 19 16:29:33 UTC 2019 armv7l GNU/Linux                                                                                                           
root@am57xx-evm:~# gst-inspect-1.0 ducatih264dec                                                                                                                                                            
Factory Details:                                                                                                                                                                                            
  Rank                     primary + 1 (257)                                                                                                                                                                
  Long-name     [1738273.031167] omap-iommu 55082000.mmu: 55082000.mmu: version 2.1                                                                                                                         
           DucatiH264Dec                                                                                                                                                                                    
  Klass                    Codec/Decoder/Video                                                                                                                                                              
  Description              Decodes video in H.264/bytestream format with ducati                                                                                                                             
  Author                   Rob Clark <rob@ti.com>                                                                                                                                                           
                                                                                                                                                                                                            
Plugin Details:                                                                                                                                                                                             
  Name                     ducati                                                                                                                                                                           
  Description              Hardware accelerated codecs for OMAP4                                                                                                                                            
  Filename                 /usr/lib/gstreamer-1.0/libgstducati.so                                                                                                                                           
  Version                  1.0.0                                                                                                                                                                            
  License                  LGPL                                                                                                                                                                             
  Source module            gst-ducati                                                                                                                                                                       
  Binary package           GStreamer                                                                                                                                                                        
  Origin URL               http://gstreamer.net/                                                                                                                                                            
                                                                                                                                                                                                            
GObject                                                                                                                                                                                                     
 +----GInitiallyUnowned                                                                                                                                                                                     
       +----GstObject                                                                                                                                                                                       
             +----GstElement                                                                                                                                                                                
                   +----GstDucatiVidDec                                                                                                                                                                     
                         +----GstDucatiH264Dec                                                                                                                                                              
                                                                                                                                                                                                            
Pad Templates:                                                                                                                                                                                              
  SINK template: 'sink'                                                                                                                                                                                     
    Availability: Always                                                                                                                                                                                    
    Capabilities:                                                                                                                                                                                           
      video/x-h264                                                                                                                                                                                          
          stream-format: byte-stream                                                                                                                                                                        
              alignment: au                                                                                                                                                                                 
                  width: [ 16, 2048 ]                                                                                                                                                                       
                 height: [ 16, 2048 ]                                                                                                                                                                       
      video/x-h264                                                                                                                                                                                          
          stream-format: byte-stream                                                                                                                                                                        
              alignment: au                                                                                                                                                                                 
                  width: [ 16, 2048 ]                                                                                                                                                                       
                 height: [ 16, 2048 ]                                                                                                                                                                       
                profile: { (string)high, (string)high-10-intra, (string)high-10, (string)high-4:2:2-intra, (string)high-4:2:2, (string)high-4:4:4-intra, (string)high-4:4:4, (string)cavlc-4:4:4-intra }    
                  level: { (string)1, (string)1b, (string)1.1, (string)1.2, (string)1.3, (string)2, (string)2.1, (string)2.2, (string)3, (string)3.1, (string)3.2, (string)4, (string)4.1, (string)4.2, (str
ing)5, (string)5.1 }                                                                                                                                                                                        
                                                                                                                                                                                                            
  SRC template: 'src'                                                                                                                                                                                       
    Availability: Always                                                                                                                                                                                    
    Capabilities:                                                                                                                                                                                           
      video/x-raw                                                                                                                                                                                           
                 format: NV12                                                                                                                                                                               
                  width: [ 1, 2147483647 ]                                                                                                                                                                  
                 height: [ 1, 2147483647 ]                                                                                                                                                                  
              framerate: [ 0/1, 2147483647/1 ]                                                                                                                                                              
                                                                                                                                                                                                            
Element has no clocking capabilities.                                                                                                                                                                       
Element has no URI handling capabilities.                                                                                                                                                                   
                                                                                                                                                                                                            
Pads:                                                                                                                                                                                                       
  SINK: 'sink'                                                                                                                                                                                              
    Pad Template: 'sink'                                                                                                                                                                                    
  SRC: 'src'                                                                                                                                                                                                
    Pad Template: 'src'                                                                                                                                                                                     
                                                                                                                                                                                                            
Element Properties:                                                                                                                                                                                         
  name                : The name of the object                                                                                                                                                              
                        flags: readable, writable                                                                                                                                                           
                        String. Default: "ducatih264dec0"                                                                                                                                                   
  parent              : The parent of the object                                                                                                                                                            
                        flags: readable, writable                                                                                                                                                           
                        Object of type "GstObject"                                                                                                                                                          
  version             : The codec version string                                                                                                                                                            
                        flags: readable                                                                                                                                                                     
                        String. Default: null                                                                                                                                                               
  max-reorder-frames  : The maximum number of frames needed for reordering output frames. Only meaningful for codecs with B frames. 0 means no reordering. This value will be used if the correct value cann
ot be inferred from the stream. Too low a value may cause misordering, too high will cause extra latency.                                                                                                   
                        flags: readable, writable                                                                                                                                                           
                        Integer. Range: 0 - 16 Default: 16                                                                                                                                                  
  codec-debug-info    : Gather and log relevant debug information from the codec. What is gathered is typically codec specific                                                                              
                        flags: readable, writable                                                                                                                                                           
                        Boolean. Default: false                                                                                                                                                             
  display-delay       : Display delay configuration of h264 decoder                                                                                                                                         
                        flags: readable, writable                                                                                                                                                           
                        Integer. Range: -1 - 16 Default: -1                                                                                                                                                 
root@am57xx-evm:~#

Here is the commit message that added the property: https://git.ti.com/cgit/glsdk/gst-plugin-ducati/commit/?id=19e911bb45757d8341ffd7819ca783ca04f5e3b1

Regards,
Krunal

0 Ali Ansari over 5 years ago in reply to Krunal Bhargav34

Expert 1260 points

Hi Krunal

thank you very much.

OK. your're right. my EVM SDK version is old (4.1.13). I used the original SDK installed on SD Card that involved in TMDSVM572x board.

I should build and use the newer version.

I will do this.

Best Regards,

Ali

0 Ali Ansari over 5 years ago in reply to Krunal Bhargav34

Expert 1260 points

Hello again Krunal

very very thanks.

I tested with newer SDK (kernel version 4.19.38).

that's OK. the element ducatih264dec has property "display-delay" as you said.

I set this parameter to 0 and tested again this pipeline.

the latency got removed and now is 2 frames. :-)

very thanks and with the Best Regards

Ali

Processors

Processors forum

AM5728: TMDSEVM572x GStreamer pipeline : appsrc h264parse ducatih264dec waylandsync : latency is high about 12 frames