Gstreamer TI plugin internal circular buffer problem

  There could be two things that can lead to this - 

1) The incoming stream is being streamed at faster that realtime. This can happen if the streaming application is not using the PCRs in the TS stream to find the correct rate of transmission. You can verify this by using a VLC player on a PC to receive the same stream and display it.

2) Decoder/element after decode is not able to cope up with the incoming video frame rate. You can perhaps verify this by playing the same content from local file source and see if the playback shows lower framerate.

Also, do you see this problem with a specific stream? Can you also try with a lower video resolution content?

Thanks,

Satish

Hi Satish

Thank you for your reply.

Using "mplayer" instead of gstreamer+DSP I can see the video stream without problems, no visible problem at all. Obviously mplayer take around 90% of CPU resources compared to the 25%/30% of gstreamer+DSP.In add of this mplayer have a different cache engine that can be set throught command line. I tried with a cache of 1240 bytes and I don't see any video problem or noise right now.

I have only this devices able to generate video stream and I don't think is possible to configure it for generate a lower resolution video. But maybe I didn't understand very well your suggestion. What do you mean with "try with a lower resolution content"?

About play local file I have the same result also. I tried to play your demo video with the following command line:

/usr/bin/gst-launch filesrc location=/home/root/videos/davincieffect_480p30.264 !  typefind ! TIViddec2 displayBuffer=true ! TIDmaiVideoSink

The result of the display buffer is the following. As you can see it happen the same "problem" of the video stream. The buffer become full and stop data acquisition but in this case I don't see any video noise. Obviously the reasons is that the buffer engine can restart to get video data from the same point of stopped before since the content is in a local file that can be managed easily. Is not the case of the video strean that can not be stopped and continue to trasmit data also if the received can not get it.

(sorry for the format of the buffer data stream. I tried to make copy and paste but this is the result)

TIDmaiVideoSink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
[B                     |                                                      ]
[RW                    |                                                      ]
[R-W                   |                                                      ]
[R--W                  |                                                      ]
[R---W                 |                                                      ]
[R----W                |                                                      ]
[R-----W               |                                                      ]
[R------W              |                                                      ]
[R-------W             |                                                      ]
[R--------W            |                                                      ]
[R---------W           |                                                      ]
[R----------W          |                                                      ]
[R-----------W         |                                                      ]
[R------------W        |                                                      ]
[R-------------W       |                                                      ]
[R--------------W      |                                                      ]
[R---------------W     |                                                      ]
[R----------------W    |                                                      ]
[R-----------------W   |                                                      ]
[R------------------W  |                                                      ]
[R-------------------W |                                                      ]
[R--------------------W|                                                      ]
[R=====================W                                                      ]
[R=====================|W                                                     ]
[R=====================|=W                                                    ]
[R=====================|==W                                                   ]
[R=====================|===W                                                  ]
[R=====================|====W                                                 ]
[R=====================|=====W                                                ]
[R=====================|======W                                               ]
[R=====================|=======W                                              ]
[R=====================|========W                                             ]
[R=====================|=========W                                            ]
[R=====================|==========W                                           ]
[R=====================|===========W                                          ]
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[R=====================|============W                                         ]
[R=====================|=============W                                        ]
[R=====================|==============W                                       ]
[R=====================|===============W                                      ]
[R=====================|================W                                     ]
[R=====================|=================W                                    ]
[R=====================|==================W                                   ]
[R=====================|===================W                                  ]
[R=====================|====================W                                 ]
[R=====================|=====================W                                ]
[R=====================|======================W                               ]
[R=====================|=======================W                              ]
[R=====================|========================W                             ]
[R=====================|=========================W                            ]
[R=====================|==========================W                           ]
[R=====================|===========================W                          ]
[R=====================|============================W                         ]
[R=====================|=============================W                        ]
[R=====================|==============================W                       ]
[R=====================|===============================W                      ]
[R=====================|================================W                     ]
[R=====================|=================================W                    ]
[R=====================|==================================W                   ]
[R=====================|===================================W                  ]
[R=====================|====================================W                 ]
[R=====================|=====================================W                ]
[R=====================|======================================W               ]
[R=====================|=======================================W              ]
[R=====================|========================================W             ]
[R=====================|=========================================W            ]
[R=====================|==========================================W           ]
[R=====================|===========================================W          ]
[R=====================|============================================W         ]
[R=====================|=============================================W        ]
[R=====================|==============================================W       ]
[R=====================|===============================================W      ]
[R=====================|================================================W     ]
[R=====================|=================================================W    ]
[R=====================|==================================================W   ]
[R=====================|===================================================W  ]
[R=====================|====================================================W ]
[R=====================|=====================================================W]
[ R=====================|====================================================W]
[  R=====================|===================================================W]
[   R=====================|==================================================W]
[    R=====================|=================================================W]
[     R=====================|================================================W]
[      R=====================|===============================================W]
[       R=====================|==============================================W]
[        R=====================|=============================================W]
[         R=====================|============================================W]
[          R=====================|===========================================W]
[           R=====================|==========================================W]
[            R=====================|=========================================W]
[             R=====================|========================================W]
[              R=====================|=======================================W]
[               R=====================|======================================W]
[                R=====================|=====================================W]
[                 R=====================|====================================W]
[                  R=====================|===================================W]
[                   R=====================|==================================W]
[                    R=====================|=================================W]
[                     R=====================|================================W]
[                      R=====================|===============================W]
[                       R=====================|==============================W]
[                        R=====================|=============================W]
[                         R=====================|============================W]
[                          R=====================|===========================W]
[                           R=====================|==========================W]
[===========================B=====================|===========================]
[===========================WR=====================|==========================]
[============================B=====================|==========================]
[============================WR=====================|=========================]
[=============================B=====================|=========================]
[=============================WR=====================|========================]
[==============================B=====================|========================]
[==============================WR=====================|=======================]
[===============================B=====================|=======================]
[===============================WR=====================|======================]
[================================B=====================|======================]
[================================WR=====================|=====================]
[=================================B=====================|=====================]
[=================================WR=====================|====================]
[==================================B=====================|====================]
[==================================WR=====================|===================]
[===================================B=====================|===================]
[===================================WR=====================|==================]
[====================================B=====================|==================]
[====================================WR=====================|=================]
[=====================================B=====================|=================]
[=====================================WR=====================|================]
[======================================B=====================|================]
[======================================WR=====================|===============]
[=======================================B=====================|===============]
[=======================================WR=====================|==============]
[========================================B=====================|==============]
[========================================WR=====================|=============]
[=========================================B=====================|=============]
[=========================================WR=====================|============]
[==========================================B=====================|============]
[==========================================WR=====================|===========]
[===========================================B=====================|===========]
[===========================================WR=====================|==========]
[============================================B=====================|==========]
[============================================WR=====================|=========]
[=============================================B=====================|=========]
[=============================================WR=====================|========]
[==============================================B=====================|========]
[==============================================WR=====================|=======]
[===============================================B=====================|=======]
[===============================================WR=====================|======]
[================================================B=====================|======]
[================================================WR=====================|=====]
[=================================================B=====================|=====]
[R=====================|==========================W                           ]
[R=====================|===========================W                          ]
[R=====================|============================W                         ]
[R=====================|=============================W                        ]
[R=====================|==============================W                       ]
[R=====================|===============================W                      ]
[R=====================|================================W                     ]
[R=====================|=================================W                    ]
[R=====================|==================================W                   ]
[R=====================|===================================W                  ]
[R=====================|====================================W                 ]
[R=====================|=====================================W                ]
[R=====================|======================================W               ]
[R=====================|=======================================W              ]
[R=====================|========================================W             ]
[R=====================|=========================================W            ]
[R=====================|==========================================W           ]
[R=====================|===========================================W          ]
[R=====================|============================================W         ]
[R=====================|=============================================W        ]
[R=====================|==============================================W       ]
[R=====================|===============================================W      ]
[R=====================|================================================W     ]
[R=====================|=================================================W    ]
[R=====================|==================================================W   ]
[R=====================|===================================================W  ]
[R=====================|====================================================W ]
[R=====================|=====================================================W]
[ R=====================|====================================================W]
[  R=====================|===================================================W]
[   R=====================|==================================================W]
[    R=====================|=================================================W]
[     R=====================|================================================W]
[      R=====================|===============================================W]
[       R=====================|==============================================W]
[        R=====================|=============================================W]
[         R=====================|============================================W]
[          R=====================|===========================================W]
[           R=====================|==========================================W]
[            R=====================|=========================================W]
[             R=====================|========================================W]
[              R=====================|=======================================W]
[               R=====================|======================================W]

Hi ,

The buffer behavior for local file playback is as per my expectation, since the file read would happen way fast then the real required rate and would end up filling the buffer. During local file playback, did you see any pauses in the video ?

By lower resolution, I meant lower video size content.  Instead of your own device, you can also try to use VLC to stream video to your system.

What platform are you on? What is the display standard? PAL/ NTSC?

Thanks,

Satish

Hi

I'm sorry, I forget to  specity that play of local file is perfect, no pauses at all.

I'm working on OMAP 3730 platform with a display connected using LVDS. I play the DVBT video stream using the original size, no resize is applied.

Do you suspect some particular problem?

It seem, but is only my impression, that the decoding DSP is not fast enought to guarantee the required frame number needed by the stream. Could this hypothesis be true?

 Since the circular buffer is overflowing , it hints that one of the elements in the pipeline following circular buffer is not keeping up with realtime. Since the pipeline has only decode and display after the circular buffer, it seems either decode or display is holding the pipeline from running realtime. To find out whether its decoder not running realtime , I wanted to check with the lower resolution content. But it seems you are decoding SD resolution , and OMAP 3730 should be able to easily decode that.

Also was doubting it the display is running at a lower fps.. for e.g. if display is running at 25fps (PAL), while content is 30 fps then you would see the similar build up. 

Thanks,

Satish 

Hi

I'm decoding standard DVBT channels so I suppose is SD resolution too.

About display FPS how can I check this? In case of gstreamer the TI plugin "copy" decoded frames directly to the display video memory than how could I know this info?

Thank you for your help

Hi again ;-)

I just finished to add to the TI gstreamer plugin the deinterlacing feature. I use the DSP internal deinterlacer "engine" for deinterlace the video as found in some source code example in the DVSDK. This modify is working very well and I can see the video denterlaced correctly. The only problem is that this "additonal" work for the DSP made worse the buffer problem and now, if deinterlacing enabled, the buffer become full more faster than whitout this feature active. This seem to confirm me my idea regardins some "problem" of DSP to decode video frames as fastern as needed. What do you think about?

Thank you

Hi

Some news regarding this problem?

regards

I have created a small fix for this that estimates the approximate delay in filling the video decoder’s input circular buffer. It then adds this delay to the timestamp for the video frames so that it matches more closely the displays timer.

The problem you were seeing was:

Decoder is decoding frames with timestamp base time 0 and giving them to display which has a time of X. Display realizes that decoded frames are in the past and so gives buffer back to decoder immediately so that it can try and catch up to display time. This has two effects:

-          Displaying frames at approx 60fps rather than 30fps, so the fast playback you saw

-          As decoder is emptying its circular input buffer faster than it is being filled by network stream it soon stalls as buffer has reached minimum level, so the stalling you saw.

As the decoder never caught up with the display time you were in an endless loop of decode fast/ stall / decode fast/ stall etc.

The problem with this solution is that as we have manipulated the video time stamps, if you have audio decode as well then the A/V sync will be wrong. I don’t think it is possible to handle this in gst-launch as you’d need to have an application querying the video decoder timestamp offset so that same adjustment can be made to the audio timestamps.

As some additional information for community the slow decode was due to your use of NFS, when you moved to local file system you saw this problem of fast playback then stall.

Iain  

Hi

Thank you for your help

Just tested your fix but it seem to don't work correctly. Once compiled and executed the plugin with the fix applied I can see the same problem of buffer empty as before and, in addition, the video showed is more fastern as normal. Is there some particular option I should use in gstreamer command line for have the video working well?.

I currently make my test using the following command line:

/usr/bin/gst-launch filesrc location=/dev/dib07x0/0/endpoint2in ! typefind ! mpegtsdemux name=demux demux.video_0101 ! TIViddec2 genTimeStamps=false codecName=
mpeg2dec engineName=codecServer framerate=25 displayBuffer=true ! TIDmaiVideoSink

Note that I currently receive the video stream from a device and not from udp but I don't think this could be a significative difference.

If you need I make some other tests for help you feel free to ask me.

Thank you

Regards

I had used genTimeStamps=true

Hi

Using genTimeStamps=true doesn't show any video on the screen for me. However, also without see any video I can see the buffer report generated by the option displayBuffer=true and the problem of buffer empty is still present as you can see below (line R---------------------W mean buffer empty):

R---------------------W                                                 ]
[     R=====================|W                                                ]
[     R=====================|==W                                              ]
[     R=====================|===W                                             ]
[     R=====================|====W                                            ]
[     R=====================|=====W                                           ]
[     R=====================|======W                                          ]
[      R=====================|=====W                                          ]
[       R=====================|====W                                          ]
[        R=====================|===W                                          ]
[         R=====================|==W                                          ]
[         R=====================|===W                                         ]
[            R=====================|W                                         ]
[             R=====================W                                         ]
[             R=====================|W                                        ]
[              R=====================W                                        ]
[              R=====================|W                                       ]
[               R=====================W                                       ]
[                 R-------------------W |                                     ]
[                 R--------------------W|                                     ]
[                 R---------------------W                                     ]
[                 R=====================|W                                    ]
[                 R=====================|=W                                   ]
[                 R=====================|==W                                  ]
[                 R=====================|===W                                 ]
[                 R=====================|====W                                ]
[                 R=====================|=====W                               ]
[                  R=====================|====W                               ]
[                   R=====================|===W                               ]
[                    R=====================|==W                               ]
[                    R=====================|===W                              ]
[                        R=====================W                              ]
[                        R=====================|=W                            ]
[                        R=====================|==W                           ]
[                        R=====================|===W                          ]
[                        R=====================|====W                         ]
[                        R=====================|=====W                        ]
[                        R=====================|======W                       ]
[                         R=====================|=====W                       ]
[                          R=====================|====W                       ]
[                           R=====================|===W                       ]
[                              R=====================|W                       ]
[                              R=====================|=W                      ]
[                               R=====================|W                      ]
[                               R=====================|=W                     ]
[                                R=====================|W                     ]
[                                 R=====================W                     ]
[                                 R=====================|W                    ]
[                                   R--------------------W|                   ]
[                                   R---------------------W                   ]
[                                   R=====================|W                  ]
[                                   R=====================|=W                 ]
[                                   R=====================|==W                ]
[                                   R=====================|===W               ]
[                                   R=====================|====W              ]
[                                   R=====================|=====W             ]
[                                    R=====================|====W             ]
[                                    R=====================|=====W            ]
[                                     R=====================|====W            ]
[                                      R=====================|===W            ]
[                                       R=====================|==W            ]
[                                         R=====================|W            ]
[                                          R=====================W            ]
[                                           R--------------------W|           ]
[                                           R=====================W           ]
[                                           R=====================|W          ]

Can you try following log options pre-pended to command line

GST_DEBUG=TIDmaiVideoSink:5,TIViddec2:5,basesink:5,TICircBuffer:5 and attach them to the post.

I get correct frame rate playback with both tidisplaysink2 and TIDmaiVideoSink sinks on the EVM.

Iain

Hi

Here you have. I added two logs. One generated with the option genTimeStamps=false and the other with genTimeStamps=true.

Thank you

Regards

1884.log.zip

 Hi FabF:

I'm doing the same thing on android (DM3730,dvsdk_dm3730-evm_4_01_00_09_setuplinux). I use the DSP internal deinterlacer "engine" for deinterlace the video as found in some source code example in the DVSDK(encode demo) after decoding in gsttidmaidec.c. But It block in UNIVERSAL_process function always. I found that because i use the omapresize in overlay first. Is the deinterlacer "engine" cann't use with omapresize together? And how can i do for this? Can you give me some suggestion?
    Thank you very much.

Hi

I'm sorry but I'm working on a standard Linux system not android. By the way DVSDK version 4.01 is unsupported anymore. I'm currently working with 4.02 version where the TI plugin for gstreamer has been modified from the old version. I just noted a new 4.03 version of DVSDK is out. Anyway my problem with circular buffer is not connected to deinterlace engine so I don't know how to help you. Maybe, if you use omapresize for the video, I could suggest you to use the "tidisplaysink2" istead of "TIDmailVideoSink" since this first one have included some params for set the position and the size of the video in the screen (check the source for more info. I fear this new component has been introduced from DVSDK 4.02).

Hi FabF:

    Thanks for your reply.

    Do you mean it's impossible to add the deinterlacing feature in the DVSDK 4.0.1? I'am warking on the BRANCH_DDOMPE before. Now i add the deinterlacing in the trunk and the block issue is not appear again. But the output buffer can't display. Here is my code:

static GstFlowReturn gst_tidmaivideosink_render(GstBaseSink * bsink,
GstBuffer * buf) {

BufferGfx_Attrs gfxAttrs = BufferGfx_Attrs_DEFAULT;
Buffer_Handle hDispBuf = NULL;
Buffer_Handle hTestDispBuf = NULL;
Buffer_Handle inBuf = NULL;
Buffer_Handle inBufTest = NULL;
static Buffer_Handle cPrevBuf = NULL;
static Int framecount = 0;
BufferGfx_Dimensions inDimSave;
GstTIDmaiVideoSink *sink = GST_TIDMAIVIDEOSINK_CAST(bsink);

GST_DEBUG("Begin, buffer %p",buf);

/* The base sink send us the first buffer twice, so we avoid processing
* it again, since the Display_put may fail on this case when using
* pad_allocation
*/
if (sink->prerolledBuffer == buf){
GST_DEBUG("Not displaying previously pre-rolled buffer");
sink->prerolledBuffer = NULL;
return GST_FLOW_OK;
}
sink->prerolledBuffer = NULL;

/* If the input buffer is non dmai buffer, then allocate dmai buffer and
* copy input buffer in dmai buffer using memcpy routine.
*/
if (GST_IS_TIDMAIBUFFERTRANSPORT(buf)) {
inBuf = GST_TIDMAIBUFFERTRANSPORT_DMAIBUF(buf);
} else {
/* allocate DMAI buffer */
if (sink->tempDmaiBuf == NULL) {

GST_DEBUG("Input buffer is non-dmai, allocating new buffer");
gfxAttrs.dim.width = sink->width;
gfxAttrs.dim.height = sink->height;
gfxAttrs.dim.lineLength = BufferGfx_calcLineLength(sink->width,
sink->colorSpace);
gfxAttrs.colorSpace = sink->colorSpace;
sink->tempDmaiBuf = Buffer_create(GST_BUFFER_SIZE(buf),
BufferGfx_getBufferAttrs(&gfxAttrs));

if (sink->tempDmaiBuf == NULL) {
GST_ELEMENT_ERROR(sink,STREAM,FAILED,(NULL),
("Failed to allocate memory for the input buffer"));
return GST_FLOW_UNEXPECTED;
}
}
inBuf = sink->tempDmaiBuf;

memcpy(Buffer_getUserPtr(inBuf), buf->data, buf->size);
}

/* Check if we can allocate a new buffer, otherwise we may need
* to drop the buffer
*/
BufferGfx_getDimensions(inBuf, &inDimSave);
if ((sink->numAllocatedBuffers >=
(BufTab_getNumBufs(Display_getBufTab(sink->hDisplay)) - 1)) &&
(sink->numUnusedBuffers == 0)){
GST_ELEMENT_WARNING(sink,RESOURCE,NO_SPACE_LEFT,(NULL),
("Dropping incoming buffers because no display buffer"
" available"));
return GST_FLOW_OK;
} else {
GST_DEBUG("Obtaining display buffer");
hDispBuf = gst_tidmaivideosink_get_display_buffer(sink,inBuf);
if (!hDispBuf){
return GST_FLOW_UNEXPECTED;
}
}
/* First time de-interlacer does not need previous buffer*/
if (framecount == 0){
/* Deinterlacing the Capture frame */
if (I2p_process(hDein, inBuf, NULL, hDispBuf) < 0) {
GST_ERROR("Failed to Deinterlace video Buffer\n");
}
}
else {
/* Deinterlacing the Capture frame */
if (I2p_process(hDein, inBuf, cPrevBuf, hDispBuf) < 0) {
GST_ERROR("Failed to Deinterlace video Buffer\n");
}
}
framecount++;
cPrevBuf = inBuf;

/* Send filled buffer to display device driver to be displayed */
if (Display_put(sink->hDisplay, hDispBuf) < 0) {
GST_ELEMENT_ERROR(sink,STREAM,FAILED,(NULL),
("Failed to put the buffer on display"));
return GST_FLOW_UNEXPECTED;
}

}

what's the problem? Can you give me some suggestion again? Or can you give me the source code about the deinterlacing feature If you are convenient?

Thank you very much.

Hi

I'm not working with deinterlace right now. I have this problem with circular buffer and this is indipendend with deinterlace feature. My suggestion to move to the new DVSDK  would not make you understand that is impossible to add deinterlace to the 4.01 version. I would only to suggest you to make your modifies to the new version of the TI gstreamer plugin since this plugin is changed from the old 4.00, 4.01 version t0 40.2 and, suppose, 4.03. If you work for a solution to the 4.01 version of risk to have to make the work again to the 4.02 version since there are some differences...

Hi FabF:

      Thans again.

      I have add the deinterlace in gsttidisplaysink2.c successful. But the video is looks like interlaced. I'am sure the function UNIVERSAL_process has been called. The source video is NTSC (H264+AAC).