I'm performing MPEG4 decoding with an adaption of the demo code that comes with the DM6437 video board. The codec is the supplied codec which is MPEG4 progressive only simple profile and I and P frames only (no B frames).
The particular MPEG4 sequence I use is filled with action but has a number of scene changes that go to black and stay like that for a second or so. This results in pictures that have 'normal' sizes of 5k to 50k or there abouts with the black scene change pictures having sizes of about 211 bytes. I have determined this by looking at the buffers I obtain from my TS demuxer before they are passed to the codec support code.
The demo code which I have adapted drives the codec and uses a threshold value in order to decide when to continue decoding frames. In other words a buffer comes in that consists of an access unit and is padded onto the remainder of the existing buffer. The existing buffer is used to decode frames until its occupancy goes below the threshold value.
However, there is a problem: When a (highly compressed) scene change is present the last action frame will freeze on the monitor and the 30 or more scene change pictures will pile up in the buffer until the threshold is exceeded. The result of this is that the last picture before the black scene change remains on the monitor considerably longer than it should because no decoding is taking place while the scene changes pictures arrive into the buffer and are not decoded because the buffer occupancy is less than the threshold.
My software takes a 13818-1 transport stream that is streamed live from VLC over Ethernet. I demux the MPEG4 video into buffers where each MPEG4 video buffer is where the payload_unit_start_indicator was set in the TS PES header.
I've tried looking for the MPEG4 start codes and observed the 0x00 0x00 0x01 0xb6 / 0xb0 / 0xff (i.e. vop_start_code, visual_object_sequence_start_code and system start code) and attempted to circumvent the threshold by telling the decoder to take only a single picture out of the buffer: i.e. no 'while' loop, only decode frame. However, when I do this the codec reports that it is missing data.
What is the threshold and why is it necessary? In the demo code it appears to be defined as an arbitrary value that no compressed MPEG4 picture could be larger than.
The present codec seems to assume that it is presented with a buffer that is at least large enough to contain a single compressed picture.
The "MPEG4 Simple Profile Decoder on C64x+" section 4.2.1.2 XDM_AlgBufInfo has a note at the bottom of the page "There is no restriction on input buffer size except that it should contain atleast one frame of encoded data.".
However, I would like to use the codec so that I can give it a smaller buffer at a time and it only spits a picture when it has had enough data. Is this possible?
Specifically, how can I avoid many highly compressed frames piling into the buffer and waiting to be decoded because of this threshold value?
I thought about putting rate control after the decoder but before the YUV output. However, this would necessitate about a second or so of buffering YUV data in order to get around the latency that is generated by the threshold when using pictures that may be highly compressed.
Do I need to "compress" (in quotes deliberately) my sequence such that the I and P pictures have roughly the same size regardless of their content, i.e. making use picture stuffing in order to achieve a near constant size for each "compressed" picture?
FYI, I'm using dvsdk 1.01 with codec engine 1.20, codecs 1.10 etc but this issue does not appear to be version specific, it seems more to do with the modus operandii of the decoder and how it is used.
