This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DM36x H.264 Decoder - Buffers...

Hi All,


I am trying to decode a live IP Camera stream using the Vdec2 H.264 decoder on the DM368, but I am having trouble with buffers.

I basically followed the encodedecode example application provided with the DVSDK.

What I am seeing is the decoder is asking for 10 buffers for one camera (720P), but a different camera (1080P) it's asking for 17 buffers.


Now, as I understand, from the documentation:

The maximum number of buffers
that is required by the codec. The
maximum number of buffers can be
IVIDDEC2_MAX_IO_BUFFERS .
In this decoder implementation it is
always set to 17, if levelLimit
field of IVIDDEC2_Params is not
set. However, in case levelLimit
is set to an appropriate valid value,
maxNumDisplayBufs is derived
based on levelLimit,
maxWidth and maxHeight
fields of IVIDDEC2_Params.

Code:

    params.maxWidth = Dmai_roundUp(IWIDTH_MAX, BUF_LINE_ALIGN);
    params.maxHeight = Dmai_roundUp(IHEIGHT_MAX, BUF_HEIGHT_ALIGN);
    params.forceChromaFormat = XDM_YUV_420SP;
    params.size = sizeof(IH264VDEC_Params);
    extnParams.viddecParams = params;

    extnParams.displayDelay = 0;
    extnParams.levelLimit = LEVEL_4_2;
    extnParams.disableHDVICPeveryFrame = 0;
    extnParams.inputDataMode = IH264VDEC_TI_ENTIREFRAME;
    extnParams.sliceFormat = IH264VDEC_TI_BYTESTREAM;
    extnParams.frame_closedloop_flag = 0;

    dynParams.displayWidth = params.maxWidth;            // line-length is not correctly interpreted by codec, should set the displayWidth parameter.
    dynParams.frameOrder = IVIDDEC2_DECODE_ORDER;             // try this
    dynParams.frameSkipMode = IVIDEO_NO_SKIP;
    dynParams.decodeHeader = XDM_DECODE_AU;

    // Create Decoder (XDM 1.2)
    hVd2 = Vdec2_create(argsp->engine, "h264dec", (VIDDEC2_Params*)&extnParams, &dynParams);
    if (hVd2 == NULL) {
        error("[DEC] Failed (h264dec)");
        cleanup(THREAD_FAILURE);
    }

    debug("[DEC] Number of codec buffers requested by Decoder [at creation]: %d - [level=%d, maxWidth=%d, maxHeight=%d]", Vdec2_getMinOutBufs(hVd2), extnParams.levelLimit, params.maxWidth, params.maxHeight);

The output from the application is:

[DEC] Number of codec buffers requested by Decoder [at creation]: 17 - [level=42, maxWidth=1920, maxHeight=1088]

I get the same value when calling Vdec2_getMinOutBufs(hVd2) after decoding the first frame.

Analysing the stream shows the profile value to be: 4d002a -- which translates to Main Profile (77), no constraints, Level 4.2.

Why is it asking for 17 buffers? How is value this calculated?

Thanks,

Martin

  • Hi,

    DM36x Decoder returns the buffer requirement for the given level limit and Video resolution. In your case level limit is 4_2 and  resolution is 1280x720. 

    DPB limit for 4_2 is 110400(in MBs, see H264 standard for more info on this)

    Video Size in MBs 1280x720/256 = 3600

    DPB frames = 110400/3600 = 30

    display frame Nos = min(17, DPB frames )

    ways to reduce DPB size:

    1: If you are decoding frames encoded by DM36x encoder, please set closed_loop_flag = 1.

    2: Allocate DPB size as SPS->MaxRefFrame + 1, instead of size returned by Vdec2_getMinOutBufs(hVd2). Disadvantage of this method is, you need to reinit buffer manager if new stream has more reference buffers than the allocated one.

     

    Thanks,

  • Hi Veeranna,

    Thank you very much for you quick response. However I do have to question...

    While looking online I'm unable to match the Max DPB MBs value you specified to Level 4.2. 110,400 appears to relate to Level 5.0. while Level 4.2 from what I can see is 34,816, with level 4.1 being shown as: 32,768.

    So from my original question :

    1) 720P Camera: Level 4.1 (32768):

        1280x720 => 1280*720/256 = 3600

        32768 / 3600 = 9.1

    This roughly aligns with the 9 buffers that the decoder asks for (and what the specification shows).

    2) But the problem comes with the 1080P Camera: Level 4.2 (34816):

        1920x1080 => 1920*1088/256 = 8160

         34816 / 8160 = 4.2

    So from this I would expect the codec ask for 4 buffers... But it actually asks for 17 - before and after processing the first frame - according to the TI H264 Decoder specification; it would default to the maximum (17) buffers if the level limit or dimensions were not set?

    The specification (ITU-T H.264 (03/2010)) also states that a 1080P HD video, level 4.2, requires 4 buffers.

    Have I misinterpreted something?

    Also just for reference, I can't set closed loop mode, since the frames are not encoded by the DM36x.

    Thanks,

    Martin

  • Sorry my mistake I misread the dpbSize from table. For 4_2 level dpbSize is 34816.

    TI Decoder asks from dpbSize + 1 as total required buffer. I have verified for 1280x720(10 buffers) and 1920x1088(5 buffers) cases, the  decoder is asking expected number of buffers. 

    Can you share the log and stream you used to run?

    Thanks,

    Veeranna

  • Hi Veeranna,

    While obtaining some more detailed debug data I've noticed that the decoder is not at all happy with the stream.

    Here is the output with DMAI_DEBUG=2:

    @0x0002c424:[T:0x406a1210] ti.sdo.dmai - [Dmai] Dmai log level set to '2'. Note that calling CERuntime_init after this point may cause unexpected change to DMAI tracing behavior.
    @0x0003bd08:[T:0x40701470] ti.sdo.dmai - [Ienc1] Buffer size 423936 obtained from XDM_GETBUFINFO control call
    @0x0003bfb6:[T:0x40701470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 635904 at 0x4190f000 (0x847ae000 phys)
    @0x0003c06e:[T:0x40701470] ti.sdo.dmai - [BufTab] Allocating BufTab for 2 buffers
    @0x0003c1ea:[T:0x40701470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 829440 at 0x419ab000 (0x8484a000 phys)
    @0x0003c39f:[T:0x40701470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 829440 at 0x41a76000 (0x84915000 phys)
    @0x00064be6:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Video decoder instance created
    @0x00064d8b:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Made XDM_SETPARAMS control call
    @0x00064e67:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Made XDM_GETBUFINFO control call
    @0x00064f36:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Made XDM_GETBUFINFO control call
    @0x0006549b:[T:0x406e1470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 4194304 at 0x41e71000 (0x84aca000 phys)
    @0x0006554f:[T:0x406e1470] ti.sdo.dmai - [BufTab] Allocating BufTab for 1 buffers
    @0x00065762:[T:0x406e1470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 3409920 at 0x42271000 (0x84eca000 phys)
    @0x00370450:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 38
    @0x0037056e:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x445
    @0x00370818:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 8
    @0x003708c6:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x445
    @0x00370b40:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 9
    @0x00370bf1:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x445
    @0x003b1e6c:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 7
    @0x003b1f86:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() failed with error (-1 ext: 0xa03e)

    Based on the initial error (0x445) I've started looking at the stream with the h264bitstream tool thinking that the first frame was not an IDR frame, but it actually is... So I stopped the application exiting at this point, which ended up producing the fatal error.

    Looking up (0x3e) I see that is unhappy with the display width parameter, so I printed some parameters out before calling vdec2_process:

    Parameters:
    * params.maxWidth: 1920
    * params.maxHeight: 1088
    * params.forceChromaFormat: 9
    * extnParams.levelLimit: 42
    * dynParams.displayWidth: 1920


    Which, I think, look correct? I've explicitly set the displayWidth parameter since the decoder fails to interpret lineLength correctly -- although I can't remember the source of that information -- I think I remember reading it on e2e.

    And here is a sample of the raw H.264, as captured by gstreamer:

    gst-launch-1.0 -v rtspsrc location=rtsp://user:pass@camera-ip/11 ! h264parse ! video/x-h264,alignment=nal,stream-format=byte-stream ! rtph264depay ! filesink location=raw.h264

    Stream available here: https://drive.google.com/file/d/0B7sxlNiO0kmoZHNkYi0tMjc2S2M/view?usp=sharing

    And here is the metadata from the stream (as decoded by h264bitstream): 0537.ipsmart.txt

    Thanks,


    Martin

  • Some more information:


    If do not set the display width parameter:

    @0x00021939:[T:0x40701470] ti.sdo.dmai - [Ienc1] Buffer size 423936 obtained from XDM_GETBUFINFO control call
    @0x00021c4d:[T:0x40701470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 635904 at 0x4190f000 (0x847ae000 phys)
    @0x00021d03:[T:0x40701470] ti.sdo.dmai - [BufTab] Allocating BufTab for 2 buffers
    @0x00021e7b:[T:0x40701470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 829440 at 0x419ab000 (0x8484a000 phys)
    @0x00022032:[T:0x40701470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 829440 at 0x41a76000 (0x84915000 phys)
    @0x0004dfa7:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Video decoder instance created
    @0x000508db:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Made XDM_SETPARAMS control call
    @0x00050a31:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Made XDM_GETBUFINFO control call
    @0x00050b0c:[T:0x406e1470] ti.sdo.dmai - [Vdec2] Made XDM_GETBUFINFO control call
    @0x0005106c:[T:0x406e1470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 4194304 at 0x41e71000 (0x84aca000 phys)
    @0x00051124:[T:0x406e1470] ti.sdo.dmai - [BufTab] Allocating BufTab for 1 buffers
    @0x00051324:[T:0x406e1470] ti.sdo.dmai - [Buffer] Alloc Buffer of size 3523584 at 0x42271000 (0x84eca000 phys)
    @0x003590f4:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 38
    @0x0035923d:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x445
    @0x003596fd:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 8
    @0x003597b1:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x445
    @0x00359bda:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 9
    @0x00359cb0:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x445
    @0x00389a16:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 193288
    @0x0038ea07:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x200
    @0x003988a2:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 21879
    @0x003989bc:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x200
    @0x00471866:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 16614
    @0x00471983:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x200
    @0x00487bbe:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 13809
    @0x00487cd7:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x200
    @0x00546545:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() ret -1 inId 0 inUse 0 consumed 10584
    @0x00546664:[T:0x406e1470] ti.sdo.dmai - [Vdec2] VIDDEC2_process() non-fatal error 0x200

    Parameters:
    * params.maxWidth: 1920
    * params.maxHeight: 1088
    * params.forceChromaFormat: 9
    * extnParams.levelLimit: 42
    * dynParams.displayWidth: 0

    I did find that I needed to set this [displayWidth] parameter for the 720P camera... I think because I create the buffers for maximum 1080P size, which is this way, as I don't know what resolution will be presented.

  • Note: the Vdec2_process call is returning 4 Dmai_EBITERROR

  • Hi,

    I think Decoder is getting corrupted bitstream, It suppose to consume one whole frame data 248259 for the given stream, but it is consuming only 38 bytes!. Can you please check the  bitstream just before calling decode process call, at least for 1st frame?

    I ran your stream at my end and getting proper output, below is log.

    Library Version: dm365_h264dec_02_00_00_13_production

    Details for XDM_GETVERSION:
    Status size = 196
    Status Extended error = 0

    Version: dm365_h264dec_02_00_00_13_production

    Details for XDM_SETPARAMS:

    Details for XDM_FLUSH:
    Status size = 196
    Status Extended error = 0

    Details for XDM_GETBUFINFO:
    Status size = 196
    Status Extended error = 0

    Min Input Buffers = 1
    Input Buf[0] size = 2097152
    Min Output Buffers = 2
    Max Display Buffers = 17

    Display Buf[0]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[1]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[2]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[3]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[4]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[5]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[6]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[7]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[8]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[9]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[10]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[11]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[12]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[13]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[14]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[15]:
    Buf0 size = 2349056 Buf1 size = 1174528
    Display Buf[16]:
    Buf0 size = 2349056 Buf1 size = 1174528


    Starting Decode on ARM968...


    first_mb_in_slice = 0
    OutArgs size = 1972
    Bytes consumed = 248259
    Out Buffer in use flag = 0
    Decoded Buffer numBufs = 2
    Decoded Buffer frameWidth = 1920
    Decoded Buffer frameHeight = 1080
    Decoded Buffer framePitch = 1984
    Decoded Buffer extendedError = 0
    Decoded Buffer frameType = 3
    Decoded Buffer topFieldFirstFlag = 0
    Decoded Buffer repeatFirstFieldFlag = 0
    Decoded Buffer frameStatus = 0
    Decoded Buffer repeatFrame = 0
    Decoded Buffer contentType = 0
    Decoded Buffer chromaFormat = 9
    Decoded Buffer Buf[0] Address = 0x40525418
    Decoded Buffer Buf[0] Size = 0x23D800
    Decoded Buffer Buf[0] Access Mask = 0x0
    Decoded Buffer Buf[1] Address = 0x4117FA18
    Decoded Buffer Buf[1] Size = 0x11EC00
    Decoded Buffer Buf[1] Access Mask = 0x0


    Details for XDM_GETSTATUS:
    Status size = 196
    Status Extended error = 0

    Output Height = 1080

    Output Width = 1920

    Max Display Buffers = 5

    Bit Rate = 0

    Frame Rate = 0

    Content Type = 0

    Chroma Format = 9

    FrameType = 3

    First MB in Slice = 0

    FrmNum in SliceHdr = 0
    Decoded Frm#:0 | BytesConsumed:248259 | LeftBytes:1848893

  • Hi Veeranna,


    I have checked the bitstream at the point I feed the decoder... and it is exactly as the shown in the file, just split up [wrongly].

    I can feed the file into the decoder and everything is OK, but when I feed the live stream from the camera - via GStreamer, it is not OK - same data, just chunked differently. I'm sure if I combine the first 3-4 buffers from the Gstreamer front end it would be fine too...

    I think we can eliminate the decoder from this problem - in as much, that it can decode the stream if fed properly. But I do wonder why the decoder does not like receiving the start of the stream like this?

    Thanks again for your help, it is appreciated :-)

    Regards,

    Martin

  • Hi,

    In default configuration decoder expects one complete frame data for each process call. You can enable CreateParams-->inputDataMode to NAL FORMAT, to feed partial frame data. Its called Low Latency Mode or Data Sync Mode.

    Please refer UserGuide and decode test application for information about low latency mode.

    Thanks,

    Veeranna