H264 dynamic IDR frames.

R1234

Other Parts Discussed in Thread: OMAP3530

I'm trying to get my beagle-Xm board to send SPS/PPS headers and IDR frame for instance every 30 frames, but so far I'm not successful. Looking at other forum posts I leaned about a idrFrameInterval variable to use but I'm don't see this variable in the codecs-omap3530_4_00_00_00 package I have.

So I went for the other approach this was using the ForceFrame = IVIDEO_IDR_FRAME setting this is done as described in following forum post http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/100/t/7826.aspx

The generation of the SPS/PPS headers is OK but when I issue the VIDENC1_control() after seting forceFrame to IDR this control returns -1 error.

Does anyone know how to influence generation of IDR frames using the omap3530 codecs package?

Thanks,

Rudger.

over 14 years ago

0 R1234 over 14 years ago

Intellectual 420 points

Please note it's version 2.01.013 of the H264 encoder, this is packaged in the DVSDK v4.

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Hi Rudger,

Please tell which command you are using for control () API e.g. GetStatus, SetParams etc? And what are parameter settings done for encoder.

Also check with the size passed through dynamic parameter and status structure. If there is any mismatch, this type of IALG_EFAIL i.e. -1 return value will occur.

Note that forceFrame is a base parameter for dynamicParameter structure IVIDENC1_DynamicParams (User Guide section 4.2.1.9).

Regards,

Jaydeep

0 R1234 over 14 years ago in reply to Jaydeep Gokhale

Intellectual 420 points

Hi Jaydeep,

I'm doing the following (note ctx is my own context structure) setup procedure:

        ctx->params = Venc1_Params_DEFAULT;
        ctx->dynparams = Venc1_DynamicParams_DEFAULT;

        ctx->vid_params = &ctx->params;
        ctx->vid_dynparams = &ctx->dynparams;

For H264 encoding I use the default supplied by Venc1_Params_DEFAULT and Venc1_DynamicParams_DEFAULT

Then in our encoding function I use this:

if(ctx->frames % 30 == 0) {

   int32_t status = 0;
        VIDENC1_Status         encStatus;
        encStatus.data.buf = NULL;
        encStatus.size = sizeof(VIDENC1_Status);

        printf("Lets create IDR SPS/PPS headers");

        ctx->vid_dynparams->generateHeader = XDM_GENERATE_HEADER;
        ctx->vid_dynparams->forceFrame = IVIDEO_NA_FRAME;
        status = VIDENC1_control(Venc1_getVisaHandle(ctx->enc), XDM_SETPARAMS, ctx->vid_dynparams, &encStatus);
        if(status != VIDENC1_EOK) {
                printf("Failed to set encoder control SPS/PPS headers");
        }

process (in_buf)

        encStatus.data.buf = NULL;
        encStatus.size = sizeof(VIDENC1_Status);

        ctx->vid_dynparams->generateHeader = XDM_ENCODE_AU;
        ctx->vid_dynparams->forceFrame = IVIDEO_IDR_FRAME;
        status = VIDENC1_control(Venc1_getVisaHandle(ctx->enc), XDM_SETPARAMS,
                                       ctx->vid_dynparams, &encStatus);
        if(status != VIDENC1_EOK) {
                        printf("Failed to set encoder control force IDR frame");
        }

process( in_buf ....)

       encStatus.data.buf = NULL;
       encStatus.size = sizeof(VIDENC1_Status);
       ctx->vid_dynparams->forceFrame = IVIDEO_NA_FRAME;
       status = VIDENC1_control(Venc1_getVisaHandle(ctx->enc), XDM_SETPARAMS,
                              ctx->vid_dynparams, &encStatus);

....

}

So the every 30th frame I encode we set theXDM_GENERATE_HEADER and IVIDEO_NA_FRAME call control function which returns OK. The process after this will return 30 bytes PSP/PPS which seems OK to me. Then we set XDM_ENCODE_AU and IVIDEO_IDR_FRAME the control function will fail.

I'm not using the extended structures for H264, so the sizes are setup by Venc1_DynamicParams_DEFAULT.

Thanks,

Rudger.

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Rudger,

Try by setting ctx->vid_dynparams->forceFrame = 0 and please let me know.

Regards,

Jaydeep

0 R1234 over 14 years ago in reply to Jaydeep Gokhale

Intellectual 420 points

This works fine and generates an PSP/PPS headers 30 bytes and after that a normal I frame.

>>> [enc_h264] Push video data 4071 bytes
>>> [enc_h264] Lets create SPS/PPS
>>> [enc_h264] Push video data 30 bytes

>>> [enc_h264] Lets force frame
>>> [enc_h264] Push video data 22616 bytes
>>> [enc_h264] Push video data 4533 bytes

Regards,

Rudger.

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Rudger,

If your query is resolved, kindly mark the answer as "verify".

Regards,

Jaydeep

0 R1234 over 14 years ago in reply to Jaydeep Gokhale

Intellectual 420 points

Dear Jaydeep,

With all respect I believe an I-Frame is not the same as an IDR-frame.

Setting force frame to 0 generates an I-Fram, but we realy need IDR frames in out stream.

The deal with IDR versus I frames in h264 is that while both have all blocks coded as intra (i.e. you can decode them independently), IDR's are always seekable but I's generally aren't. While all IDR-frames are I-frames, not all I-frames are IDR-frames. An IDR-frame starts a new GOP and frames after it cannot reference frames before it. A normal I-frame on the other hand can have P- or B-frames after the I-frame referencing frames before the I-frame, so you can't just start decoding from the I-frame, you need to go back to the previous IDR.

Regards,

Rudger.

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Rudger,

I have suggested you the work-around to get IDR frames at periodic interval. Please check the generated stream for IDR frames.

By setting ctx->vid_dynparams->forceFrame = 0 will do that task of having IDR frame at the desired point.

To get the periodic I frames, you just need to set the IntraPeriod parameter according to I frame period and forceFrame as Default.

This issue will be fixed in next release which is not yet planned. But this issue has above mentioned work-around.

Regards,

Jaydeep

0 R1234 over 14 years ago in reply to Jaydeep Gokhale

Intellectual 420 points

Ok now I get it.

So there is a BUG in the encoder setting when forceframe is set to IDR_FRAME. When you set forceframe to 0 the encoder will generate an IDR frame.

I've verified the output of the encoder and it contains PSP/PPS on each IDR frame, so there is no need to generate PSP/PPS headers since the encoder will insert those automatically.

Thanks,

Rudger.

0 R1234 over 14 years ago in reply to R1234

Intellectual 420 points

Hi Jaydeep,

Now the next problem comes up.

We had to enable the INTRA method IH264_INTRAREFRESH_CYCLIC_SLICES to get some error recovery in our application.

This works fine untill the first IDR frame is generated. After that the INTRAREFRESH seem to stop working and the screen degrades slowly (packet loss or something).

I've tried to re-enable the INTRAREFRESH method using a separate control API call but it doesn't seem to help, the method just stops after the first time forceframe=0 was set.

Do you have any resolution for this problem?

Best regards,

Rudger.

Psuedo code...

init()

ctx->h264dynparams.intraRefreshMethod = IH264_INTRAREFRESH_CYCLIC_SLICES;

ctx->h264dynparams.sliceRefreshRowNumber = 1;

while(frames) {

if(time to send idr?) send_idr = 1;

if(send_idr) {

ctx->vid_dynparams->generateHeader = XDM_ENCODE_AU;

ctx->vid_dynparams->forceFrame = 0;

contro();

}

Process()

if(send_idr) {

ctx->vid_dynparams->forceFrame = IVIDEO_NA_FRAME;

control();

send_idr = 0;

}

....

}

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Rudger,

It is needed to set airMbPeriod in order to set any intra refresh method.

Please refer Appendix C from user guide for further information.

Regards,

Jaydeep

0 R1234 over 14 years ago in reply to Jaydeep Gokhale

Intellectual 420 points

Hi Jaydeep,

I've tried to set airMbPeriod also but that didn't help at al. I've read Appendix C again and again but paragraph C.3 doesn't mention airMbPeriod at all.

But nevertheless i set the airMbPeriod to 10 and re-run the tests.

I've taken some pictures of the phone displaying the encoded images. In the left top you can see the frame number this is the frame number put in the image before encoding. We trigger IDR every 500 frames starting with frame 500. You can clearly see in the pictures the screen slowly degrades at the bottom of the screen and this starts after the first forceFrame. Strange thing though is degration only takes place in the bottom half of the screen.

I've setup ctx->h264dynparams.intraRefreshMethod = IH264_INTRAREFRESH_CYCLIC_SLICES; ctx->h264dynparams.sliceRefreshRowNumber = 1; ctx->h264dynparams.airMbPeriod = 10;

When i disable setting the forceframe IDR no degration of the image occures.

Thanks,

Rudger.

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Rudger,

What is bit-rate, RC and other dyncamic parameters used ?

also can you share encoded bit-stream?

Regards,

Jaydeep

0 R1234 over 14 years ago in reply to Jaydeep Gokhale

Intellectual 420 points

No problem

The H264 settings are:

const IH264VENC_Params h264_def_params = {

{ sizeof(VIDENC1_Params)}, /* size */

66, //!< profile idc

IH264_LEVEL_30, //!< level idc

0, //!< Algorithm to be used by Rate Control Scheme Range[0,1]

64 //!< search range - integer pel search and 16x16 blocks. The search window is

//!< generally around the predicted vector. Max vector is 2xmcrange. For 8x8

//!< and 4x4 block sizes the search range is 1/2 of that for 16x16 blocks

};

and

const IH264VENC_DynamicParams h264_def_dynparams = {

{ sizeof(IVIDENC1_DynamicParams)},

24, //!< initial QP of I frames Range[-1,51]. -1 is for auto initialization.

24, //!< initial QP of P frames Range[-1,51]. -1 is for auto initialization.

51, //!< Maximum QP to be used Range[0,51]

0, //!< Minimum QP to be used Range[0,51]

0, //!< Controls enable/disable loop filter, See IH264VENC_LoopFilterParams for more details

1, //!< enable/disable Quarter Pel Interpolation

0, //!< Adaptive Intra Refesh MB Period: Period at which intra macro blocks should be insterted in a frame

0, //!< Maximum number of macro block in a slice <minimum value is 8>

0, //!< Maximum number of bytes in a slice

0, //!< Row number from which slice needs to be intra coded

0, //!< Number of rows to be coded as intra slice

0, //!< alpha offset for loop filter [-12, 12] even number

0, //!< beta offset for loop filter [-12, 12] even number

0, //!< Limits the maximum frame number in the bit-stream to (1<< (log2MaxFNumMinus4 + 4)) Range[0,12]

0, //!< Specifies offset to be added to luma QP for addressing QPC values table for chroma components.

//! Valid value is between -12 and 12, (inclusive)

0, //!< Controls the intra macroblock coding in P slices [0,1]

0, //!< Picture Order count type Valid values 0, 2

1, //!< enable/Disable Multiple Motion vector per MB, valid values are [1, 4]

// [For DM6446, allowed value is only 1]

INTRA4x4_NONE, //!< See IH264VENC_Intra4x4Params for more details

0, //!< enable/Disable Motion vector access

1, //!< Enable/Disable Hierarchical P Frame (non-reference P frame) Coding. [Not useful for DM6446]

IH264_BYTE_STREAM, //!< Signals the type of stream generated with Call-back

IH264_INTRAREFRESH_NONE,//!< Mechanism to do intra Refresh, see IH264VENC_IntraRefreshMethods for valid values

0, //!< Enable Perceptual Quantization a.k.a. Perceptual Rate Control

0, //!< Enable Scene Change Detection

// XDAS_Int32 enableFrameSkip ; //!< frameSkip control for VBR RC. (For CBR frame skip is always on) [0, 1].

NULL,

NULL, //!< pointer to context structure used during callback

//!< Following Parameter are related to Arbitrary Slice Ordering (ASO)

0, //!< Number of valid enteries in asoSliceOrder array valid range is [0,8],

//!< where 0 and 1 doesn't have any effect

{0, 0, 0, 0, 0, 0, 0, 0 },//!< Array containing the order of slices in which they should

//!< be present in bit-stream. vaild enteries are [0, any entry lesser than numSlicesASO]

//!< Following Parameter are related to Flexible macro block ordering (FMO)

0, //!< Total Number of slice groups, valid enteries are [0,8]

0, //!< Slice GroupMapType : For Valid enteries see IH264VENC_SliceGroupMapType

0, //!< Slice Group Change Direction Flag: Only valid when sliceGroupMapType

//!< is equal to IH264_RASTER_SCAN_SLICE_GRP.

//!< For valid values refer IH264VENC_SliceGroupChangeDirection

0, //!< Slice Group Change Rate: Only valid when sliceGroupMapType

//!< is equal to IH264_RASTER_SCAN_SLICE_GRP.

//!< valid values are : [0, factor of number of Mbs in a row]

0, //!< Slice Group Change Cycle: Only valid when sliceGroupMapType

//!< is equal to IH264_RASTER_SCAN_SLICE_GRP.

//!< Valid values can be 0 to numMbsRowsInPicture, also constrained

//!< by sliceGroupChangeRate*sliceGroupChangeCycle < totalMbsInFrame

{0, 0, 0, 0, 0, 0, 0, 0 } //!< This field is useful in case of sliceGroupMapType equal to either

//!< IH264_INTERLEAVED_SLICE_GRP or IH264_FOREGRND_WITH_LEFTOVER_SLICE_GRP

//!< In both cases it has different meaning:

//!< In case of IH264_INTERLEAVED_SLICE_GRP:

//!< The i-th entery in this array is used to specify the number of consecutive

//!< slice group macroblocks to be assigned to the i-th slice group in

//!< raster scan order of slice group macroblock units.

//!< Valid values are 0 to totalMbsInFrame again constrained by sum of all the elements

//!< shouldn't exceed totalMbsInFrame

//!< In case of IH264_FOREGRND_WITH_LEFTOVER_SLICE_GRP:

//!< First entry in the array specify the start position of foreground region in terms

//!< of macroblock number, valid values are [0, totalMbsInFrame-1]

//!< Second entry in the array specify the end position of foreground region in terms

//!< of macroblock number, valid values are [0, totalMbsInFrame-1] with following constrains:

//!< endPos > startPos && endPos%mbsInOneRow > startPos%mbsInOneRow

};

These are the defaults i've found in the documentation.

During initialisation I change these values:

ctx->h264dynparams.intraRefreshMethod = IH264_INTRAREFRESH_CYCLIC_SLICES;

ctx->h264dynparams.sliceRefreshRowNumber = 1;

ctx->h264dynparams.airMbPeriod = 10;

The target and max bitrate is set to 1024000, intraFrameInterval=0, rateControlPreset = IVIDEO_LOW_DELAY

You can find an encoded stream here: http://www.eliso.nl/div/video_degration.tar.gz

This contains a 320x240 raw H264 stream.

Thanks,

Rudger.

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Rudger,

Can you please check with setting encoder preset = IVIDEO_USER_DEFINED? and rcAlgo = 1 for CBR or rcAlgo = 0 for VBR along with desired bit rate

Regards,

Jaydeep

0 R1234 over 14 years ago in reply to Jaydeep Gokhale

Intellectual 420 points

Hi Jaydeep,

I think you mean setting this?

ctx->h264params.videncParams.encodingPreset = XDM_USER_DEFINED;

ctx->h264params.rcAlgo = 0; /* CBR */

This kind of works for the problem (so I don't see the screen degration problems in the bottom of the screen), but I see other problems in the image.

The bitrate gets higher and/or the framerate drops from 30fps to approx 15. Further the motion in the picture is very ugly.

Is there any planning to fix the initial IDR bug?

Cause when I disable this work-around to get IDR frames the picture is perfect. So when there is a possibility to get IDR frames on command or every X frames I would be very happy.

It seems to me the forceFrame setting each X frames is the root cause of this problem.

Thanks,

Rudger.

0 Jaydeep Gokhale over 14 years ago in reply to R1234

TI__Prodigy 330 points

Rudger,

At present, there is no plan for release of H264 encoder version on OMAP3530.

Regards,

Jaydeep

Processors

Processors forum

H264 dynamic IDR frames.