Issue with IDR Frame generation with H264 Encoder on DM365

I am not familiar with DM365 codecs but normally the codec (h.264 in this case) user guide and data-sheet included in the DVSDK are a good source of information for features supported by the codec and any restrictions on usage of APIs; if you have not done so, I would encourage giving these docs a look over.

Hi Juan,

Thanks for response.

I referred the user guide document for H264 Encoder. I am able to generate IDR frame at specifc interval of time. previously I was able to generate IDR frame but I was searching for SPS/PPS data with each IDR-frame.

H264 encoder generates the IDR-Frame but It is not able to generate SPS/PPS data and as per usr guide H264 encoder is sent SPS/PPS data with each IDR frame but I am not getting SPS/PPS data with each IDR frame. SPS/PPS data is generated at start of stream(Only once).

Do I need to configure specifically to generate IDR frame with SPS/PPS data?

Thanks,

Dilip

Dilip,

Unfortunately, I am not familiar enough with this topic; I have forwarded your questions to our codec team and should hear shortly, or they may respond to your question directly via this thread.

I'm interested in this too as I've just starting working on the DM365 and implementing a streaming H.264 device.  I noticed that similar infomation in the MPEG-4 encoder needed to be manually inserted by the application programmer.  But in this case the manual clearly states "With each IDR frame, SPS and PPS is sent".

Dilip,

Internal testing has shown that the SPS/PPS are included on the IDR frames, though I am looking into the details of that (I suspect the latest codec version and DVTB were used though).

In the mean time do you have a small encoded stream showing this issue you could send in that the codec folks could analyze?

Hi Bernie,

Thanks for response.

Please find attached test.264(TestFile.zip) for your reference.

Let me know in case of any issue.

-Dilip

Dilip, I looked at your test file and it appears that you are right.  I don't see the SPS/PPS header repeated even though IDR slices appear later in the file.

Also, I can play your file with VLC and get an image.  If I try to play a file that is created using the "encode" application I only get a black image and the playback slider does not smoothly move through the file as it does with your file.

John

Edit: my file does play fine with the "decode" app.

Just to confirm.... I'm trying the same thing as Dilip i.e. forcing the IDR frame periodically and there is no SPS/PPS generated.  At least not at the front of the buffer.  It's the end of the day.  I'll write the buffers out tomorrow and scan to see if the SPS/PPS is at the end of the buffer preceeding the IDR frame.

The development team reported that they were unable to reproduce the issue in 2.10.00.16 so it looks like this may have been fixed already. I am looking to see if I can get a public link to download this newer version of the software.

Further feedback from the codec folks, it seems that they are unable to reproduce the issue even in 2.10.00.14, however all their testing is done with DVTB, if you use DVTB on your end do you see the same failure to generate SPS/PPS?

I think we finally have this issue down, it looks like DVTB configures things slightly differently with the codec.

The codec does not insert SPS and PPS when forceFrame parameter is used to insert an IDR frame. This is because some customers might not need SPS and PPS inserted in this scenario. But to meet the requirement of customers who need SPS and PPS with IDR frame when forceFrame is used, please follow the below sequence. This has been tested to be working.

In the encoder thread, When IDR frame is to be inserted

1. Set the following:

dynamicParams.generateHeader = XDM_GENERATE_HEADER;

dynamicParams.forceFrame = IVIDEO_NA_FRAME;

2. Call VIDENC1_control() for XDM_SETPARAMS. This will set the parameter to generate header (SPS and PPS)

3. call VIDENC1_process(). This will be a process call just to create header

4. Set the following:

dynamicParams.generateHeader = XDM_ENCODE_AU;

dynamicParams.forceFrame = IVIDEO_IDR_FRAME;

5. Call VIDENC1_control() for XDM_SETPARAMS. This will set Force IDR Frame parameter

6. call VIDENC1_process(). This will generate an IDR frame

7. Set the following:

dynamicParams.generateHeader = XDM_ENCODE_AU;

dynamicParams.forceFrame = IVIDEO_NA_FRAME;

8. Call VIDENC1_control() for XDM_SETPARAMS. This will set the original parameters for encoding and remove force IDR frame.

Hi Bernie,

Thanks for the info.  I have a question about the process you suggest.

Do you call the VIDENC1_process() multiple times in addition to the the calls for actually processing new video frames, or  do you change the dynamic params before the normal VIDENC1_process() for each new video frame to process?

If it is in addition to the normal video frame processing, what input buffer should you use, just the last unprocessed buffer?

simon

msj said:

Do you call the VIDENC1_process() multiple times in addition to the the calls for actually processing new video frames, or  do you change the dynamic params before the normal VIDENC1_process() for each new video frame to process?

You end up with one extra VIDENC1_process call for the frame you want SPS and PPS inserted with the force frame generated IDR.

msj said:

If it is in addition to the normal video frame processing, what input buffer should you use, just the last unprocessed buffer?

I suspect you would end up passing the frame you plan to actually encode twice, i.e. the last unprocessed buffer. The XDM_GENERATE_HEADER flag tells the codec that you only want to encode header information, not an actual frame, so you are using this to force it to make a SPS/PPS header, which you follow by a more typical process call for the frame with XDM_ENCODE_AU which tells the codec that you want to encode an entire access unit.

If you passed two different sequential frames in this sequence to each VIDENC1_process call you would end up missing the frame that you tried to process with the XDM_GENERATE_HEADER flag.

EDIT: I see you have your post tagged with OMAP3, note that this is all in regards to the DM365 h.264 encode codec, and I am not sure how much applies to OMAP3.

Hi Bernie,

Thanks for the reply.

This is essentially what I am doing.  I haven't yet confirmed it is generating the correct IDR frames.

Yes I am using the OMAP3530, I hope it works the same as the DM365.(I didn't realize this was the DM365 list until after I had posted, just searched for H.264 IDR and found this post.)

Hi Bernie, thank you for your solution.

At the end, it seems to me it is easier to extract SPS/PPS from the first frame, keep it and paste it before each IDR when required. But I have a question on your method: the first VIDENC1_process call generates SPS/PPS, the second call generates the actual AU. I agree that the input frame should be the same, but what about the output buffer? Supposing that normally SPS/PPS should precede the AU, is there any way to reuse the same buffer and tell the encoder to put the AU after the SPS/PPS data? This could be useful to avoid to use 2 different buffer and then memcpy them together or make actual sending of the encoded frame more complicated.

Marco Braga said:

At the end, it seems to me it is easier to extract SPS/PPS from the first frame, keep it and paste it before each IDR when required.

This is what I do.

But I have a question on your method: the first VIDENC1_process call generates SPS/PPS, the second call generates the actual AU. I agree that the input frame should be the same, but what about the output buffer?

The SPS/PPS preceeds the compressed frame.  On my version of the SDK  (DVSDK_2_10_00_13) the SPS/PPS only comes out int he beginning.

John Anderson said:

But I have a question on your method: the first VIDENC1_process call generates SPS/PPS, the second call generates the actual AU. I agree that the input frame should be the same, but what about the output buffer?

The SPS/PPS preceeds the compressed frame.  On my version of the SDK  (DVSDK_2_10_00_13) the SPS/PPS only comes out int he beginning.

[/quote]

That's what happens with mt SDK too (2.10.00.17). I have still to decide which method is better. Luckly in my app I can send the SPS/PPS easily even if in a different buffer from AU but I was asking if there is a way to let the encoder produce a single buffer with SPS + PPS + Encoded frame all in one single step or if it is possible to call the second "process" with an offset in the buffer or if there is some specific requirement (alignment?) in the buffer start. To explain better:

1) buffer is allocated and its address is in pointer "ptr"

2) Enable SPS/PPS generation

3) Call "process(..., ptr, ...)", this puts SPS/PPS at address pointed ptr and returns size x

4) Disable SPS/PPS generation

5) Call "process(..., ptr + x, ...)" to produce the actual encoded image putting the data in the same buffer, after the SPS/PPS

If the encoder has no buffer address restrictions, this should be possible...

 

Just started playing in this area today.  Need to get periodic I-frames and periodic IDR frames at a slower rate. This is for H.264 encoder on DM365. Code based on encode demo using DMAI interface.

Tried setting intraFrameInterval = 30 and idrFrameInterval to 300 as recommended in the H.264 encoder user's guide.  No effect.  Single IDR frame at the beginning.  No IDR frames.  Not sure if I'm getting I-frames cuz DMAI in it's wisdom hides the outArgs and I ran out of time.  Will check on that in the morning. [Edit] I-frames are working.

Is there something else that needs to be set to get the IDR frames?  Or is there something that could be overriding  idrFrameInterval?

[Edit] FYI, I'm using dvsdk_2_10_01_18

Thanks!

Bernie Thompson said:

The codec does not insert SPS and PPS when forceFrame parameter is used to insert an IDR frame.

What about when idrFrameInterval is used?  Does it work as documented? i.e. periodic IDR frames with SPS and PPS?

Tom Hanson said:

What about when idrFrameInterval is used?  Does it work as documented? i.e. periodic IDR frames with SPS and PPS?

Yes, idrFrameInterval = n should give you SPS/PPS on an IDR frame every n frames. I have not tried this out personally but I have worked with people who have used it with success, though last time I believe the setting used was intraFrameInterval of 0 and idrFrameInterval of 30. I am not familiar enough with the inner workings of DMAI to say if it could be masking your idrFrameInterval setting.

Tom Hanson said:

What about when idrFrameInterval is used?  Does it work as documented? i.e. periodic IDR frames with SPS and PPS?

Bernie Thompson said:

Yes, idrFrameInterval = n should give you SPS/PPS on an IDR frame every n frames

This does not appear to be working.  I can control I-frame rate with intraFrameInterval, but no matter what I set idrFrameInterval to, I do not get get IDR frames unless I force them.

I did discover today that there is a behavioural difference between newer and older versions of the encoder.
  - In codec_engine_2_22 (approx dvsdk_1_30_01_41) the encoder returns every I-frame in an IDR Picture slice nal unit (nal_unit_type = 5) but the encodedFrameType in outargs is set to IVIDEO_I_FRAME, not IVIDEO_IDR_FRAME. (I'm guessing this was reported as a bug.)
  - In codec_engine_2_24 (dvsdk_2_10_01_18) the encoder returns I-frames in a non-IDR picture slice nal unit (nal_unit_type =1) when encodedFrameType  is set to IVIDEO_I_FRAME and uses an IDR Frame (5) when encodedFrameType  is set to IVIDEO_IDR_FRAME.

Is it possible that the people who were using idrFrameInterval with success were using an older version?

Can you have someone verify that idrFrameInterval is working as documented in dvsdk_2_10_01_18 please? (In a non-DVTB environment.)

Thanks!

dvsdk_2_10_01_18,  dm365_codecs_01_00_06,  set idrFrameInterval = intraFrameInterval, and then I got SPS/PPS followed by an IDR frame every "intraFrameInterval" frames.

Thanks!  Just got back to this after playing with audio for a while.  Knowing it worked for you motivated me to dig a little more.

Turned out to be operator error.  I'm doing both MPEG-4 and H.264.  I set up the common parameters and then copy them into the extended H.264 structure if I'm going to do H.264.  Well, this resets the "size" parameter back to the standard size instead of the extended size.  This keeps the encoder from seeing the extended parameters.  Added a line to reset the size correctly and all is working.  Lesson learned (and now shared!).

Hi :

could you tell me, how to set idrFrameInterval = intraFrameInterval, using dmai API ???

thanks a lot!!!

Sincerely

Antony

could you tell  me how to set idrFrameInterval=intarFrameInterval ? the  idrFrameInterval is in dm365_codec_01_00_06(Ih264venc.h),I can't find api to set it .

thanks a lot,!!!