Linux/DRA744: DRA74xx media decoder playback with Android app.

Kelvin Tom

Part Number: DRA744

Tool/software: Linux

Hi,

I am seeing an error when performing playback with an Android app on my device. The error log is below:

E/OMX_VIDDEC_COMMON( 4725):
E/OMX_VIDDEC_COMMON( 4725): Process function returned an Error...
E/OMX_VIDDEC_COMMON( 4725): Codec Extended - 0x408001
E/OMX_VIDDEC_COMMON( 4725): Input Buffer Size provided to codec is : 25649
E/OMX_VIDDEC_COMMON( 4725): Frame count is : 1
E/OMX_VIDDEC_COMMON( 4725): Bytes consumed - 734
E/IMG_Gralloc( 4725): spsMaxRefFrames = 2, nRefFrames set initially = 1
E/IMG_Gralloc( 4725): Resetting level of the stream to Level 4.1
E/IMG_Gralloc( 4725): nBufferCountMin_old = -1837558940, nBufferCountMin_new = 1
E/OMX_VIDDEC_COMMON( 4725): Port disable/reconfiguration needed
D/UL-HLSPlayer( 5166): Fetch AssetInfo for 8db4b0b454334b508c2f36a0888732c7

The bits set for the error 0x408001 are as follows:
BIT 1: IH264VDEC_ERR_NOSLICE : Not even a single error-free slice header found in this process call, did not start MB loop
BIT 15: XDM_FATALERROR: Fatal error
BIT 22: IH264VDEC_ERR_NUMREF_FRAMES : The num_ref_frames in active SPS is more than the supported max_num_ref_frames.

Could you please explain what this error really means and if there's a way to fix it? I see it occur very frequently although it doesn't hinder the video from being decoded and displayed, however after a while of playing i get a black screen.

Any thoughts on this issue would be very appreciated.

Thanks!

over 8 years ago

0 Yordan Kamenov over 8 years ago

TI__Mastermind 42515 points

Hi Kelvin,

which Android version you use?

Regards,
Yordan

0 Kelvin Tom over 8 years ago in reply to Yordan Kamenov

Prodigy 100 points

Hi Yordan,

It's Lollipop.

0 Yordan Kamenov over 8 years ago in reply to Kelvin Tom

TI__Mastermind 42515 points

Hi Kelvin,

I have forwarded your question to Android experts.

Regards,
Yordan

0 Buddy L. over 8 years ago

TI__Intellectual 2540 points

Hi Kevin,

You can find more information by looking into the H.264 codec User Guide. See "Appendix C - Error Handling"

When the codec returns an error with Bit 15 "XDM_FATAL_ERROR", it means that the codec will not be able to continue decoding as severe issue has been detected while decoding. That is probably why you see the black screen as decoder has stopped and cannot continue.

And based on the other error bit being set, either the parser is not parsing the video file properly or the video itself is corrupted (Not based on H.264 standard specification).

As described in the user guide,

Bit 0 - IH264VDEC_ERR _NOSLICE. <-- This indicates corruption on the input data to be decoded.

Bit 22 - IH264VDEC_ERR _NUMREF_FRAMES <-- This is the error bit that causes codec to throw the FATAL ERROR.

As explained, the num_ref_frames specified in the H.264 SPS header is more than what it is specified in the SPS max_num_ref_frames (which should be in the VUI parameters - max_dec_frame_buffering).

Please check the video clip using H.264 video parser tool eg. Tektronix, Elecard, or VEGA to see the SPS and PPS of the clip.

From the Android perspective, the error can happen because at the beginning of the clip, TI OMX will set the num_ref_frames to 1 since TI OMX doesn't have the parser data of SPS and PPS. It will pass the 1st input data to the codec and codec will throw that error when the max_num_ref_frames required by codec to decode the clip is more than the configured number reference frames (where the output buffers are being configured earlier). This will trigger the PORT RECONFIG on TI OMX (see OMXH264VD_HandleError()) and stagefright will handle the port re-config, which will flush, reset and reconfigure the number of output buffers required to be able to decode the video.

Thanks.
//Buddy

0 Kelvin Tom over 8 years ago in reply to Buddy L.

Prodigy 100 points

Hi Buddy,

I have located where the SPS max_num_ref_frames is set. It is set in omx_h264dec.c around line 87:
((IH264VDEC_Status *)pVidDecComp->pDecStatus)->spsMaxRefFrames = 1;
I changed that to 2 and now I don't see the error reported.

However, after a while of playing, I get a new error:
E/OMX_VIDDEC_COMMON( 6533): Process function returned an Error...
E/OMX_VIDDEC_COMMON( 6533): Codec Extended - 0x100101
E/OMX_VIDDEC_COMMON( 6533): Input Buffer Size provided to codec is : 15422
E/OMX_VIDDEC_COMMON( 6533): Frame count is : 51664
E/OMX_VIDDEC_COMMON( 6533): Bytes consumed - 727
D/UL-MediaCodecDecoder( 7218): dequeueInputBuffer Returned Try Again
D/UL-AndroidHLSSource( 7218): Changing resolution from W/H 768 x 432 to 992 x 558
D/UL-MediaCodecDecoder( 7218): dequeueInputBuffer Returned Try Again
E/OMX_BASE_PROCESS( 6533): Failed check: OMX_ErrorNone == eError
E/OMX_BASE_PROCESS( 6533): Returning error: eError

BIT 0: IH264VDEC_ERR_NOSLICE : as you mentioned, it indicates corruption on the input data to be decoded.
BIT 8: XDM_PARAMSCHANGE: sequence parameter change
BIT 20: IH264VDEC_ERR_PICSIZECHANGE : resolution change detected.

When I see this error due to a resolution change, the decoder stopped on the last decoded frame and didn't recover. I saw this error from the drivers:
[73311.068291] omap_hwmod: mmu_ipu2: _wait_target_disable failed

and I kept getting these errors in the logcat:
D/UL-MediaCodecDecoder( 7218): dequeueOutputBuffer returned AMEDIACODEC_INFO_TRY_AGAIN_LATER
D/UL-MediaCodecDecoder( 7218): MediaCodeReadAndPost returned MEDIACODEC_TRY_AGAIN
D/UL-MediaCodecDecoder( 7218): dequeueInputBuffer Returned Try Again
D/UL-MediaCodecDecoder( 7218): dequeueOutputBuffer returned AMEDIACODEC_INFO_TRY_AGAIN_LATER
D/UL-MediaCodecDecoder( 7218): dequeueInputBuffer Returned Try Again
D/UL-MediaCodecDecoder( 7218): dequeueOutputBuffer returned AMEDIACODEC_INFO_TRY_AGAIN_LATER
D/UL-MediaCodecDecoder( 7218): dequeueInputBuffer Returned Try Again
D/UL-MediaCodecDecoder( 7218): dequeueOutputBuffer returned AMEDIACODEC_INFO_TRY_AGAIN_LATER
E/UL-MediaCodecDecoder( 7218): ERROR: Bad Input Buffer Index : -1
W/UL-MediaCodecDecoder( 7218): [MediaCodec] Read Count: 1 Returned: -2147483648
D/UL-MediaCodecDecoder( 7218): dequeueInputBuffer Returned Try Again

Any ideas on what will cause this issue? it looks like a kernel issue.

Thanks,
Kelvin

0 Buddy L. over 8 years ago in reply to Kelvin Tom

TI__Intellectual 2540 points

Hi Kelvin,

Changing the spsMaxRefFrames from 1 to 2 might work for the particular clip, but from the system level perspective, it might not be a good idea since you are now by default creating an extra output buffer. For a different clip that has reference frame > 2, you will get the error again which will trigger the Port reconfig. Port reconfig is anticipated by design because OMX don't have the SPS information. By passing the 1st input to codec, OMX gets the information from codec on the required reference frames which determine on how many output buffers required to decode the clip. With that said, I am just stating that Port reconfig is a normal flow when dealing with a clip that has >=2 reference frames.

For the error "Codec Extended - 0x100101", as the log and the error specify that a resolution change has been detected from 768x432 to 992x558. Again, in this situation, Port reconfig should take place where it should get all the buffers returned from codec through flush and tear down the codec instance and re-create codec instance. The output buffer will need to be reconfigured with the new resolution. I don't see more on the log after OMX returning with an error, but you should be seeing a similar log like "E/OMX_VIDDEC_COMMON( 4725): Port disable/reconfiguration needed". When you said Android App, is it the default Android Media Player App? Or a different Media app? If it is different, then can you try to play the clip on the default Android Media Player App? It should handle port reconfig as it should be.

Thanks.

//Buddy

0 Kelvin Tom over 8 years ago in reply to Buddy L.

Prodigy 100 points

Hi Buddy,

thanks for your response. I'll revert my change to spsMaxRefFrames. From your response, it seems to me that the system will recover safely from the errors I reported by performing a Port reconfig, correct?
I'm not seeing the log: "E/OMX_VIDDEC_COMMON( 4725): Port disable/reconfiguration needed" when these errors show up but I see the log a few times during playback when no errors occur. Could this be the problem? the system isn't reconfiguring the port?

The Android app being tested here is Disney Channel. I'm not sure how to play a clip from this app since it is streaming the content live from their servers.

Thanks,
Kelvin

0 Vishal Mahaveer over 8 years ago

TI__Mastermind 33420 points

Hi Kelvin,

Could you also mention which TI Android release are you using?

0 Kelvin Tom over 8 years ago in reply to Vishal Mahaveer

Prodigy 100 points

I'm not using a TI Android release specifically, just the android lollipop-mr1 release from omapzoom for the dra7xx project. I have ported that to the TI platform. which is running lollipop.

0 Vishal Mahaveer over 8 years ago in reply to Kelvin Tom

TI__Mastermind 33420 points

Hi Kelvin,

You took the latest d-lollipop-mr1-release branch of hardware/ti/dra7xx project.
Is my interpretation of your statement correct?

We have enabled Adaptive playback on latest lollipop-mr1 code base of TI J6.

You have to take/pick all the relevant patches from

[1] git.omapzoom.org/
[2] git.omapzoom.org/

Note that we have only tested this feature as part of Android CTS tests.

0 Kelvin Tom over 8 years ago in reply to Vishal Mahaveer

Prodigy 100 points

Yes your interpretation of my statement is correct.

Do you mean I should only pick the changes pertaining to Adaptive playback and ignore others?

I noticed in the omx_h264dec.c that for when resetting the stream to level 4.1, the flags for output port reconfiguration and codec recreation are not set but are set for other levels as seen in the code snippet below. Is there a reason why they're not set? I'm wondering if setting them might fix the issue i'm having since then I'm assuming the port would be reconfigured and the codec recreated.
...
} else if( pDecStatus->spsMaxRefFrames > nRefFrames41 ) {
pH264VidDecComp->tH264VideoParam.eLevel = OMX_VIDEO_AVCLevel5;
staticparams->presetLevelIdc = IH264VDEC_LEVEL5;
OSAL_ErrorTrace("Resetting level of the stream to Level 5");
pVidDecComp->nCodecRecreationRequired = 1;
pVidDecComp->nOutPortReconfigRequired = 1;
} else if( pDecStatus->spsMaxRefFrames > nRefFramesOld ) {
pH264VidDecComp->tH264VideoParam.eLevel = OMX_VIDEO_AVCLevel41;
OSAL_ErrorTrace("Resetting level of the stream to Level 4.1");
}

0 Sunita Nadampalli over 8 years ago in reply to Kelvin Tom

TI__Expert 5970 points

Hi Kelvin Tom,

By default codec is created with 4.1 level and currently the port reconfiguration is used mainly to handle the buffer padding requirements at the beginning of the playback, we always trigger one port reconfig cycle right after the first frame decode. Thats why if the parameters set are same as default we dont trigger port reconfig.

However, as you mentioned above, this logic seems to be missing the scenario where the parameter mismatch happened in the middle of the playback.

We will fix this by adding explicit port reconfig even for the default settings (because once decoding is started for a stream with different properties, codec is no longer in the default settings).

For now please go ahead with adding those two lines for last else if () case as well, we will review the entire path and provide a proper fix.

Thanks & Regards,

Sunita.

0 Kelvin Tom over 8 years ago in reply to Sunita Nadampalli

Prodigy 100 points

Thanks Sunita,

I think your comments and an update to the app resolved the issue.

- Kelvin

Processors

Processors forum

Linux/DRA744: DRA74xx media decoder playback with Android app.