H.264 on dm36x - Adaptive Longterm frame insertion

The H.264 platinum codecs feature support for "Adaptive Longterm Frame Insertion", see here:

http://processors.wiki.ti.com/index.php/H.264_DM36x_Ver_2.0_Codec#Adaptive_Longterm_Frame_Insertion

 

 I'm attempting to use this feature in a DM368-based system but I'm not sure I'm setting it up correctly. With the feature turned off and a simulated packet loss of 0.1% the decoder behaves as expected. I get a decode error on the frame with the dropped packet and then every frame after that reports extended error code 0x4f "The current picture refers to an already decoded error frame" ...until the next I-frame comes along and stops the error being propagated. All as expected.

 

With the feature enabled the reporting from the decoder looks correct. I still get the initial error from the frame with the dropped packet, then a couple of 0x4f warnings and then they stop. The error propagation appears to have been halted by instructing the encoder to "UseLongTermFrame=1". However the picture looks awful. It looks to me like setting UseLongTermFrame=1 on the encoder immediately corrupts the image. The result is much worse than the original error left to propagate. I'm assuming I'm not setting LongTermFrames correctly, or there's a mismatch between the references used by encode and decode.

 

My settings are;

EnableLongTermFrame = 1;

LongTermRefreshInterval = 30;

UseLongTermFrame is set to 1 whenever the decoder signals an error

SetLongTermFrame = 0; // Do I have to set this explicitly if I have set the refresh interval?

 

I've also tried setting LongTermRefreshInterval = 0 because the user guide says that I frames are automatically counted as long term frames. However this makes no difference.

 

Hope someone can shed some light on the right way to use this feature.

 

Alex 

  • In reply to Adithya Banninthaya:

    Hi Adithya,

    Thanks for checking the steam fro me. We did have frame_closedloop_flag=1, but I've just tried setting it to 0 and it makes no difference. Long term reference frames still produce corruption in the decoder output.

    Are you able to decode the stream I sent using a DM368 ?

    Thanks,

    Alex

  • In reply to Alex Nancekievill:

    Hi Alex,

    That is strange. frame_closedloop_flag=0 corresponds to universal decoder. I will check it and let you know

  • In reply to Adithya Banninthaya:

    Hi Alex,

    We decoded the stream setting frame_closedloop_flag to 0 and decoded video is fine. There is no corruption.

    Are you sure the flag is set to 0. Which version of decoder are you using. You can get the latest decoder from

    http://software-dl.ti.com/dsps/dsps_public_sw/codecs/DM36x/index_FDS.html

  • In reply to Adithya Banninthaya:

    Hi Adithya,

     

    Sounds like something strange is going on. I was on decoder version 2.00.00.10 so I upgraded to  2.00.00.13 but I'm afraid it didn't make any difference. I can't see anything in the release notes to suggest it should have done either.

     

    I believe I'm setting the closedloop flag correctly, but here is how I create the decoder so you can check: 

     

    memcpy(&pObj->h264Params, &IH264VDEC_PARAMS, sizeof(pObj->h264Params));

      pObj->params.size               = sizeof(VIDDEC2_Params);

      pObj->params.maxHeight          = create->maxHeight;

      pObj->params.maxWidth           = create->maxWidth;

      pObj->params.maxFrameRate       = 0;

      pObj->params.maxBitRate         = ALG_VID_DEC_MAX_BITRATE;

      pObj->params.dataEndianness     = XDM_BYTE;

      pObj->params.forceChromaFormat  = 9;


      pObj->h264Params.displayDelay = 0;

      pObj->h264Params.hdvicpHandle = NULL;

      pObj->h264Params.disableHDVICPeveryFrame = 0;

      pObj->h264Params.levelLimit= 0;

      pObj->h264Params.frame_closedloop_flag= 0; //0 - Universal; 1 - closed loop

      pObj->h264Params.inputDataMode= 1;  

      pObj->h264Params.sliceFormat= IH264VDEC_TI_BYTESTREAM;

      pObj->h264Params.viddecParams = pObj->params;

      pObj->h264Params.viddecParams.size = sizeof(pObj->h264Params);


      /* Create video decoder instance */

      pObj->hDecode = VIDDEC2_create(gALG_hEngine, pObj->algName, (VIDDEC2_Params*)&pObj->h264Params);

      if (pObj->hDecode == NULL) {

        OSA_ERROR("Failed to open video decode algorithm (%s)\n", pObj->algName);

        OSA_memFree(pObj);

        return NULL; 

      }

     

    Later I set the dynamic params like this:

     

        pObj->dynamicParams.size          = sizeof(VIDDEC2_DynamicParams);

        pObj->dynamicParams.decodeHeader  = XDM_DECODE_AU;

        pObj->dynamicParams.displayWidth  = displayWidth;

        pObj->dynamicParams.frameSkipMode = IVIDEO_NO_SKIP;

        pObj->dynamicParams.frameOrder    = IVIDDEC2_DECODE_ORDER;

        h264DynParams.viddecDynamicParams = pObj->dynamicParams;

        h264DynParams.viddecDynamicParams.size = sizeof(h264DynParams);

        h264DynParams.resetHDVICPeveryFrame = 1; // Set to 1 because we are running both the encoder and decoder in parallel

        h264DynParams.dataSyncHandle = NULL;

        h264DynParams.getDataFxn = NULL;    

            

        status = VIDDEC2_control(pObj->hDecode, XDM_SETPARAMS, (VIDDEC2_DynamicParams*)&h264DynParams, &pObj->decStatus);

        if (status != VIDDEC2_EOK) {

          OSA_ERROR("XDM_SETPARAMS failed, status=%ld\n", status);

          return OSA_EFAIL;

        }

     

     

  • In reply to Alex Nancekievill:

    Hi Adithya,

     

    A couple of other thoughts occurred to me.

    1) Is there any debug output from the decoder I could turn on that would confirm it's running with closedloop = 0 and/or shed more light on what's going wrong?

    2) From the way the decoded video looks, it's as though the decoder's copy of the long term frame is corrupt, so when it tries to decode a new P-frame relative to that buffer the image become distorted. I'm wondering if that could be caused by a memory problem. Is there anything I can do to make sure the decoder is allocating all the memory it needs and that memory isn't being overwritten?

     

    I don't have an easy way to screen capture a distorted image, but the best way I can describe it is that small patches of the screen that would have contained movement are garbled. I think the patches are macroblocks encoded in the new P-frame which must have been applied to a poor reference point. The rest of the image is fine. So for example if the video shows a person talking, the background behind the person is unaffected, but areas of movement on their face suddenly show the wrong colours and shapes. Since all the following frames are still P-frames, the error persist across all the frames that follow. 

     

    Hope that's useful,

     

    Alex

     

  • In reply to Alex Nancekievill:

    Hi Alex,

    It looks like decoder is not getting the correct reference. Are you allocating sufficient number of buffers?

    You can also check with decoder application in the package app/client/test/src/TestappDecoder_ih264_arm926.c. We have tested your stream with this application.

    You can also refer to the link for more about buffer mechanism

    http://www.ti.com/lit/an/sprab88/sprab88.pdf

  • In reply to Adithya Banninthaya:

    Hi Adithya,

    Thanks for your help here. I've marked it as verified because it was our mistake with buffer management that was causing the corruption. The link you provided made it clear what we needed to do to fix.

    However running in universal mode is bad news for us as it has a significant impact on performance. I thought the idea of closed loop was that it could decode anything the TI encoder produced That is clearly not the case if the decoder can't decode long term reference frames with closedloop=1. Please could we raise a bug report/feature request against this?

    Many thanks,

    Alex

  • In reply to Alex Nancekievill:

    Hi Alex,

    We will look into this but please dont expect any near term fix. Closed loop decoder was meant to decode streams of ver 2.0 encoder. We added more features in later version of encoder but the closed loop decoder did not get upgraded.

    regards

    Yashwant

  • In reply to Yashwant Dutt:

    Hi Yashwant,

    Thanks for the honest assessment. I see that the last release for the decoder was almost a year ago. Shouldn't a new release be due by now?

    Also, is there anything I can do to speed up the universal decoder since I'm not using lots of other features it has? Perhaps by specifying a lower levelIdc in the encoder?

    All the best,

    Alex