This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

h.264 encoder framerate problem

I'm having a problem with viewing h.264 from the DM8168 omx encoder.  I'm using code from the capture encode example.  I've tried viewing the video stream with vlc, but it doesn't work unless I force vlc to use the h264 demuxer.  vlc complains that it doesn't look like an h.264 stream but continues anyway.  Linux's mplayer complains that the stream does not contain a valid fps and I have to force the fps with -fps 30.

It seems that the h.264 encoder is not generating a valid fps in the header.  I've set the frame rate in my code in several places, but nothing seems to correct this problem.

Any ideas?

  • Hello,

    What is the software release that you are using here?

    Could you try to play with vlc and mplayer some other elementary video stream which is not encoded with TI codecs.

    Could you onfirm that the problem does not exist with different h264 elementary streams with the both player on the PC.

    Best Regards,

    Margarita

  • Hi Carl,


    When I was working with the TI encoders a few years ago, I looked deeply into the fps problem and found that the TI encoders were not generating it.  I could not view the videos at anything but 30fps and that was not good considering we were supposed to be at 8fps.

    You can use a tool called StreamEye to view the headers and see that the bit for FPS is not enabled.  I had to manually generate the header and overwrite the one that TI generated.  It involved actually reading the H264 documentation - not my favorite task in the world...

    Not sure if you will find its the same problem, but I suspect it might be. 

    Brandy J

    PS - StreamEye is a great tool if you have not already discovered it.  It made it very easy to see exactly what was being generated when.

  • Carl,


    Here is another post that might be helpful too.

    http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/99/p/195909/699434.aspx#699434

    (I can't figure out why this isn't hyperlinking - sorry.)

  • Brandy,

    Ah.  This seems likely, especially given other threads I've read. For example:

    http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/716/t/236619.aspx?

    The link to the thread you mentioned in your next message was very enlightening.  Thank you.

    I'll have to look and see if I can make your suggestion work.  This issue seems to have been around for a while.  Is TI ever going to address it?

  • Hi,

          Something important to mention is that the capture encode demo just gives the .h264 file so it is not in a container, if you use something like qtmux in gstreamer you can create the correct video and VLC will play it.

    https://developer.ridgerun.com/wiki/index.php/Gstreamer_pipelines_for_DM816x_and_DM814x#Video_Encoding_H264_1080.4060fps

    -David

  • Hello Carl,

    I am not sure if TI considers it an issue.  This quote here from Feroz "Frame rate is part of VUI not SPS/PPS.", leads me to believe it was intentional.

    According to the H264 specification the VUI portion of the headers is not mandatory but optional. 

    Perhaps Margarita can ping K Md Feroz Irfan to find out.

    Once I was able to see the bit mapping for the new parameters, it was not so hard to add in the FPS. 

    Brandy

  • David Soto said:
          Something important to mention is that the capture encode demo just gives the .h264 file so it is not in a container, if you use something like qtmux in gstreamer you can create the correct video and VLC will play it.

    Yes, I remember this is true too - which is why I mentioned StreamEye - it would play the files directly from the encoder.  This is how I did my testing before our KLV wrapper was ready.

    Brandy

  • TI h264 encoder fully supports insertion of timing info.Refer h264 encoder user guide 

    Check this thread where timingInfo insertion is discussed in dvrrdk

    http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/717/p/237004/830263.aspx#830263

  • Thank you for the link.

    I followed their suggestions, but it made no difference.

    I'm using OMX.

    I'm setting the dynamic and static params as follows:

    OMX_INIT_PARAM (&tDynParams);

    tDynParams.nPortIndex = OMX_VIDENC_OUTPUT_PORT;

    omx_err = OMX_GetParameter( enc.handle,
    OMX_TI_IndexParamVideoDynamicParams,
    &tDynParams );

    /* setting I frame interval */
    tDynParams.videoDynamicParams.h264EncDynamicParams.videnc2DynamicParams.intraFrameInterval = 90;
    tDynParams.videoDynamicParams.h264EncDynamicParams.videnc2DynamicParams.targetFrameRate = framerate * 1000;
    /* tDynParams.videoDynamicParams.h264EncDynamicParams.videnc2DynamicParams.forceFrame = IVIDEO_NA_FRAME; */
    tDynParams.videoDynamicParams.h264EncDynamicParams.videnc2DynamicParams.forceFrame = IVIDEO_IDR_FRAME;
    tDynParams.videoDynamicParams.h264EncDynamicParams.videnc2DynamicParams.generateHeader = XDM_GENERATE_HEADER;
    tDynParams.videoDynamicParams.h264EncDynamicParams.videnc2DynamicParams.targetBitRate = enc.bitrate;
    tDynParams.videoDynamicParams.h264EncDynamicParams.rateControlParams.rateControlParamsPreset = IH264_RATECONTROLPARAMS_DEFAULT;
    tDynParams.videoDynamicParams.h264EncDynamicParams.rateControlParams.HRDBufferSize = enc.bitrate * 2;

    omx_err = OMX_SetParameter( enc.handle,
    OMX_TI_IndexParamVideoDynamicParams,
    &tDynParams );

    OMX_INIT_PARAM (&tStaticParam);

    tStaticParam.nPortIndex = OMX_VIDENC_OUTPUT_PORT;

    omx_err = OMX_GetParameter (enc.handle, OMX_TI_IndexParamVideoStaticParams,
    &tStaticParam);

    tStaticParam.videoStaticParams.h264EncStaticParams.IDRFrameInterval = 1;
    tStaticParam.videoStaticParams.h264EncStaticParams.videnc2Params.rateControlPreset = IVIDEO_LOW_DELAY;

    /* We want SPS and PPS to be set for every intra frame. Hence
    * configure the the NALU control params to force encoder to
    * insert SPS/PPS on every I frame
    */

    tStaticParam.videoStaticParams.h264EncStaticParams.nalUnitControlParams
    .naluControlPreset = IH264_NALU_CONTROL_USERDEFINED;
    ENCLINK_H264_SETNALU_MASK_SPS(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskIntraPicture);
    ENCLINK_H264_SETNALU_MASK_PPS(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskIntraPicture);
    ENCLINK_H264_SETNALU_MASK_SPS(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskIDRPicture);
    ENCLINK_H264_SETNALU_MASK_PPS(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskIDRPicture);
    ENCLINK_H264_SETNALU_MASK_SEI(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskIDRPicture);
    ENCLINK_H264_SETNALU_MASK_SPS(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskStartOfSequence);
    ENCLINK_H264_SETNALU_MASK_PPS(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskStartOfSequence);
    ENCLINK_H264_SETNALU_MASK_PPS(tStaticParam.videoStaticParams.h264EncStaticParams
    .nalUnitControlParams.
    naluPresentMaskStartOfSequence);

    tStaticParam.videoStaticParams.h264EncStaticParams.vuiCodingParams
    .vuiCodingPreset = IH264_VUICODING_USERDEFINED;
    tStaticParam.videoStaticParams.h264EncStaticParams.vuiCodingParams
    .hrdParamsPresentFlag = 1;
    tStaticParam.videoStaticParams.h264EncStaticParams.vuiCodingParams
    .timingInfoPresentFlag = 1;

    omx_err = OMX_SetParameter (enc.handle, OMX_TI_IndexParamVideoStaticParams,
    &tStaticParam);

  • I see following code missing

    1. Set encoding preset as user defined as I mentioned.

    /* Encoder Preset settings */
      OMX_INIT_PARAM (&tEncoderPreset);
      tEncoderPreset.nPortIndex = OMX_VIDENC_OUTPUT_PORT;
      eError = OMX_GetParameter (pHandle, OMX_TI_IndexParamVideoEncoderPreset,
                                 &tEncoderPreset);
    
      tEncoderPreset.eEncodingModePreset = OMX_Video_Enc_User_Defined;
      tEncoderPreset.eRateControlPreset = OMX_Video_RC_Storage;
    
      eError = OMX_SetParameter (pHandle, OMX_TI_IndexParamVideoEncoderPreset,
                                 &tEncoderPreset);
      if (eError != OMX_ErrorNone)
      {
        ERROR ("failed to Encoder Preset \n");
      }


    2. Set NALU configuration and VUI configuration

      OMX_INIT_PARAM (&tNaluControlParam); 
        
      tNaluControlParam.nPortIndex = OMX_VIDENC_OUTPUT_PORT;
      tNaluControlParam.nStartofSequence  = 0x3FC4;
      tNaluControlParam.nEndofSequence    = 3072;
      tNaluControlParam.nIDR              = 0x3FC4;
      tNaluControlParam.nIntraPicture     = 0x42;
      tNaluControlParam.nNonIntraPicture  = 0x42;
    
      eError = OMX_SetParameter (pHandle, OMX_TI_IndexParamVideoNALUsettings,
                                 &tNaluControlParam);
      if (eError != OMX_ErrorNone)
      {
        ERROR ("failed to Encoder NALU control params  \n");
      }
    
      OMX_INIT_PARAM (&tVuiInfoParam);
      tVuiInfoParam.nPortIndex = OMX_VIDENC_OUTPUT_PORT;
      tVuiInfoParam.bHrdTimingPresent = 1;  
      
      eError = OMX_SetParameter (pHandle, OMX_TI_IndexParamVideoVUIsettings,
                                  &tVuiInfoParam);
    
      if (eError != OMX_ErrorNone)
      {
        ERROR ("failed to Encoder VUI control params  \n");
      }
    
    

  • There's a problem with your suggested code:  There is no bHrdTimingPresent member for the OMX_VIDEO_PARAM_VUIINFOTYPE structure (tVuiInfoParam).  I cannot find a reference to this member in any omx code, nor can I find a reference to this in the dvrrdk.  My version of OMX that I'm working with is 05_02_00_48.  Perhaps there has been a change in a later version of omx that I haven't seen yet?  If so getting this later version might solve this problem for me.  In any case I did set similar structure members in the static parameters:

    tStaticParam.videoStaticParams.h264EncStaticParams.vuiCodingParams
    .vuiCodingPreset = IH264_VUICODING_USERDEFINED;
    tStaticParam.videoStaticParams.h264EncStaticParams.vuiCodingParams
    .hrdParamsPresentFlag = 1;
    tStaticParam.videoStaticParams.h264EncStaticParams.vuiCodingParams
    .timingInfoPresentFlag = 1;

    Would these do the same thing?

    I tried as many of the changes you suggested as I could, but it still behaves the same way - no timing information such as FPS is present in the encoded video.