This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DM385: Unconstant Framerate @60Hz and Frame skips || SPS paket in p-frame

Part Number: DM385



Hello TI experts,

 

I have two topics on the DM385 behavior in on of our products. I would highly appriciate your insights on these.

You already worked with one of my colleages on the chip:

https://e2e.ti.com/support/processors/f/791/p/615132/2266279?tisearch=e2e-sitesearch&keymatch=sps%2525252525252520header%2525252525252520dm385#2266279

 

Frameskips @60Hz

Our customer usually encodes a video stream generated by camera 1080p@60Hz.. The encodes video seen here is a static linux background, so nothing too complicated to encode. When analysing the udp streams after encoding we see a strange behaviour:

1. There seem to be sporadic "gaps" between P-Frames. Can this be normal?

2. When operarting at 60Hz and looking at the debug output of the DM385, it records Frameskips. Only about 57FPS are leaving the chip. This can be seen in the UDP stream, too. I tried to change encoder profiles (base, main, high) with no improvement. When changing the encoderPreset to "high speed", the behavior improves, but this semms to cause other problems. Changing Framerate to 50FPS or lower, the problem disapears.

 [m3video]  *** ENCODE Statistics ***
 [m3video]
 [m3video]  Elasped Time           : 117 secs
 [m3video]
 [m3video]
 [m3video]  CH  | In Recv In Skip In User  Out Latency
 [m3video]  Num | FPS     FPS     Skip FPS FPS Min / Max
 [m3video]  --------------------------------------------
 [m3video]    0 |      59       1        0  58 255 / 67781770
 [m3video]
 [m3video] Multi Channel Encode Average Submit Batch Size
 [m3video] Max Submit Batch Size : 24
 [m3video] IVAHD_0 Average Batch Size : 1
 [m3video] IVAHD_0 Max achieved Batch Size : 1
 [m3video]
 [m3video] Multi Channel Encode Batch break Stats
 [m3video] Total Number of Batches created: 6795
 [m3video] All numbers are based off total number of Batches created
 [m3video]       Batch breaks due to batch sizeexceeding limit: 100 %
 [m3video]       Batch breaks due to ReqObj Que being empty: 0 %
 [m3video]       Batch breaks due to changed resolution class: 0 %
 [m3video]       Batch breaks due to interlace and progressivecontent mix: 0 %
 [m3video]       Batch breaks due to channel repeat: 0 %
 [m3video]       Batch breaks due to different codec: 0 %
[m3vpss ]  CAPTURE : Capture FPS: 60.0 fps ... in 10.0 secs
 [m3vpss ]  CAPTURE : Capture FPS: 59.9 fps ... in 10.0 secs

SPS, PPS Packets in front of I Frame

We send SPS and PPS infomation in fornt of every I Frame, so that the decoder can start the stream at any timing. Customer application uses this infomation to gernerate timestamps for further processing (as no timestamps are generated/tansported by DM385). Customer had some problems with this application, and when analysing the case, he noticed that sometimes (spordic, every 2-15min) a SPS header 00 00 01 27 is sent with a P-Frame which causes the app to crash. Of course this can be eliminated by the application.
I am wondering about the case of sending SPS header with a P-frame sporadicly. In the encLink_h264.c, I found the parts that generate the SPS and PPS Header in front of every I Frame. But what could cause this strange behavior?

/* We want SPS and PPS to be set for every intra frame. Hence configure the
     * the NALU control params to force encoder to insert SPS/PPS on every
     * I frame
     */
    staticParams->nalUnitControlParams.naluControlPreset =
                                             IH264_NALU_CONTROL_USERDEFINED;
    ENCLINK_H264_SETNALU_MASK_SPS(staticParams->nalUnitControlParams.
                                                naluPresentMaskIntraPicture);
    ENCLINK_H264_SETNALU_MASK_PPS(staticParams->nalUnitControlParams.
                                                naluPresentMaskIntraPicture);

    ENCLINK_H264_SETNALU_MASK_SPS(staticParams->nalUnitControlParams.
                                                naluPresentMaskIDRPicture);
    ENCLINK_H264_SETNALU_MASK_PPS(staticParams->nalUnitControlParams.
                                                naluPresentMaskIDRPicture);

Thanks in advance for your help!

Best regards,

Christopher Bräuer

  • Hello Christopher,

    At this time, we do not have dedicated support for the DM385 and support must be gotten through a third party as indicated on the device product folder.

    You can find FAQ information for this device in the following E2E post: 

    This post also contains contact information for various third parties who can support DM385.

  • Hi,

    Can you share the codec version and all the encoder parameters used? also the bit stream dump from the codec output stating the issue for further analysis

  • Hi,

    the encoder release used by our customer is H264 encoder v02.00.04.01.

    We already did a update to v02.00.09.00, but this causes other problems and the version has not been confirmed by the customer.

    If possible, we would like to stick with the old version.

    Variation of dynamic parameter like GOP size and target bit rate show now no change.

    --------- Static Params -------   
     [m3video] Set dynamicParams VBR                                                
     [m3video] videnc2Params -> size                 : 312                          
     [m3video] videnc2Params -> encodingPreset       : 3                            
     [m3video] videnc2Params -> rateControlPreset    : 5                            
     [m3video] videnc2Params -> maxHeight            : 1080                         
     [m3video] videnc2Params -> maxWidth             : 1920                         
     [m3video] videnc2Params -> dataEndianness       : 1                            
     [m3video] videnc2Params -> maxInterFrameInterval: 1                            
     [m3video] videnc2Params -> maxBitRate           : -1                    
     [m3video] videnc2Params -> minBitRate           : 0                            
     [m3video] videnc2Params -> inputChromaFormat    : 9                            
     [m3video] videnc2Params -> inputContentType     : 0                            
     [m3video] videnc2Params -> operatingMode        : 1                            
     [m3video] videnc2Params -> profile              : 77                           
     [m3video] videnc2Params -> level                : 42                           
     [m3video] videnc2Params -> inputDataMode        : 3                            
     [m3video] videnc2Params -> outputDataMode       : 3                            
     [m3video] videnc2Params -> numInputDataUnits    : 1                            
     [m3video] videnc2Params -> numOutputDataUnits   : 1    


    --------- Dynamic Params -------    
     [m3video]                                                                      
      [m3video] --------- CH 0 : H264 ENC : Dynamic Params -------
     [m3video]
     [m3video] videnc2DynamicParams -> inputHeight             : 1080
     [m3video] videnc2DynamicParams -> inputWidth              : 1920
     [m3video] videnc2DynamicParams -> refFrameRate            : 25000
     [m3video] videnc2DynamicParams -> targetFrameRate         : 60000
     [m3video] videnc2DynamicParams -> targetBitRate           : 8000000
     [m3video] videnc2DynamicParams -> intraFrameInterval      : 30
     [m3video] videnc2DynamicParams -> generateHeader          : 0
     [m3video] videnc2DynamicParams -> captureWidth            : 1920
     [m3video] videnc2DynamicParams -> forceFrame              : -1
     [m3video] videnc2DynamicParams -> interFrameInterval      : 1
     [m3video] videnc2DynamicParams -> mvAccuracy              : 2
     [m3video] videnc2DynamicParams -> sampleAspectRatioHeight : 1
     [m3video] videnc2DynamicParams -> sampleAspectRatioWidth  : 1
     [m3video] videnc2DynamicParams -> ignoreOutbufSizeFlag    : 1
     [m3video] videnc2DynamicParams -> lateAcquireArg          : -1
     [m3video]
     [m3video] rateControlParams -> rateControlParamsPreset        : 1
     [m3video] rateControlParams -> scalingMatrixPreset            : 0
     [m3video] rateControlParams -> rcAlgo                         : 0
     [m3video] rateControlParams -> qpI                            : -1
     [m3video] rateControlParams -> qpMaxI                         : 44
     [m3video] rateControlParams -> qpMinI                         : 8
     [m3video] rateControlParams -> qpP                            : -1
     [m3video] rateControlParams -> qpMaxP                         : 44
     [m3video] rateControlParams -> qpMinP                         : 8
     [m3video] rateControlParams -> qpOffsetB                      : 4
     [m3video] rateControlParams -> qpMaxB                         : 44
     [m3video] rateControlParams -> qpMinB                         : 10
     [m3video] rateControlParams -> allowFrameSkip                 : 1
     [m3video] rateControlParams -> removeExpensiveCoeff           : 0
     [m3video] rateControlParams -> chromaQPIndexOffset            : 0
     [m3video] rateControlParams -> IPQualityFactor                : 0
     [m3video] rateControlParams -> initialBufferLevel             : 16000000
     [m3video] rateControlParams -> HRDBufferSize                  : 16000000
     [m3video] rateControlParams -> minPicSizeRatioI               : 0
     [m3video] rateControlParams -> maxPicSizeRatioI               : 640
     [m3video] rateControlParams -> minPicSizeRatioP               : 0
     [m3video] rateControlParams -> maxPicSizeRatioP               : 0
     [m3video] rateControlParams -> minPicSizeRatioB               : 0
     [m3video] rateControlParams -> maxPicSizeRatioB               : 0
     [m3video] rateControlParams -> enablePRC                      : 1
     [m3video] rateControlParams -> enablePartialFrameSkip         : 1
     [m3video] rateControlParams -> discardSavedBits               : 1
     [m3video] rateControlParams -> VBRDuration                    : 8
     [m3video] rateControlParams -> VBRsensitivity                 : 0
     [m3video] rateControlParams -> skipDistributionWindowLength   : 5
     [m3video] rateControlParams -> numSkipInDistributionWindow    : 2
     [m3video] rateControlParams -> enableHRDComplianceMode        : 1
     [m3video] rateControlParams -> frameSkipThMulQ5               : 0
     [m3video] rateControlParams -> vbvUseLevelThQ5                : 0
     [m3video]
     [m3video] interCodingParams -> interCodingPreset  : 0
     [m3video] interCodingParams -> searchRangeHorP    : 144
     [m3video] interCodingParams -> searchRangeVerP    : 32
     [m3video] interCodingParams -> searchRangeHorB    : 144
     [m3video] interCodingParams -> searchRangeVerB    : 16
     [m3video] interCodingParams -> interCodingBias    : 1
     [m3video] interCodingParams -> skipMVCodingBias   : 1
     [m3video] interCodingParams -> minBlockSizeP      : 0
     [m3video] interCodingParams -> minBlockSizeB      : 0
     [m3video] interCodingParams -> meAlgoMode         : 0
     [m3video]
     [m3video] intraCodingParams -> intraCodingPreset          : 0
     [m3video] intraCodingParams -> lumaIntra4x4Enable         : 255
     [m3video] intraCodingParams -> lumaIntra8x8Enable         : 0
     [m3video] intraCodingParams -> lumaIntra16x16Enable       : 15
     [m3video] intraCodingParams -> chromaIntra8x8Enable       : 15
     [m3video] intraCodingParams -> chromaComponentEnable      : 1
     [m3video] intraCodingParams -> intraRefreshMethod         : 0
     [m3video] intraCodingParams -> intraRefreshRate           : 0
     [m3video] intraCodingParams -> gdrOverlapRowsBtwFrames    : 0
     [m3video] intraCodingParams -> constrainedIntraPredEnable : 0
     [m3video] intraCodingParams -> intraCodingBias            : 0
     [m3video]
     [m3video] sliceCodingParams -> sliceCodingPreset  : 1
     [m3video] sliceCodingParams -> sliceMode          : 416
     [m3video] sliceCodingParams -> sliceUnitSize      : 416
     [m3video] sliceCodingParams -> sliceStartOffset   : [-126 1 2]
     [m3video] sliceCodingParams -> streamFormat       : 0
     [m3video]
     [m3video] sliceGroupChangeCycle           : 3072
     [m3video] searchCenter                    : 0
     [m3video] enableStaticMBCount             : 0
     [m3video] enableROI                       : 262400
     [m3video]
     [m3video]

    In the attached file you can find the SPS header incident at packet 959. Here what should be a P-frame contains the SPS-header only.

    Also in this file you can see the unconstant packet output of the DM385 with gaps in between P-frames.

    Regards,

    Christopher

     

     

     

     

  • Hi,

    cbraeuer said:

    he encoder release used by our customer is H264 encoder v02.00.04.01.

    We already did a update to v02.00.09.00, but this causes other problems and the version has not been confirmed by the customer.

    If you are using v02.00.04.01, then it is recommended to use v02.00.09.01. Also we recommend to use the latest codec version, which has lot of bug fixes and feature enhancements done over the old version.

    you can find the bugs fixed in the release notes done here :

    cbraeuer said:
    [m3video] sliceCodingParams -> sliceCodingPreset  : 1
     [m3video] sliceCodingParams -> sliceMode          : 416
     [m3video] sliceCodingParams -> sliceUnitSize      : 416
     [m3video] sliceCodingParams -> sliceStartOffset   : [-126 1 2]
     [m3video] sliceCodingParams -> streamFormat       : 0

    Are you using slice mode? If no, can you disable it by setting sliceCodingPreset  : 0?

  • Hi,

    thank you for the provided files.

    I changed the sliceCodingPreset to 0, but with no improvement of our problems.

     [m3video] sliceCodingParams -> sliceCodingPreset  : -96
     [m3video] sliceCodingParams -> sliceMode          : 416
     [m3video] sliceCodingParams -> sliceUnitSize      : 131074
     [m3video] sliceCodingParams -> sliceStartOffset   : [0 12 0]
     [m3video] sliceCodingParams -> streamFormat       : 0

    Do you have any other possible solution?

    Is the unconstant output of frames normal?

    Thank you!

  • Hi,

    As it is very older version it is difficult to comment on the issue.

    Recommendation is to use the latest codec version to see if the issue is still seen there!

  • Hi,

    I am now using the latest codec version. Still the same behavior.

    I also tried to play with Bitrate, Profiles, GOP size. No change.

    One difference is, that now I can set encodingPreset=2 -> HIGH_SPEED.

    This seems to solve the problem (at least for low effort pictures). Unfortunatly, the decoder does not work with this setting.

    We are nomally sending SPS/PPS Header with every I-Frame, but when changing encodingPreset=2 they seem to appear only once at the start of the stream.

    When changing the input framerate to 50fps or lower, Frameskips and the arising SPS-Header in P-Frame seem to be gone. But this can not be a permanent solution, as customer expects 60fps.

    Regards,

    Christopher

  • Hi,

    Can you please refer Chapter 5 in User Guide? (Frequently Asked Questions), you will find information related to frame rate.

    Also other sections in user guide might answer most of your questions.

    IF you set IDRFrameInterval = 1 you will get SPS& PPS for all I frames.

  • Hi,

    IDRFrameInterval = 1 is already set. It normaly works as expected. Only if I set encodingPreset=2 this seems to be ignored by the codec.

    Regards,

    Christopher

  • Hi,

    do you have any other idea how I can get encodingPreset=2 and SPS/PPS in every I-Frame running?

    Have you ever seen the "gaps" I descibed in my initial post?

    Thanks!

    Regards,

    Christopher

  • Hi,

    Have you tried naluControlPreset as in Appendix B to get SPS & PP in every I frame with encodingPreset=HIGH_SPEED?

    cbraeuer said:
    Have you ever seen the "gaps" I descibed in my initial post?

    No the gaps were not observed at our side.

  • Hi,
    I have tried to implement EncodingPreset=HIGH_SPEED, but as in the manual described, this changes the NALUpreset from USER_DEFINED to DEFAULT, so that the SPS/PPS Headers are gone.

    When I try to set the parameter, that EncodingPreset=HIGH_SPEED defines (exept NALU), the recorded FrameSkips seem to become less, but the needed 60FPS are not measured.

    Is it a normal behavior of the encoder that 60FPS1080p input are converted to less (58/57 FPS) output?

    Regards,
    Christopher

  • Hi,

    I have tried to implement EncodingPreset=HIGH_SPEED, but as in the manual described, this changes the NALUpreset from USER_DEFINED to DEFAULT, so that the SPS/PPS Headers are gone.

    Yes, user can play with NAUL presets when EncodingPreset is USER_DEFINED only. With HIGH_SPEED, codec will optimize the parameters to give best results for HIGH_SPEED mode.

    When I try to set the parameter, that EncodingPreset=HIGH_SPEED defines (exept NALU), the recorded FrameSkips seem to become less, but the needed 60FPS are not measured.

    This is expected and true with the complexity of the video.

    Is it a normal behavior of the encoder that 60FPS1080p input are converted to less (58/57 FPS) output?

    Yes, it depends on the complexity of the video.