AM5728: H.264 encoder output data sync issue

Darshan Patel7

Part Number: AM5728
Other Parts Discussed in Thread: SYSBIOS

Hi,

We observed behaviour difference with putData data sync callback in H264 Encoder (IVAHD) on AM5728 when integrated with Processor SDK where the H264 Encoder (IVAHD) putData data sync call backs gives random number of output blocks/slices per callback, where as our standalone implementation on OMAP4 & J6, the putData callbacks always gives expected number of output blocks/slices for all frames.

For e.g, On OMPA4/J6, when we do H264 encode with 8 slices, we observed 8 putData call backs per frame, where each call back produces one block/slice. But when the same encoder library integrated with processor SDK on AM5728, we observed different number of putData call backs, and non-uniform too i.e. sometimes we see 2 call backs in one frame, some times 3 or 5 or any number less than or equal to 8, each call back produces more than one block/slices for some frames though the overall output H264 generated stream is having 8 slices.

Please give inputs to fix this issue.

Please let us know if you need more/ specific details.

over 8 years ago

0 Biser Gatchev-XID over 8 years ago

TI__Guru**** 393215 points

Hi,

Which SDK is this? Linux, RTOS, which version?

0 Darshan Patel7 over 8 years ago in reply to Biser Gatchev-XID

Intellectual 455 points

This is processor SDK 3.3 (built with ).

0 Biser Gatchev-XID over 8 years ago in reply to Darshan Patel7

TI__Guru**** 393215 points

Thanks. The software team have been notified. They will respond here.

0 Buddy L. over 8 years ago

TI__Intellectual 2540 points

Hi Darshan,
AFAIK, even though TI H.264 encoder library provides the capability for Sub Frame Level Synchronization (Slice mode), the PSDK doesn't support slice mode encoding. I am suspecting that you have your own implementation of slice mode encoding that utilizes the TI H.264 encoder library. If the H.264 codec encoder library version is the same when you testing on AM5728 and OMAP4/J6, then it has to be the configuration on your implementation software. Are you using the same configuration on your implementation between AM5728 and OMAP4/J6?

Looking into the TI H.264 encoder User Guide: git.ti.com/.../H264_Encoder_HDVICP2_UserGuide.pdf, Appendix G "Low Latency / Sub Frame Level Synchronization", Sec. G.3 H.264 Encoder Output with sub frame level synchronization, Table 24 specified the creation parameter. What is the value being set for VIDENC2_Params::numOutputDataUnits?

if outputDataMode == IVIDEO_SLICEMODE then it indicates the basic unit of slices at which encoder should produce the bit-stream.
Eg: Here 5 means that after encoding a set of 5 NALUs, encoder should inform to application.

Thanks.
//Buddy

0 Darshan Patel7 over 8 years ago in reply to Buddy L.

Intellectual 455 points

Hi Buddy,

Thanks for the support.

Yes. We have added Sub Frame Level Synchronization in PSDK. We have modified IPUMM FW, libdce, omapdrmtest.

H264 library version and all encoder static and dynamic configuration parameters are exactly same.

VIDENC2_Params::numOutputDataUnits = 1;

In standalone testapp, There are two callbacks for SPS and PPS. But in PSDK, it gives SPS and PPS in a single callback. In put data callback XDM_DataSyncDesc::numBlocks value varies from 1 to 4. We expect it to be exactly 1 every time. Which would give one callback for each block/ slice.

I have attached all codec parameter configuration.

Fullscreen H264_Encoder_paramerts_logs.txt Download

encodingPreset           3
rateControlPreset        2
maxHeight                1088
maxWidth                 1920
dataEndianness           1
maxBitRate               -1
minBitRate               0
inputChromaFormat        9
inputContentType         0
operatingMode            1
profile                  100
level                    41
inputDataMode            3
outputDataMode           1
numInputDataUnits        1
numOutputDataUnits       1
metadataType[0]          -1
metadataType[1]          -1
metadataType[2]          -1
interlaceCodingType      3
bottomFieldIntra         0
gopStructure            0
entropyCodingMode       1
transformBlockSize      2
log2MaxFNumMinus4       8
picOrderCountType       0
enableWatermark         0
IDRFrameInterval        0
pConstantMemory         0
maxIntraFrameInterval   30
debugTraceLevel          1
lastNFramesToLog         0
enableAnalyticinfo       0
enableGMVSei             0
constraintSetFlags       0
enableRCDO               0
enableLongTermRefFrame   0
LTRPPeriod               -1
numTemporalLayer                                                 1
referencePicMarking                                              1
reservedParams[0]                                                0
reservedParams[1]                                                31
reservedParams[2]                                                0
rateControlParams.rateControlParamsPreset                       1
rateControlParams.scalingMatrixPreset                            1
rateControlParams.rcAlgo                                         0
rateControlParams.qpI                                            -1
rateControlParams.qpMaxI                                         40
rateControlParams.qpMinI                                         10
rateControlParams.qpP                                            -1
rateControlParams.qpMaxP                                         51
rateControlParams.qpMinP                                         10
rateControlParams.qpOffsetB                                      4
rateControlParams.qpMaxB                                         51
rateControlParams.qpMinB                                         10
rateControlParams.allowFrameSkip                                 0
rateControlParams.removeExpensiveCoeff                           0
rateControlParams.chromaQPIndexOffset                            0
rateControlParams.IPQualityFactor                                0
rateControlParams.initialBufferLevel                             1536000
rateControlParams.HRDBufferSize                                  1536000
rateControlParams.minPicSizeRatioI                               0
rateControlParams.maxPicSizeRatioI                               640
rateControlParams.minPicSizeRatioP                               0
rateControlParams.maxPicSizeRatioP                               0
rateControlParams.minPicSizeRatioB                               0
rateControlParams.maxPicSizeRatioB                               0
rateControlParams.enablePRC                                      1
rateControlParams.enablePartialFrameSkip                         0
rateControlParams.discardSavedBits                               0
rateControlParams.reserved                                       0
rateControlParams.VBRDuration                                    8
rateControlParams.VBRsensitivity                                 0
rateControlParams.skipDistributionWindowLength                   5
rateControlParams.numSkipInDistributionWindow                    1
rateControlParams.enableHRDComplianceMode                        1
rateControlParams.frameSkipThMulQ5                               0
rateControlParams.vbvUseLevelThQ5                                0
rateControlParams.reservedRC[0]                                  0
rateControlParams.reservedRC[1]                                  0
rateControlParams.reservedRC[2]                                  0
interCodingParams.interCodingPreset                              1
interCodingParams.searchRangeHorP                                144
interCodingParams.searchRangeVerP                                32
interCodingParams.searchRangeHorB                                144
interCodingParams.searchRangeVerB                                16
interCodingParams.interCodingBias                                1
interCodingParams.skipMVCodingBias                               1
interCodingParams.minBlockSizeP                                  1
interCodingParams.minBlockSizeB                                  1
interCodingParams.meAlgoMode                                     0
intraCodingParams.intraCodingPreset                              0
intraCodingParams.lumaIntra4x4Enable                             255
intraCodingParams.lumaIntra8x8Enable                             255
intraCodingParams.lumaIntra16x16Enable                           15
intraCodingParams.chromaIntra8x8Enable                           15
intraCodingParams.chromaComponentEnable                          1
intraCodingParams.intraRefreshMethod                             0
intraCodingParams.intraRefreshRate                               0
intraCodingParams.gdrOverlapRowsBtwFrames                        0
intraCodingParams.constrainedIntraPredEnable                     0
intraCodingParams.intraCodingBias                                0
nalUnitControlParams.naluControlPreset                           0
nalUnitControlParams.naluPresentMaskStartOfSequence              416
nalUnitControlParams.naluPresentMaskIDRPicture                   416
nalUnitControlParams.naluPresentMaskIntraPicture                 2
nalUnitControlParams.naluPresentMaskNonIntraPicture              2
nalUnitControlParams.naluPresentMaskEndOfSequence                3072
sliceCodingParams.sliceCodingPreset                              1
sliceCodingParams.sliceMode                                      1
sliceCodingParams.sliceUnitSize                                  1080
sliceCodingParams.sliceStartOffset[0]                            0
sliceCodingParams.sliceStartOffset[1]                            0
sliceCodingParams.sliceStartOffset[2]                            0
sliceCodingParams.streamFormat                                   0
loopFilterParams.loopfilterPreset                                0
loopFilterParams.loopfilterDisableIDC                            0
loopFilterParams.filterOffsetA                                   0
loopFilterParams.filterOffsetB                                   0
fmoCodingParams.fmoCodingPreset                                  0
fmoCodingParams.numSliceGroups                                   1
fmoCodingParams.sliceGroupMapType                                4
fmoCodingParams.sliceGroupChangeDirectionFlag                    0
fmoCodingParams.sliceGroupChangeRate                             0
fmoCodingParams.sliceGroupChangeCycle                            0
fmoCodingParams.sliceGroupParams[0]                              0
fmoCodingParams.sliceGroupParams[1]                              0
vuiCodingParams.vuiCodingPreset                                  0
vuiCodingParams.aspectRatioInfoPresentFlag                       1
vuiCodingParams.aspectRatioIdc                                   1
vuiCodingParams.videoSignalTypePresentFlag                       0
vuiCodingParams.videoFormat                                      2
vuiCodingParams.videoFullRangeFlag                               0
vuiCodingParams.timingInfoPresentFlag                            1
vuiCodingParams.hrdParamsPresentFlag                             1
vuiCodingParams.numUnitsInTicks                                  1000
stereoInfoParams.stereoInfoPreset                                0
stereoInfoParams.topFieldIsLeftViewFlag                          1
stereoInfoParams.viewSelfContainedFlag                           0
framePackingSEIParams.framePackingPreset                         0
framePackingSEIParams.framePackingType                           3
framePackingSEIParams.frame0PositionX                            0
framePackingSEIParams.frame0PositionY                            0
framePackingSEIParams.frame1PositionX                            0
framePackingSEIParams.frame1PositionY                            0
framePackingSEIParams.reservedByte                               0
svcCodingParams.svcExtensionFlag                                 0
svcCodingParams.dependencyID                                     0
svcCodingParams.qualityID                                        0
svcCodingParams.enhancementProfileID                             0
svcCodingParams.layerIndex                                       0
svcCodingParams.refLayerDQId                                     0
inputHeight                  1080
inputWidth                   1920
refFrameRate                 30000
targetFrameRate              30000
targetBitRate                768000
intraFrameInterval           30
generateHeader               0
captureWidth                 1920
forceFrame                   -1
sampleAspectRatioHeight      1
sampleAspectRatioWidth       1
ignoreOutbufSizeFlag         1
sliceGroupChangeCycle        0
searchCenter.x               32767
searchCenter.y               0
enableStaticMBCount          0
enableROI                    0
reservedDynParams[0]         0
reservedDynParams[1]         0
reservedDynParams[2]         0
rateControlParams.rateControlParamsPreset                    1
rateControlParams.scalingMatrixPreset                        1
rateControlParams.rcAlgo                                     0
rateControlParams.qpI                                        -1
rateControlParams.qpMaxI                                     40
rateControlParams.qpMinI                                     10
rateControlParams.qpP                                        -1
rateControlParams.qpMaxP                                     51
rateControlParams.qpMinP                                     10
rateControlParams.qpOffsetB                                  4
rateControlParams.qpMaxB                                     51
rateControlParams.qpMinB                                     10
rateControlParams.allowFrameSkip                             0
rateControlParams.removeExpensiveCoeff                       0
rateControlParams.IPQualityFactor                            0
rateControlParams.chromaQPIndexOffset                        0
rateControlParams.initialBufferLevel                         1536000
rateControlParams.HRDBufferSize                              1536000
rateControlParams.enablePartialFrameSkip                     0
rateControlParams.minPicSizeRatioI                           0
rateControlParams.maxPicSizeRatioI                           640
rateControlParams.minPicSizeRatioP                           0
rateControlParams.maxPicSizeRatioP                           0
rateControlParams.minPicSizeRatioB                           0
rateControlParams.maxPicSizeRatioB                           0
rateControlParams.enablePRC                                  1
rateControlParams.enableHRDComplianceMode                    1
rateControlParams.reserved                                   0
rateControlParams.VBRDuration                                8
rateControlParams.VBRsensitivity                             0
rateControlParams.skipDistributionWindowLength               5
rateControlParams.numSkipInDistributionWindow                1
rateControlParams.enableHRDComplianceMode                    1
rateControlParams.frameSkipThMulQ5                           0
rateControlParams.vbvUseLevelThQ5                            0
rateControlParams.reservedRC[0]                              0
rateControlParams.reservedRC[1]                              0
rateControlParams.reservedRC[2]                              0
interCodingParams.interCodingPreset                          1
interCodingParams.searchRangeHorP                            144
interCodingParams.searchRangeVerP                            32
interCodingParams.searchRangeHorB                            144
interCodingParams.searchRangeVerB                            16
interCodingParams.interCodingBias                            1
interCodingParams.skipMVCodingBias                           1
interCodingParams.minBlockSizeP                              1
interCodingParams.minBlockSizeB                              1
interCodingParams.meAlgoMode                                 0
intraCodingParams.intraCodingPreset                          0
intraCodingParams.lumaIntra4x4Enable                         255
intraCodingParams.lumaIntra8x8Enable                         255
intraCodingParams.lumaIntra16x16Enable                       15
intraCodingParams.chromaIntra8x8Enable                       15
intraCodingParams.chromaComponentEnable                      1
intraCodingParams.intraRefreshMethod                         0
intraCodingParams.intraRefreshRate                           0
intraCodingParams.gdrOverlapRowsBtwFrames                    0
intraCodingParams.constrainedIntraPredEnable                 0
intraCodingParams.intraCodingBias                            0
sliceCodingParams.sliceCodingPreset                          1
sliceCodingParams.sliceMode                                  1
sliceCodingParams.sliceUnitSize                              1080
sliceCodingParams.sliceStartOffset[0]                        0
sliceCodingParams.sliceStartOffset[1]                        0
sliceCodingParams.sliceStartOffset[2]                        0
sliceCodingParams.streamFormat                               0

Thanks,

Darshan

0 Buddy L. over 8 years ago in reply to Darshan Patel7

TI__Intellectual 2540 points

Hi Darshan,

Since we don't have the Sub Frame Level Synchronization with SLICE_MODE in PSDK, it is a bit hard for us to debug the issue.
I presume that the standalone testapp is the test app provided by the H264 Encoder codec library. Can you provide us the test configuration when you run the slice mode since I don't see a default test cfg for slicemode under h264enc/app/test/testvecs/config. This is to see if there is something different between the standalone testapp configuration and the codec parameter configuration in PSDK.

If you look into the H.264 Enc user guide that I have pointed earlier and see section G.3.2 "H.264 Encoder behavior with outputDataMode as IVIDEO_SLICEMODE"; On the 3rd paragraph, there is a potential that numOfSlice will be bigger than numOutputDataUnit when codec makes a putData Fxn call back. On the 5th-9th paragraph, it shows that even though numOutputDataUnit is set to 64, there is a potential that numBlocks is 127.
Also on Table 28, the valid values for XDM_DataSyncDesc::numBlocks states "Any Value and it is the number of slices generated till the point of putData call. If outputDataUnit is 7, in the page cross over which would be the communication point and it generated 8 slices, then numBlocks is 8 and all 8 slices will be informed to App. Codec can generate following possible values of numBlocks 1 <= numBlocks <= 127"
The point is that numBlocks can be bigger than what numOutputDataUnit is set at create parameter. When encoder is encoding fast due to full input data availability (inputDataMode = IVIDEO_ENTIREFRAME (3)); it causes more numBlocks is available at communication point by codec.

I think the standalone testapp has a way to fill the input buffer with slice data based on the numOutputDataUnit which results in consistent numBlocks. Where in PSDK, it fills the input buffer with one full frame.

Looking into the H264_Encoder_parameter_logs.txt, can you try the setting on sliceCodingParams with the same configuration as the standalone testapp:
params->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_DEFAULT; // 0
params->sliceCodingParams.sliceMode = IH264_SLICEMODE_DEFAULT; // 0
params->sliceCodingParams.sliceUnitSize = 0;
params->sliceCodingParams.sliceStartOffset[0] = 0;
params->sliceCodingParams.sliceStartOffset[1] = 0;
params->sliceCodingParams.sliceStartOffset[2] = 0;
params->sliceCodingParams.streamFormat = IH264_STREAM_FORMAT_DEFAULT; // 0

Currently you have it as:
sliceCodingParams.sliceCodingPreset 1 <-- IH264_SLICECODING_USERDEFINED
sliceCodingParams.sliceMode 1 <-- IH264_SLICEMODE_MBUNIT
sliceCodingParams.sliceUnitSize 1080 <-- this parameter informs the number of macro blocks in one slice.
sliceCodingParams.sliceStartOffset[0] 0
sliceCodingParams.sliceStartOffset[1] 0
sliceCodingParams.sliceStartOffset[2] 0
sliceCodingParams.streamFormat 0

I am not sure on why 1080 for the number of Macroblocks. Per standard, H.264 level 4.1 should have 8192 Macroblocks per Frame. Reference: en.wikipedia.org/.../MPEG-4_AVC

Thanks.
//Buddy

0 Darshan Patel7 over 8 years ago in reply to Buddy L.

Intellectual 455 points

Hi Buddy,

Please find configuration attached.

I understand that numBlocks can be more than NumOutputUnits. But we want consistent callbacks for each block for low latency. Below is the log from application for number of putdata callbacks for 9 frame encoding (resolution 1920x1080, ).

Frame 1: Number of callbacks 9
Frame 2: Number of callbacks 8
Frame 3: Number of callbacks 2
Frame 4: Number of callbacks 2
Frame 5: Number of callbacks 1
Frame 6: Number of callbacks 3
Frame 7: Number of callbacks 2
Frame 8: Number of callbacks 3
Frame 9: Number of callbacks 3

There's another strange behavioural difference.

Standalone TestApp: SPS and PPS is given with two putdata callbacks (each callback numBlocks = 1 )

PSDK App: SPS and PPS is given in single putdata callback (numBlocks = 2)

I have done below experiments also. Behaviour doesn't change in any case.

- Reduce IVA frequency to 266 MHz. I have even tried to set IVA frequency to 46 MHz (though not in OPP)

- Set highest priority to process call thread.

- Set Task.defaultAffinity = 0, so that all threads run on Core 0 (kind of disable SMP).

I guess this negates "encoder running faster" and "Page cross over" cases. Does SYS BIOS running on SMP mode can affect anything? Because SYSBIOS is running in non-SMP mode on DM385 and DM8127 on which data sync works well.

Please provide your input on what factors can cause this behaviour.

Alexander_1080p_420sp_30fr_datasync.cfg

Thanks,

Darshan

0 Buddy L. over 8 years ago in reply to Darshan Patel7

TI__Intellectual 2540 points

Hi Darshan,

I presume that you are testing using the same input file: Alexander_1080p_420sp_30fr.yuv on Standalone TestApp and PSDK App.
Can you check to make sure that the same input buffer data and size being sent to codec for the first 3 frames?

When using PSDK App, can you please check the numBlocks when IPUMM receives the putData Fxn callback from codec? What you have provided is the log from application on the MPU side for number of putdata callbacks? i am wondering if there is a sync issue between MPU and M4. When you run Standalone TestApp, all context is running on M4 side. On PSDK App, it is running from MPU side.
I am just wondering if there is a possibility that the numBlocks that is received by IPUMM is not the same as the numBlocks being received by PSDK App.
Btw, where is the location of this PSDK App in the PSDK 3.3?
Thanks.
//Buddy

0 Gajanan Maske over 8 years ago in reply to Buddy L.

Expert 2325 points

Hi Buddy/Darshan,

The codec behavior is same on Standalone and PSDK App. Please refer the User Guide G3.2

"Communication point by codec to application about data availability is one of the below whichever is later"

- numof slices( numOutputDataUnit) is encoded i.e. if in the current page ,numOfSlice >= numOutputDataUnit then make a putData call.

- Minimum 1K of data is encoded i.e. numOfSlices exceeds numOutputDataUnit in the first page cross itself."

So H.264 Encoder will communicate through putData call only if satisfies the above conditions. Minimum 1K of data will be encoded and if there are multiple slices within this data, then Encoder will consolidate the numbers of blocks (slices) and update the putData call regarding the numbers of blocks(slices) available in the 1k of data.

e.g Lets say entire frame is black image (same data) & User requires 8 slices with output datasync enabled. Encoder will encode this frame and output generated bytes will be less than the 1k of data. So putdata function will be called only in the end of the frame with 8 blocks (slices).

This scenario occurs in low bitrate cases and more numbers of slices configured. To avoid this number of slices required should be reduced.

Regards

Gajanan

Processors

Processors forum

AM5728: H.264 encoder output data sync issue