TDA2P-ABZ: Encoder H264 Baseline Profile - TDA2P

Franz Schmid

Part Number: TDA2P-ABZ

Dear Processor team,

Our target OS is RTOS, We use now VSDK 3.3.

We want to use encoder link in our usecase, and then the encoded image will be sent out over rtp. In this case we also develop our rtp.

Now we need to set our H264 codec to send some slices from a frame, so we need to choose slice mode. As described in TI H264 user manual, to activate slice mode we should use baseline profile.

We have following settings:

pLinkChPrm->format = SYSTEM_IVIDEO_H264BP;

pLinkChPrm->profile = 66;

pLinkChPrm->dataLayout = VENC_FIELD_SEPARATED;

pLinkChPrm->fieldMergeEncodeEnable = FALSE;

pLinkChPrm->enableAnalyticinfo = 0;

pLinkChPrm->enableWaterMarking = 0;

pLinkChPrm->maxBitRate = -1;

pLinkChPrm->encodingPreset = SYSTEM_XDM_MED_SPEED_HIGH_QUALITY;

pLinkChPrm->rateControlPreset = SYSTEM_IVIDEO_LOW_DELAY;

pLinkChPrm->enableHighSpeed = FALSE;

pLinkChPrm->enableSVCExtensionFlag = FALSE;

pLinkChPrm->numTemporalLayer = 0;

pLinkChPrm->overrideInputScanFormat= 0;

pLinkChPrm->fieldPicEncode = 0;

pLinkChPrm->IDRFrameInterval = 1;

pLinkDynPrm->intraFrameInterval = 30;

pLinkDynPrm->targetBitRate = 10*1000*1000;

pLinkDynPrm->interFrameInterval = 1;

pLinkDynPrm->mvAccuracy = SYSTEM_IVIDENC2_MOTIONVECTOR_QUARTERPEL;

pLinkDynPrm->inputFrameRate = 30;

pLinkDynPrm->rcAlg = 0;

pLinkDynPrm->qpMin = 0;

pLinkDynPrm->qpMax = 51;

pLinkDynPrm->qpInit = 25;

pLinkDynPrm->vbrDuration = 8;

pLinkDynPrm->vbrSensitivity = 0;

pLinkDynPrm->maxPicSizeRatioI = 640;

In encLink_h264.c I make also some setting:

/*Slice Coding Parameters*/

staticParams->videnc2Params.level = IH264_LEVEL_42;

staticParams->intraCodingParams.lumaIntra4x4Enable=0xFF;

staticParams->intraCodingParams.lumaIntra8x8Enable=0x00;

staticParams->transformBlockSize = IH264_TRANSFORM_4x4;

staticParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_USERDEFINED;

staticParams->sliceCodingParams.sliceMode = IH264_SLICEMODE_BYTES;

staticParams->sliceCodingParams.streamFormat = IH264_BYTE_STREAM;

staticParams->sliceCodingParams.sliceUnitSize =1500;

staticParams->videnc2Params.outputDataMode = IVIDEO_SLICEMODE;

with those settings I get following error:

IPU1-0] 33.856707 s: ENCODE: Creating CH0 of 1280 x 720, pitch = (1280, 1280) [1] [0], bitrate = 10000 K

[IPU1-0] 33.921917 s: ENCODE: CH0: H264EncCreated:H264ENC_TI DEV.500.V.H264AVC.E.IVAHD.02.01.09.01

[IPU1-0] 33.922131 s: ENCODE: ERROR: ALGCONTROL FAILED CMD=0x00000001 (status=ffffffff) !!!

[IPU1-0] 33.922253 s: LINK ERR::linkID:52::channelID:0::errorCode:-5::FileName:iva_enc/encLink_h264.c::lin

[IPU1-0] 33.922405 s: ENCODE: ERROR: ALGCONTROL FAILED CMD=0x00000000 (status=ffffffff) !!!

[IPU1-0] 33.922527 s: ENCODE: ERROR: ALGCONTROL FAILED CMD=0x00000005 (status=ffffffff) !!!

[IPU1-0] 33.922619 s: LINK ERR::linkID:52::channelID:0::errorCode:-8::FileName:iva_enc/encLink_h264.c::lin

We have error by setting dynamic parameters (ENC_LINK_E_ALGSETPARAMSFAILED) and by getting buffer (ENC_LINK_E_ALGGETBUFINFOFAILED).

Can you check my setting? Or maybe you can suggest some settings for baseline profile?

over 6 years ago

0 Gaviraju Doddabettahalli Bettegowda over 6 years ago

TI__Genius 14305 points

I will check with codec team & get back

Thanks

Gaviraju

0 Ramprasad over 6 years ago in reply to Gaviraju Doddabettahalli Bettegowda

TI__Mastermind 30480 points

Hi Franz,

Set staticParams->videnc2Params.outputDataMode = IVIDEO_ENTIREFRAME;

and try.

outputDataMode=IVIDEO_SLICEMODE support is not available in visionsdk as this requires callbacks from codecs to share the decoded slices with low-latency.

Thanks

RamPrasad

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi RamPrasad,

We don’t need to implement Baseline Profile as entioned first. There was a misunderstanding from our side.
We can use the already implemented High Profile. But we need the Slice mode!

We are implementing a RTP network Video Stream so the NALUs need to be smaller than 64Kb in size.

Could you please provide us a sequence diagram and/or pseudo code for Api Calls and Callbacks, that enables us
to implement IVIDEO_SLICEMODE?

Regards,

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

slicemode encoding is supported for high-profile also but some configurations like B frames, CABAC etc can not be enabled when using slice mode.

Please refer H264 encoder User Guide Notes after Table 4-12.

outputDataMode = ENTIREFRAME means, the bitstream will have enabled slice mode but the output is available only after encoding complete frame.

This is the default and only option available in visionsdk encLink.

outputDataMode = SLICEMODE means, after every slice is encoded, encoder gives a callback to application to collect the encoded slice. Eventhough encoder suppports this , visionSDK's encLInk is not supporting this as the data exchanges need to happen for every slice.

Thanks

RamPrasad

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi RamPrasad,

Thank you a lot for your Answer!

The next point we are facing to:

We are using IH264_BYTE_STREAM so every Buffer comes with an start code identifier. 0 0 0 1

What we also see is the following NAL sequence:

SPS, NON IDR Slice, NON IDR Slice, NON IDR Slice, SPS…

The SPS has ~28KB (expected are 23Bytes) the NON IDR Slices from ~2Kb … ~20Kb

What we expect and want to have is something like

SPS, PPS, SEI, IDR, IDR, IDR…, NON IDR Slice, …

What are the settings to achieve the expected (last) sequence? And why are the sizes of the buffers too high?

Regards,

Sebastian

0 Sebastian Hauschild over 6 years ago in reply to Sebastian Hauschild

Expert 1070 points

It seems, that SPS, PPS and SEI are packet into one buffer which comes from the encoder. So that could be the reason for that!

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

Encoder inserts SPS, PPS and SEI in the first IDR frame , so the complete frame will have all these + payload slice.

payload slice will have size which will be within your configuration i,e 1500 bytes.

does it meet your requirement?

Thanks

RamPrasad

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi RamPrasad,

thank you! It almost meets our requirements. Is there any option or way to split SPS, PPS, SEI and IDR slice to different buffers in the Encoder or Processor SDK or
do we have to do it manually?

Regards and thank you again,

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian

There is an option in encoder to output only HEADER(SPS, PPS etc) using generateHeader in dynapicParameter but it is bit complicated to implement in visionsdk.

Pseudo code looks something like

dynParams.generateHeader = XDM_GENERATE_HEADER;

VIDENC2_control(enc, XDM_SETPARAMS, (VIDENC2_DynamicParams *) dynParams, (VIDENC2_Status *) status);

err = VIDENC2_process(enc, .inBufs, outBufs, inArgs, outArgs);

/* Now output buffer will have only HEADER. Dump it to file */

/* Reset generateHeader */

dynParams.generateHeader = XDM_ENCODE_AU;

VIDENC2_control(enc, XDM_SETPARAMS, (VIDENC2_DynamicParams *) dynParams, (VIDENC2_Status *) status);

Thanks

RamPrasad

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Thank you RamPrasad for your help!

At the moment we parsed the encoder buffer manually an sent the RTP stream out. But we have still the problem, that the IDR Slice is too big.
The encoder generates an single IDR slice with about 20Kb. Can you tell us where we can set this size exactly?

And we have another issue with our actual configuration that generates the following sequence:

SPS,PPS,SEI, IDR (single slice)
SPS, PPS,SEI, NON IDR (single slice)
SPS, PPS,SEI, NON IDR (single slice)
SPS, PPS,SEI, NON IDR (single slice)
SPS, PPS,SEI, NON IDR (single slice)
SPS, PPS,SEI, NON IDR (single slice)
SPS, PPS,SEI, NON IDR (single slice)
SPS,PPS,SEI, IDR (single slice)
SPS, PPS,SEI, NON IDR (single slice)
...

We would rather expect somethig like that:
SPS,PPS,SEI, IDR (some slices)
NON IDR (some slices)
NON IDR (some slices)
...
SPS,PPS,SEI, IDR (some slices)
NON IDR (some slices)

How can we set the expected sequence?

Regards,

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

If I understand correctly, the frame size of an IDR frame is > 20kb.

This will include SPS< PPS and SEI if any. and the payload will have multiple slices with each slice is within your sliceUnitSize (1500 in your case).

Is this the observation?

You can use Elecard Stream Analyzer to analyze the stream.

Thanks

RamPrasad

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi RamPrasad,

almost correct :)
We observe our stream with wireshark.
Our IDR frame is between ~20 and 45kB.

We have now managed that SPS, PPS and SEI is only generated with an IDR frame (as it should be in my view). So the sequence is now correct.
But our problem is still, that we get only one IDR slice (buffer) from encoder which is about these 20Kb-45kb in size.
What we want is to have several/multiple slices of that IDR frame that we could send in multiple network RTP packages. These multiple Slices should be max 1500bytes in size.

Our current test encoder configuration is:

pLinkChPrm->format                 = SYSTEM_IVIDEO_H264BP;
        pLinkChPrm->profile                = 66;
        pLinkChPrm->dataLayout             = VENC_FIELD_SEPARATED;
        pLinkChPrm->fieldMergeEncodeEnable = FALSE;
        pLinkChPrm->enableAnalyticinfo     = 0;
        pLinkChPrm->enableWaterMarking     = 0;
        pLinkChPrm->maxBitRate             = -1;
        //pLinkChPrm->encodingPreset         = SYSTEM_XDM_MED_SPEED_HIGH_QUALITY;
        /* Set to SYSTEM_XDM_USER_DEFINED if SEI/meta data needs to be enabled */
        pLinkChPrm->encodingPreset         = SYSTEM_XDM_USER_DEFINED;
        pLinkChPrm->rateControlPreset      = SYSTEM_IVIDEO_LOW_DELAY;
        pLinkChPrm->enableHighSpeed        = FALSE;
        pLinkChPrm->enableSVCExtensionFlag = FALSE;
        pLinkChPrm->numTemporalLayer       = 0;
        pLinkChPrm->overrideInputScanFormat= 0;
        pLinkChPrm->fieldPicEncode         = 0;
        pLinkChPrm->IDRFrameInterval       = 1;
 
        pLinkDynPrm->intraFrameInterval    = 30;
        pLinkDynPrm->targetBitRate         = 10*1000*1000;
        pLinkDynPrm->interFrameInterval    = 1;
        pLinkDynPrm->mvAccuracy            = SYSTEM_IVIDENC2_MOTIONVECTOR_QUARTERPEL;
        pLinkDynPrm->inputFrameRate        = 30;
        pLinkDynPrm->rcAlg                 = 0;
        pLinkDynPrm->qpMin                 = 0;
        pLinkDynPrm->qpMax                 = 51;
        pLinkDynPrm->qpInit                = 25;
        pLinkDynPrm->vbrDuration           = 8;
        pLinkDynPrm->vbrSensitivity        = 0;
        pLinkDynPrm->maxPicSizeRatioI      = 640;            
 
In encoder Link
 
/*Slice Coding Parameters*/
    staticParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_USERDEFINED;
    staticParams->sliceCodingParams.sliceMode = IH264_SLICEMODE_BYTES;
    staticParams->sliceCodingParams.sliceUnitSize = 8000U;
    staticParams->sliceCodingParams.streamFormat = IH264_NALU_STREAM;
 
staticParams->videnc2Params.level = IH264_LEVEL_42;
 
staticParams->videnc2Params.metadataType[0] = IH264_SEI_USER_DATA_UNREGISTERED;

Thank you in advance!

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

Yes, your understanding is correct. SPS, PPS and SEI need to be available only in IDR frame.

The 20kb size of the IDR frame may be because of the bitrate 10MBps.

For your current configuration, each slice in IDR frame will be within 8000 bytes(sliceUnitSize) , you need to set

sliceUnitSize to 1500 to get each slice to be within 1500 bytes.


Thanks
RamPrasad

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi RamPrasad,

the problem is that we get only one IDR slice with the size of e.g. 40kB.
Not many Slices in length of 8000 bytes.
Adjusting the parameter sliceUnitSize has either no effect in the amount of slices (it's always only one)
nor in the size of it.

Regards,

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

It looks like to me ,your static Parameter setting for slice encoding is over ridden by dynamic parameter setting which sets the parameters to default and hence you are not getting expected results.

Can you make slice coding settings in the dynamicParameters also (in enclink_h264_set_dynamic_params)

You can print the current setting by the encoder with the print from enclink_h264_print_dynamic_params() to confirm if slice coding is set or not.

Thanks

RamPrasad

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

This is the parameter dump:

[IPU1-0]     14.173549 s: --------- CH 0 : H264 ENC : Dynamic Params -------
[IPU1-0]     14.173641 s:
[IPU1-0]     14.173671 s: videnc2DynamicParams -> inputHeight             : 1080
[IPU1-0]     14.173763 s: videnc2DynamicParams -> inputWidth              : 1920
[IPU1-0]     14.173824 s: videnc2DynamicParams -> refFrameRate            : 30000
[IPU1-0]     14.173915 s: videnc2DynamicParams -> targetFrameRate         : 30000
[IPU1-0]     14.174007 s: videnc2DynamicParams -> targetBitRate           : 10000000
[IPU1-0]     14.174098 s: videnc2DynamicParams -> intraFrameInterval      : 30
[IPU1-0]     14.174159 s: videnc2DynamicParams -> generateHeader          : 0
[IPU1-0]     14.174251 s: videnc2DynamicParams -> captureWidth            : 1920
[IPU1-0]     14.174342 s: videnc2DynamicParams -> forceFrame              : -1
[IPU1-0]     14.174403 s: videnc2DynamicParams -> interFrameInterval      : 1
[IPU1-0]     14.174495 s: videnc2DynamicParams -> mvAccuracy              : 2
[IPU1-0]     14.174586 s: videnc2DynamicParams -> sampleAspectRatioHeight : 1
[IPU1-0]     14.174678 s: videnc2DynamicParams -> sampleAspectRatioWidth  : 1
[IPU1-0]     14.174739 s: videnc2DynamicParams -> ignoreOutbufSizeFlag    : 1
[IPU1-0]     14.174830 s: videnc2DynamicParams -> lateAcquireArg          : -1
[IPU1-0]     14.174922 s:
[IPU1-0]     14.174952 s: rateControlParams -> rateControlParamsPreset        : 0
[IPU1-0]     14.175013 s: rateControlParams -> scalingMatrixPreset            : 0
[IPU1-0]     14.175105 s: rateControlParams -> rcAlgo                         : 0
[IPU1-0]     14.175196 s: rateControlParams -> qpI                            : -1
[IPU1-0]     14.175288 s: rateControlParams -> qpMaxI                         : 36
[IPU1-0]     14.175349 s: rateControlParams -> qpMinI                         : 10
[IPU1-0]     14.175440 s: rateControlParams -> qpP                            : -1
[IPU1-0]     14.175532 s: rateControlParams -> qpMaxP                         : 40
[IPU1-0]     14.175623 s: rateControlParams -> qpMinP                         : 10
[IPU1-0]     14.175715 s: rateControlParams -> qpOffsetB                      : 4
[IPU1-0]     14.175776 s: rateControlParams -> qpMaxB                         : 44
[IPU1-0]     14.175867 s: rateControlParams -> qpMinB                         : 10
[IPU1-0]     14.175959 s: rateControlParams -> allowFrameSkip                 : 0
[IPU1-0]     14.176050 s: rateControlParams -> removeExpensiveCoeff           : 0
[IPU1-0]     14.176111 s: rateControlParams -> chromaQPIndexOffset            : 0
[IPU1-0]     14.176203 s: rateControlParams -> IPQualityFactor                : 0
[IPU1-0]     14.176294 s: rateControlParams -> initialBufferLevel             : 0
[IPU1-0]     14.176355 s: rateControlParams -> HRDBufferSize                  : 20000000
[IPU1-0]     14.176447 s: rateControlParams -> minPicSizeRatioI               : 0
[IPU1-0]     14.176538 s: rateControlParams -> maxPicSizeRatioI               : 640
[IPU1-0]     14.176630 s: rateControlParams -> minPicSizeRatioP               : 0
[IPU1-0]     14.176721 s: rateControlParams -> maxPicSizeRatioP               : 0
[IPU1-0]     14.176813 s: rateControlParams -> minPicSizeRatioB               : 0
[IPU1-0]     14.176874 s: rateControlParams -> maxPicSizeRatioB               : 0
[IPU1-0]     14.176965 s: rateControlParams -> enablePRC                      : 1
[IPU1-0]     14.177057 s: rateControlParams -> enablePartialFrameSkip         : 0
[IPU1-0]     14.177118 s: rateControlParams -> discardSavedBits               : 0
[IPU1-0]     14.177209 s: rateControlParams -> VBRDuration                    : 8
[IPU1-0]     14.177301 s: rateControlParams -> VBRsensitivity                 : 0
[IPU1-0]     14.177362 s: rateControlParams -> skipDistributionWindowLength   : 5
[IPU1-0]     14.177453 s: rateControlParams -> numSkipInDistributionWindow    : 1
[IPU1-0]     14.177545 s: rateControlParams -> enableHRDComplianceMode        : 1
[IPU1-0]     14.177636 s: rateControlParams -> frameSkipThMulQ5               : 0
[IPU1-0]     14.177728 s: rateControlParams -> vbvUseLevelThQ5                : 0
[IPU1-0]     14.177789 s:
[IPU1-0]     14.177850 s: interCodingParams -> interCodingPreset  : 0
[IPU1-0]     14.177911 s: interCodingParams -> searchRangeHorP    : 144
[IPU1-0]     14.178002 s: interCodingParams -> searchRangeVerP    : 32
[IPU1-0]     14.178063 s: interCodingParams -> searchRangeHorB    : 144
[IPU1-0]     14.178124 s: interCodingParams -> searchRangeVerB    : 16
[IPU1-0]     14.178216 s: interCodingParams -> interCodingBias    : 1
[IPU1-0]     14.178277 s: interCodingParams -> skipMVCodingBias   : 1
[IPU1-0]     14.178368 s: interCodingParams -> minBlockSizeP      : 1
[IPU1-0]     14.178429 s: interCodingParams -> minBlockSizeB      : 1
[IPU1-0]     14.178521 s: interCodingParams -> meAlgoMode         : 0
[IPU1-0]     14.178673 s:
[IPU1-0]     14.178704 s: intraCodingParams -> intraCodingPreset          : 0
[IPU1-0]     14.178795 s: intraCodingParams -> lumaIntra4x4Enable         : 255
[IPU1-0]     14.178856 s: intraCodingParams -> lumaIntra8x8Enable         : 0
[IPU1-0]     14.178948 s: intraCodingParams -> lumaIntra16x16Enable       : 15
[IPU1-0]     14.179009 s: intraCodingParams -> chromaIntra8x8Enable       : 15
[IPU1-0]     14.179100 s: intraCodingParams -> chromaComponentEnable      : 1
[IPU1-0]     14.179192 s: intraCodingParams -> intraRefreshMethod         : 0
[IPU1-0]     14.179253 s: intraCodingParams -> intraRefreshRate           : 0
[IPU1-0]     14.179344 s: intraCodingParams -> gdrOverlapRowsBtwFrames    : 0
[IPU1-0]     14.179405 s: intraCodingParams -> constrainedIntraPredEnable : 0
[IPU1-0]     14.179527 s: intraCodingParams -> intraCodingBias            : 0
[IPU1-0]     14.179619 s:
[IPU1-0]     14.179649 s: sliceCodingParams -> sliceCodingPreset  : -96
[IPU1-0]     14.179710 s: sliceCodingParams -> sliceMode          : 416
[IPU1-0]     14.179802 s: sliceCodingParams -> sliceUnitSize      : 131074
[IPU1-0]     14.179863 s: sliceCodingParams -> sliceStartOffset   : [0 12 -66]
[IPU1-0]     14.179954 s: sliceCodingParams -> streamFormat       : -66
[IPU1-0]     14.180046 s:
[IPU1-0]     14.180076 s: sliceGroupChangeCycle           : 0
[IPU1-0]     14.180137 s: searchCenter                    : 0
[IPU1-0]     14.180198 s: enableStaticMBCount             : 0
[IPU1-0]     14.180259 s: enableROI                       : 262400
[IPU1-0]     14.180351 s:
[IPU1-0]     14.180381 s:

We set the sliceUnitSize to 8000 dynamicly but the dump still remains 131074.

0 Sebastian Hauschild over 6 years ago in reply to Sebastian Hauschild

Expert 1070 points

Hi RamPrasad,

We observed, that targetbitrate, interframeinterval and intraframeinterval is adjustable of the dynamic parameters. The other dyn. parms are not adjustable. There is no effect if we set the other parms.

Regards and thank you!

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastina,
Yes, in enclink_codec_set_ch_alg_default_dynamic_params() targetbitrate, intraframeinterval etc are set.
Can you set slice paramters in enclink_h264_set_dynamic_params() and check?

Thanks
Ram

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

we already did that as you have suggested (in Apr 15, 2020 1:29 PM) with no effect :(

Thank you

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

I found out one issue in configuration in the application setting

pLinkChPrm->encodingPreset = SYSTEM_XDM_MED_SPEED_HIGH_QUALITY;

This should have been set to

pLinkChPrm->encodingPreset = SYSTEM_XDM_USER_DEFINED;

UserGuide says,

Any field from the IH264ENC_Params (excluding IVIDENC2_Params) structure is useful only when the encodingPreset field of IVIDENC2_Params data structure is equal to XDM_USER_DEFINED.

Any field from the IH264ENC_DynamicParams excluding IVIDENC2_DynamicParams)structure is useful only when the encodingPreset field of IVIDENC2_Params data structure is equal to XDM_USER_DEFINED.

This is the reason why slice coding settings from both static and dynamic params is not taking the effect.

I found out this from my encoder source code analyis and UserGuide .

Can you please make the settings recommended and try now.

I am also trying the same.

Thanks

Ram

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

we switched the parameter as you have suggested.

"pLinkChPrm->encodingPreset = SYSTEM_XDM_USER_DEFINED"

We set:

    dynamicParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_USERDEFINED;
    dynamicParams->sliceCodingParams.sliceMode = IH264_SLICEMODE_BYTES;
    dynamicParams->sliceCodingParams.sliceUnitSize = 8000U;
in enclink_h264_set_dynamic_params

and

/*Slice Coding Parameters*/
    staticParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_USERDEFINED;
    staticParams->sliceCodingParams.sliceMode = IH264_SLICEMODE_BYTES;
    staticParams->sliceCodingParams.sliceUnitSize = 8000U;
    staticParams->sliceCodingParams.streamFormat = IH264_NALU_STREAM;//IH264_STREAM_FORMAT_DEFAULT;
in enclink_h264_set_static_params.

The sliceCodingParams still remain as before (unset with random values)

[IPU1-0]     29.080034 s: --------- CH 0 : H264 ENC : Dynamic Params -------
[IPU1-0]     29.080095 s:
[IPU1-0]     29.080156 s: videnc2DynamicParams -> inputHeight             : 1080
[IPU1-0]     29.080217 s: videnc2DynamicParams -> inputWidth              : 1920
[IPU1-0]     29.080308 s: videnc2DynamicParams -> refFrameRate            : 30000
[IPU1-0]     29.080400 s: videnc2DynamicParams -> targetFrameRate         : 30000
[IPU1-0]     29.080461 s: videnc2DynamicParams -> targetBitRate           : 10000000
[IPU1-0]     29.080613 s: videnc2DynamicParams -> intraFrameInterval      : 1
[IPU1-0]     29.080705 s: videnc2DynamicParams -> generateHeader          : 0
[IPU1-0]     29.080796 s: videnc2DynamicParams -> captureWidth            : 1920
[IPU1-0]     29.080857 s: videnc2DynamicParams -> forceFrame              : -1
[IPU1-0]     29.080949 s: videnc2DynamicParams -> interFrameInterval      : 1
[IPU1-0]     29.081010 s: videnc2DynamicParams -> mvAccuracy              : 2
[IPU1-0]     29.081101 s: videnc2DynamicParams -> sampleAspectRatioHeight : 1
[IPU1-0]     29.081193 s: videnc2DynamicParams -> sampleAspectRatioWidth  : 1
[IPU1-0]     29.081254 s: videnc2DynamicParams -> ignoreOutbufSizeFlag    : 1
[IPU1-0]     29.081345 s: videnc2DynamicParams -> lateAcquireArg          : -1
[IPU1-0]     29.081406 s:
[IPU1-0]     29.081467 s: rateControlParams -> rateControlParamsPreset        : 1
[IPU1-0]     29.081559 s: rateControlParams -> scalingMatrixPreset            : 0
[IPU1-0]     29.081650 s: rateControlParams -> rcAlgo                         : 0
[IPU1-0]     29.081711 s: rateControlParams -> qpI                            : 25
[IPU1-0]     29.081803 s: rateControlParams -> qpMaxI                         : 51
[IPU1-0]     29.081894 s: rateControlParams -> qpMinI                         : 0
[IPU1-0]     29.081986 s: rateControlParams -> qpP                            : 25
[IPU1-0]     29.082047 s: rateControlParams -> qpMaxP                         : 51
[IPU1-0]     29.082138 s: rateControlParams -> qpMinP                         : 0
[IPU1-0]     29.082230 s: rateControlParams -> qpOffsetB                      : 4
[IPU1-0]     29.082291 s: rateControlParams -> qpMaxB                         : 44
[IPU1-0]     29.082382 s: rateControlParams -> qpMinB                         : 10
[IPU1-0]     29.082474 s: rateControlParams -> allowFrameSkip                 : 0
[IPU1-0]     29.082565 s: rateControlParams -> removeExpensiveCoeff           : 0
[IPU1-0]     29.082657 s: rateControlParams -> chromaQPIndexOffset            : 0
[IPU1-0]     29.082718 s: rateControlParams -> IPQualityFactor                : 0
[IPU1-0]     29.082809 s: rateControlParams -> initialBufferLevel             : 0
[IPU1-0]     29.082901 s: rateControlParams -> HRDBufferSize                  : 2500000
[IPU1-0]     29.082992 s: rateControlParams -> minPicSizeRatioI               : 0
[IPU1-0]     29.083053 s: rateControlParams -> maxPicSizeRatioI               : 0
[IPU1-0]     29.083145 s: rateControlParams -> minPicSizeRatioP               : 0
[IPU1-0]     29.083236 s: rateControlParams -> maxPicSizeRatioP               : 0
[IPU1-0]     29.083297 s: rateControlParams -> minPicSizeRatioB               : 0
[IPU1-0]     29.083389 s: rateControlParams -> maxPicSizeRatioB               : 0
[IPU1-0]     29.083480 s: rateControlParams -> enablePRC                      : 1
[IPU1-0]     29.083572 s: rateControlParams -> enablePartialFrameSkip         : 0
[IPU1-0]     29.083663 s: rateControlParams -> discardSavedBits               : 1
[IPU1-0]     29.083724 s: rateControlParams -> VBRDuration                    : 8
[IPU1-0]     29.083816 s: rateControlParams -> VBRsensitivity                 : 0
[IPU1-0]     29.083907 s: rateControlParams -> skipDistributionWindowLength   : 5
[IPU1-0]     29.083999 s: rateControlParams -> numSkipInDistributionWindow    : 2
[IPU1-0]     29.084060 s: rateControlParams -> enableHRDComplianceMode        : 1
[IPU1-0]     29.084151 s: rateControlParams -> frameSkipThMulQ5               : 0
[IPU1-0]     29.084243 s: rateControlParams -> vbvUseLevelThQ5                : 0
[IPU1-0]     29.084304 s:
[IPU1-0]     29.084334 s: interCodingParams -> interCodingPreset  : 0
[IPU1-0]     29.084426 s: interCodingParams -> searchRangeHorP    : 144
[IPU1-0]     29.084487 s: interCodingParams -> searchRangeVerP    : 32
[IPU1-0]     29.084578 s: interCodingParams -> searchRangeHorB    : 144
[IPU1-0]     29.084670 s: interCodingParams -> searchRangeVerB    : 16
[IPU1-0]     29.084731 s: interCodingParams -> interCodingBias    : 1
[IPU1-0]     29.084822 s: interCodingParams -> skipMVCodingBias   : 1
[IPU1-0]     29.084883 s: interCodingParams -> minBlockSizeP      : 0
[IPU1-0]     29.084944 s: interCodingParams -> minBlockSizeB      : 0
[IPU1-0]     29.085036 s: interCodingParams -> meAlgoMode         : 0
[IPU1-0]     29.085097 s:
[IPU1-0]     29.085127 s: intraCodingParams -> intraCodingPreset          : 0
[IPU1-0]     29.085219 s: intraCodingParams -> lumaIntra4x4Enable         : 255
[IPU1-0]     29.085310 s: intraCodingParams -> lumaIntra8x8Enable         : 0
[IPU1-0]     29.085371 s: intraCodingParams -> lumaIntra16x16Enable       : 15
[IPU1-0]     29.085463 s: intraCodingParams -> chromaIntra8x8Enable       : 15
[IPU1-0]     29.085554 s: intraCodingParams -> chromaComponentEnable      : 1
[IPU1-0]     29.085646 s: intraCodingParams -> intraRefreshMethod         : 0
[IPU1-0]     29.085737 s: intraCodingParams -> intraRefreshRate           : 0
[IPU1-0]     29.085798 s: intraCodingParams -> gdrOverlapRowsBtwFrames    : 0
[IPU1-0]     29.085890 s: intraCodingParams -> constrainedIntraPredEnable : 0
[IPU1-0]     29.085951 s: intraCodingParams -> intraCodingBias            : 0
[IPU1-0]     29.086042 s:
[IPU1-0]     29.086073 s: sliceCodingParams -> sliceCodingPreset  : -96
[IPU1-0]     29.086164 s: sliceCodingParams -> sliceMode          : 24992
[IPU1-0]     29.086225 s: sliceCodingParams -> sliceUnitSize      : 155906
[IPU1-0]     29.086317 s: sliceCodingParams -> sliceStartOffset   : [0 12 -66]
[IPU1-0]     29.086408 s: sliceCodingParams -> streamFormat       : -66
[IPU1-0]     29.086469 s:
[IPU1-0]     29.086500 s: sliceGroupChangeCycle           : 131073
[IPU1-0]     29.086591 s: searchCenter                    : 8000
[IPU1-0]     29.086683 s: enableStaticMBCount             : 0
[IPU1-0]     29.086744 s: enableROI                       : 262400

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

Thanks for checking . I see some difference now. qp Values are different from your earlier print.

I am now seeing qp values what is set from the application. But slice params is still has issues. I will check it.

Did you check the bitstream? Does it have multi slices with each slice within 8000 bytes?

Thanks

RamPrasad

0 Ramprasad over 6 years ago in reply to Ramprasad

TI__Mastermind 30480 points

Hi Sebastian,

There is some issue with enclink_h264_print_dynamic_params(). It was not passing correct pointer to print the paramters. Now I have corrected it and getting expected prints.

encodingPreset setting to USER_DEFINED is also a must to set slice encoding.

I have attached the changes I have added in encLink_h264.c and the print I am getting here.

0001-encLink-Slice-Mode-support.zip

[HOST] [IPU2 ] 49.563106 s:
[HOST] [IPU2 ] 49.563167 s: --------- CH 0 : H264 ENC : Dynamic Params -------
[HOST] [IPU2 ] 49.563258 s:
[HOST] [IPU2 ] 49.563289 s: videnc2DynamicParams -> inputHeight : 720
[HOST] [IPU2 ] 49.563380 s: videnc2DynamicParams -> inputWidth : 1280
[HOST] [IPU2 ] 49.563472 s: videnc2DynamicParams -> refFrameRate : 30000
[HOST] [IPU2 ] 49.563533 s: videnc2DynamicParams -> targetFrameRate : 30000
[HOST] [IPU2 ] 49.563624 s: videnc2DynamicParams -> targetBitRate : 10000000
[HOST] [IPU2 ] 49.563716 s: videnc2DynamicParams -> intraFrameInterval : 30
[HOST] [IPU2 ] 49.563807 s: videnc2DynamicParams -> generateHeader : 0
[HOST] [IPU2 ] 49.563899 s: videnc2DynamicParams -> captureWidth : 1280
[HOST] [IPU2 ] 49.563960 s: videnc2DynamicParams -> forceFrame : -1
[HOST] [IPU2 ] 49.564112 s: videnc2DynamicParams -> interFrameInterval : 1
[HOST] [IPU2 ] 49.564204 s: videnc2DynamicParams -> mvAccuracy : 2
[HOST] [IPU2 ] 49.564295 s: videnc2DynamicParams -> sampleAspectRatioHeight : 1
[HOST] [IPU2 ] 49.564356 s: videnc2DynamicParams -> sampleAspectRatioWidth : 1
[HOST] [IPU2 ] 49.564448 s: videnc2DynamicParams -> ignoreOutbufSizeFlag : 1
[HOST] [IPU2 ] 49.564539 s: videnc2DynamicParams -> lateAcquireArg : -1
[HOST] [IPU2 ] 49.564631 s:
[HOST] [IPU2 ] 49.564692 s: rateControlParams -> rateControlParamsPreset : 1
[HOST] [IPU2 ] 49.564753 s: rateControlParams -> scalingMatrixPreset : 0
[HOST] [IPU2 ] 49.564844 s: rateControlParams -> rcAlgo : 0
[HOST] [IPU2 ] 49.564936 s: rateControlParams -> qpI : 25
[HOST] [IPU2 ] 49.565058 s: rateControlParams -> qpMaxI : 51
[HOST] [IPU2 ] 49.565149 s: rateControlParams -> qpMinI : 0
[HOST] [IPU2 ] 49.565241 s: rateControlParams -> qpP : 25
[HOST] [IPU2 ] 49.565302 s: rateControlParams -> qpMaxP : 51
[HOST] [IPU2 ] 49.565393 s: rateControlParams -> qpMinP : 0
[HOST] [IPU2 ] 49.565485 s: rateControlParams -> qpOffsetB : 4
[HOST] [IPU2 ] 49.565576 s: rateControlParams -> qpMaxB : 44
[HOST] [IPU2 ] 49.565668 s: rateControlParams -> qpMinB : 10
[HOST] [IPU2 ] 49.565729 s: rateControlParams -> allowFrameSkip : 0
[HOST] [IPU2 ] 49.565820 s: rateControlParams -> removeExpensiveCoeff : 0
[HOST] [IPU2 ] 49.565912 s: rateControlParams -> chromaQPIndexOffset : 0
[HOST] [IPU2 ] 49.566003 s: rateControlParams -> IPQualityFactor : 0
[HOST] [IPU2 ] 49.566125 s: rateControlParams -> initialBufferLevel : 0
[HOST] [IPU2 ] 49.566186 s: rateControlParams -> HRDBufferSize : 2500000
[HOST] [IPU2 ] 49.566278 s: rateControlParams -> minPicSizeRatioI : 0
[HOST] [IPU2 ] 49.566369 s: rateControlParams -> maxPicSizeRatioI : 0
[HOST] [IPU2 ] 49.566461 s: rateControlParams -> minPicSizeRatioP : 0
[HOST] [IPU2 ] 49.566552 s: rateControlParams -> maxPicSizeRatioP : 0
[HOST] [IPU2 ] 49.566613 s: rateControlParams -> minPicSizeRatioB : 0
[HOST] [IPU2 ] 49.566705 s: rateControlParams -> maxPicSizeRatioB : 0
[HOST] [IPU2 ] 49.566796 s: rateControlParams -> enablePRC : 1
[HOST] [IPU2 ] 49.566888 s: rateControlParams -> enablePartialFrameSkip : 0
[HOST] [IPU2 ] 49.566979 s: rateControlParams -> discardSavedBits : 1
[HOST] [IPU2 ] 49.567071 s: rateControlParams -> VBRDuration : 8
[HOST] [IPU2 ] 49.567162 s: rateControlParams -> VBRsensitivity : 0
[HOST] [IPU2 ] 49.567254 s: rateControlParams -> skipDistributionWindowLength : 5
[HOST] [IPU2 ] 49.567345 s: rateControlParams -> numSkipInDistributionWindow : 2
[HOST] [IPU2 ] 49.567406 s: rateControlParams -> enableHRDComplianceMode : 1
[HOST] [IPU2 ] 49.567498 s: rateControlParams -> frameSkipThMulQ5 : 0
[HOST] [IPU2 ] 49.567650 s: rateControlParams -> vbvUseLevelThQ5 : 0
[HOST] [IPU2 ] 49.567742 s:
[HOST] [IPU2 ] 49.567772 s: interCodingParams -> interCodingPreset : 0
[HOST] [IPU2 ] 49.567864 s: interCodingParams -> searchRangeHorP : 144
[HOST] [IPU2 ] 49.567925 s: interCodingParams -> searchRangeVerP : 32
[HOST] [IPU2 ] 49.568047 s: interCodingParams -> searchRangeHorB : 144
[HOST] [IPU2 ] 49.568108 s: interCodingParams -> searchRangeVerB : 16
[HOST] [IPU2 ] 49.568199 s: interCodingParams -> interCodingBias : 1
[HOST] [IPU2 ] 49.568260 s: interCodingParams -> skipMVCodingBias : 1
[HOST] [IPU2 ] 49.568352 s: interCodingParams -> minBlockSizeP : 0
[HOST] [IPU2 ] 49.568413 s: interCodingParams -> minBlockSizeB : 0
[HOST] [IPU2 ] 49.568504 s: interCodingParams -> meAlgoMode : 0
[HOST] [IPU2 ] 49.568565 s:
[HOST] [IPU2 ] 49.568596 s: intraCodingParams -> intraCodingPreset : 0
[HOST] [IPU2 ] 49.568687 s: intraCodingParams -> lumaIntra4x4Enable : 255
[HOST] [IPU2 ] 49.568779 s: intraCodingParams -> lumaIntra8x8Enable : 0
[HOST] [IPU2 ] 49.568870 s: intraCodingParams -> lumaIntra16x16Enable : 15
[HOST] [IPU2 ] 49.568931 s: intraCodingParams -> chromaIntra8x8Enable : 15
[HOST] [IPU2 ] 49.569053 s: intraCodingParams -> chromaComponentEnable : 1
[HOST] [IPU2 ] 49.569114 s: intraCodingParams -> intraRefreshMethod : 0
[HOST] [IPU2 ] 49.569206 s: intraCodingParams -> intraRefreshRate : 0
[HOST] [IPU2 ] 49.569297 s: intraCodingParams -> gdrOverlapRowsBtwFrames : 0
[HOST] [IPU2 ] 49.569358 s: intraCodingParams -> constrainedIntraPredEnable : 0
[HOST] [IPU2 ] 49.569450 s: intraCodingParams -> intraCodingBias : 0
[HOST] [IPU2 ] 49.569541 s:
[HOST] [IPU2 ] 49.569572 s: sliceCodingParams -> sliceCodingPreset : 1
[HOST] [IPU2 ] 49.569633 s: sliceCodingParams -> sliceMode : 2
[HOST] [IPU2 ] 49.569724 s: sliceCodingParams -> sliceUnitSize : 2500
[HOST] [IPU2 ] 49.569816 s: sliceCodingParams -> sliceStartOffset : [0 0 0]
[HOST] [IPU2 ] 49.569907 s: sliceCodingParams -> streamFormat : 0
[HOST] [IPU2 ] 49.569968 s:
[HOST] [IPU2 ] 49.570029 s:

I hope you should get expected results now.

please make these change and let me know.

Thanks

Ram

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Thank you Ram,

we will check that on monday!

Have a nice weekend!

Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

I am getting the expected results. I dumped the bitstream and checked with an analyzer.

I have attached the bitstream of 4 frames generated for sliceUnitSize = 2500.

1280x720_2500sliceSize.zip

Thanks

Ram

0 Ramprasad over 6 years ago in reply to Ramprasad

TI__Mastermind 30480 points

Hi Sebastian,

I referred H264 Technical reference manual, but I didn't find anything on indicating a slice start postiion.

But I think you will still be able to make use of SLICE MODE based encoding with low-latency with a single output buffer to get the index position.

It doesn't seem to be complex but needs some time to implement it in visionSDK encLink.

I would recommend to go through first the document H264_Encoder_HDVICP2_UserGuide.pdf Appendix G: Low Latency / Sub Frame Level Synchronization.

Refer Section G.3 H.264 Encoder Output with sub frame level synchronization for your requirement.

Thanks

Ram

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

thank you, we will check that!
At the moment we need a fast solution, so we are still working on byte streams. But we have trouble to get a working stream.
We are still analyzing it.

Sebastian

0 Sebastian Hauschild over 6 years ago in reply to Sebastian Hauschild

Expert 1070 points

Hi Ram,

In attachment in our e mail you will find dump buffers the encoder puts out after our usecase is started.
1buf is the first buffer. Inside the array we will see SPS, PPS and one IDR slice. In 2buf, 3buf and 4buf, only non IDR slice are inside of the files. Then in 5buf we find SPS, PPS again but with non IDR slice.

Our question is

As we already knew, we get encoded frame only in form of bitstream. How to parse the binary buffer correctly? In the buffer normally we search for the start code (0 0 0 1) in front of the nal header. For example, we have

00 00 00 01 27 42 80 2a 8b 95 00 f0 04 4f cb 00 40
00 00 00 01 28 de 1e 20 00 00 00 ff 06 80
00 00 00 01 25 88 80 01 00 9f ff fc 3d 14 00 04 0c 7c 98 b9 39 39 39 39 39 39 .. until end of buffer(outBuf->fillLength)
Is it correct how we parse the buffer?

I see IDR slice only in first frame or sometimes also in some following frames, as we see in 5buf. After that we get only SPS, PPS, non IDR, non IDR …
Could you explain that? Because i think with our setting IDR should always come after SPS and PPS?

I found also one buffer with length more than 100-200kBytes. We know that inside the buffer we will find multiple slices. But are the slices inside the buffer parts of one frame? Or more than 1 frame?

Thanks,
Sebastian

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

I concatenated the bin file you shared and analyzed the stream. Please find the attachment. Buf.zip

Please have all bin files in a directory and upload the directory always. This website doesn't allow to upload bin files.

I am using SAAnalyzer to view the stream. This is a freely available viewer to view h264 stream. Please use this on your window machine to view the bitstream. SAnalyzer.zip

1) Yes, currently finding the start code in the bitstream and traverse till next start-code is the only way to find out the slize size. I am working on implementing Low latency based encoding which I metnioned . This will avoid you parsing and you should get indexes directly. I

2) Yes, I see SPS and PPS header in the non-IDR (actually it is I frame) in the 5th buf you sent. I am also seeing this issue. This is due to NALU setting of the encoder. If your expectation is to get this enabled for only IDR frame, that can be done. I will send you the NALu setting required.

3) From the encoder when you get one buffer, it is a complete frame with multiple slices in it.

Can you let me know which chains-link usecase is run by you?

I have a simple chain of capture--> Encode-->NULLsink usecase to get the bitstream dump in a file. and I am the SAAnalyzer to view the bitstream.

Let me know if you need this.

Thanks

Ram

0 Ramprasad over 6 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

If you look at encLink_h264.c, you can see this comment.

/* We want SPS and PPS to be set for every intra frame. Hence configure the
* the NALU control params to force encoder to insert SPS/PPS on every
* I frame
*/

Comment these lines to avoid insertion of SPS, PPS and SEI in I frames.

ENCLINK_H264_SETNALU_MASK_SPS(staticParams->nalUnitControlParams. naluPresentMaskIntraPicture);
ENCLINK_H264_SETNALU_MASK_PPS(staticParams->nalUnitControlParams. naluPresentMaskIntraPicture);
ENCLINK_H264_SETNALU_MASK_SEI(staticParams->nalUnitControlParams. naluPresentMaskIntraPicture);

you will not observe SPS, PPS and SEI in I frames. I have verified this setting.

Thanks

Ram

0 Sebastian Hauschild over 6 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

we have our own test usecase which has the following structure:

IssCapture -> IssM2mIsp -> Dup -> IssM2mResizer -> Display_Video
IssM2mIsp -> Alg_IssAewb
Dup -> VPE -> Encode -> RtpTx (A15)

We send a camera stream to a display an a copy of the stream to a rtp link, that we are implementing at the moment.

We are trying now to use a deterministic input insted of a camera stream. So we exchange IssCapture with a NullSrcDisplay to inject a test pattern image.

Do you need the usecase? I can provide you the sources of the rtp link and the UC.

Sebastian

0 Sebastian Hauschild over 5 years ago in reply to Sebastian Hauschild

Expert 1070 points

Hi Ram,

at the moment the encoder looks good! We have still trouble with our RTP stream, but the buffers we have dumped from the encoder are playable with VLC Player.
Now we have issues with the TDA IPC. The encoder buffers sent to the A15 (RTP Link) are not refreshed. But thats another topic we have to clarify.

If we have solved the IPC issue I will maybe come back to you if there are still some questions regarding the encoder.

See you and thank you very much for your help Ram!! :)
Sebastian

0 Ramprasad over 5 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

Thanks for confirmation.

Please create a new thread on any issue reported.

Thanks

Ram

0 Sebastian Hauschild over 5 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

we found out that the link buffers sent from encoder link to RTP link (A15) are read by the RTP link from the cache.
If we invalidate the cache from A15 side the buffers are updated. We don't know if that is an IPC "bug"? But we solved that in the RTP link.
By the way we saw that you have created the UC you sent us for the linux based VSDK. We are using TI Bios only.

Now back to the encoder :)
We have a stream now. But our stream has only one IDR at the beginning.

Buffer1: sps pps idr -> non -> non -> non

Buffer2: sps pps non -> non -> non ->non
Buffer3: sps pps non -> non -> non -> non
...
BufferN: sps pps non -> non -> non -> non

We need more IDRs according to the parameter pLinkChPrm->IDRFrameInterval = different to 0. But this parameter is ignored!?

Regards and thank you,
Sebastian

0 Ramprasad over 5 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

I am also observing the same issue. I even printed status->IDRFrameInterval that also shows 0.

In enclink_h264.c

/* To set IDR frame periodically instead of I Frame */
staticParams->IDRFrameInterval = 1;
/* Override with App settings */
staticParams->IDRFrameInterval = algCreateParams->IDRFrameInterval;

I am seeing eventhough the application sets IDRFrameInterval =1 , I am seeing algCreateParams->IDRFrameInterval = 0, not sure why this is happening.

I commented this line

staticParams->IDRFrameInterval = algCreateParams->IDRFrameInterval;

and now I seen every I frame as IDR (due to this instruction staticParams->IDRFrameInterval = 1)

I will try to find out why this value is changing from 1 to 0 when sent from application, but rest all elements are taking their values.

Thanks

Ram

0 Ramprasad over 5 years ago in reply to Ramprasad

TI__Mastermind 30480 points

Hi Sebastian,

IDRFrameInterval value sent from chain application was not being copied to internal structure and this made to set IDRFrameInterval as 0.

Please make this change, it should work.

--- a/links_fw/src/rtos/links_ipu/iva/iva_enc/encLink_common.c
+++ b/links_fw/src/rtos/links_ipu/iva/iva_enc/encLink_common.c
@@ -831,6 +831,7 @@ static Int32 enclink_codec_set_ch_alg_create_params(EncLink_Obj * pObj,
pChAlgCreatePrm->numTemporalLayer = pChCreatePrm->numTemporalLayer;
pChAlgCreatePrm->enableSVCExtensionFlag = pChCreatePrm->enableSVCExtensionFlag;
pChAlgCreatePrm->enableWaterMarking = pChCreatePrm->enableWaterMarking;
+ pChAlgCreatePrm->IDRFrameInterval = pChCreatePrm->IDRFrameInterval;

return ENC_LINK_S_SUCCESS;
}

0 Sebastian Hauschild over 5 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

thank you! Now IDRFrameInterval is working well! :)

We have now the problem, that the SEI sometimes has another start code (00 00 00 ff) with a special parameterset:

[IPU1-0]     38.853931 s:
[IPU1-0]     38.853992 s: --------- CH 0 : H264 ENC : Dynamic Params -------
[IPU1-0]     38.854053 s:
[IPU1-0]     38.854114 s: videnc2DynamicParams -> inputHeight             : 1080
[IPU1-0]     38.854236 s: videnc2DynamicParams -> inputWidth              : 1920
[IPU1-0]     38.854328 s: videnc2DynamicParams -> refFrameRate            : 30000
[IPU1-0]     38.854419 s: videnc2DynamicParams -> targetFrameRate         : 30000
[IPU1-0]     38.854480 s: videnc2DynamicParams -> targetBitRate           : 10000000
[IPU1-0]     38.854572 s: videnc2DynamicParams -> intraFrameInterval      : 4
[IPU1-0]     38.854663 s: videnc2DynamicParams -> generateHeader          : 0
[IPU1-0]     38.854724 s: videnc2DynamicParams -> captureWidth            : 1920
[IPU1-0]     38.854816 s: videnc2DynamicParams -> forceFrame              : -1
[IPU1-0]     38.854907 s: videnc2DynamicParams -> interFrameInterval      : 1
[IPU1-0]     38.854968 s: videnc2DynamicParams -> mvAccuracy              : 2
[IPU1-0]     38.855060 s: videnc2DynamicParams -> sampleAspectRatioHeight : 1
[IPU1-0]     38.855121 s: videnc2DynamicParams -> sampleAspectRatioWidth  : 1
[IPU1-0]     38.855243 s: videnc2DynamicParams -> ignoreOutbufSizeFlag    : 1
[IPU1-0]     38.855334 s: videnc2DynamicParams -> lateAcquireArg          : -1
[IPU1-0]     38.855395 s:
[IPU1-0]     38.855426 s: rateControlParams -> rateControlParamsPreset        : 1
[IPU1-0]     38.855517 s: rateControlParams -> scalingMatrixPreset            : 0
[IPU1-0]     38.855609 s: rateControlParams -> rcAlgo                         : 0
[IPU1-0]     38.855700 s: rateControlParams -> qpI                            : 25
[IPU1-0]     38.855761 s: rateControlParams -> qpMaxI                         : 51
[IPU1-0]     38.855853 s: rateControlParams -> qpMinI                         : 0
[IPU1-0]     38.855944 s: rateControlParams -> qpP                            : 25
[IPU1-0]     38.856036 s: rateControlParams -> qpMaxP                         : 51
[IPU1-0]     38.856097 s: rateControlParams -> qpMinP                         : 0
[IPU1-0]     38.856219 s: rateControlParams -> qpOffsetB                      : 4
[IPU1-0]     38.856310 s: rateControlParams -> qpMaxB                         : 44
[IPU1-0]     38.856402 s: rateControlParams -> qpMinB                         : 10
[IPU1-0]     38.856463 s: rateControlParams -> allowFrameSkip                 : 0
[IPU1-0]     38.856554 s: rateControlParams -> removeExpensiveCoeff           : 0
[IPU1-0]     38.856646 s: rateControlParams -> chromaQPIndexOffset            : 0
[IPU1-0]     38.856707 s: rateControlParams -> IPQualityFactor                : 0
[IPU1-0]     38.856798 s: rateControlParams -> initialBufferLevel             : 0
[IPU1-0]     38.856890 s: rateControlParams -> HRDBufferSize                  : 2500000
[IPU1-0]     38.856981 s: rateControlParams -> minPicSizeRatioI               : 0
[IPU1-0]     38.857042 s: rateControlParams -> maxPicSizeRatioI               : 0
[IPU1-0]     38.857134 s: rateControlParams -> minPicSizeRatioP               : 0
[IPU1-0]     38.857225 s: rateControlParams -> maxPicSizeRatioP               : 0
[IPU1-0]     38.857317 s: rateControlParams -> minPicSizeRatioB               : 0
[IPU1-0]     38.857408 s: rateControlParams -> maxPicSizeRatioB               : 0
[IPU1-0]     38.857500 s: rateControlParams -> enablePRC                      : 1
[IPU1-0]     38.857561 s: rateControlParams -> enablePartialFrameSkip         : 0
[IPU1-0]     38.857652 s: rateControlParams -> discardSavedBits               : 1
[IPU1-0]     38.857744 s: rateControlParams -> VBRDuration                    : 8
[IPU1-0]     38.857835 s: rateControlParams -> VBRsensitivity                 : 0
[IPU1-0]     38.857896 s: rateControlParams -> skipDistributionWindowLength   : 5
[IPU1-0]     38.857988 s: rateControlParams -> numSkipInDistributionWindow    : 2
[IPU1-0]     38.858079 s: rateControlParams -> enableHRDComplianceMode        : 1
[IPU1-0]     38.858140 s: rateControlParams -> frameSkipThMulQ5               : 0
[IPU1-0]     38.858262 s: rateControlParams -> vbvUseLevelThQ5                : 0
[IPU1-0]     38.858354 s:
[IPU1-0]     38.858384 s: interCodingParams -> interCodingPreset  : 0
[IPU1-0]     38.858476 s: interCodingParams -> searchRangeHorP    : 144
[IPU1-0]     38.858537 s: interCodingParams -> searchRangeVerP    : 32
[IPU1-0]     38.858628 s: interCodingParams -> searchRangeHorB    : 144
[IPU1-0]     38.858689 s: interCodingParams -> searchRangeVerB    : 16
[IPU1-0]     38.858750 s: interCodingParams -> interCodingBias    : 1
[IPU1-0]     38.858842 s: interCodingParams -> skipMVCodingBias   : 1
[IPU1-0]     38.858903 s: interCodingParams -> minBlockSizeP      : 0
[IPU1-0]     38.858994 s: interCodingParams -> minBlockSizeB      : 0
[IPU1-0]     38.859055 s: interCodingParams -> meAlgoMode         : 0
[IPU1-0]     38.859147 s:
[IPU1-0]     38.859177 s: intraCodingParams -> intraCodingPreset          : 0
[IPU1-0]     38.859269 s: intraCodingParams -> lumaIntra4x4Enable         : 255
[IPU1-0]     38.859360 s: intraCodingParams -> lumaIntra8x8Enable         : 0
[IPU1-0]     38.859452 s: intraCodingParams -> lumaIntra16x16Enable       : 15
[IPU1-0]     38.859513 s: intraCodingParams -> chromaIntra8x8Enable       : 15
[IPU1-0]     38.859604 s: intraCodingParams -> chromaComponentEnable      : 1
[IPU1-0]     38.859696 s: intraCodingParams -> intraRefreshMethod         : 0
[IPU1-0]     38.859757 s: intraCodingParams -> intraRefreshRate           : 0
[IPU1-0]     38.859848 s: intraCodingParams -> gdrOverlapRowsBtwFrames    : 0
[IPU1-0]     38.859909 s: intraCodingParams -> constrainedIntraPredEnable : 0
[IPU1-0]     38.860001 s: intraCodingParams -> intraCodingBias            : 0
[IPU1-0]     38.860092 s:
[IPU1-0]     38.860123 s: sliceCodingParams -> sliceCodingPreset  : 1
[IPU1-0]     38.860214 s: sliceCodingParams -> sliceMode          : 2
[IPU1-0]     38.860275 s: sliceCodingParams -> sliceUnitSize      : 65000
[IPU1-0]     38.860367 s: sliceCodingParams -> sliceStartOffset   : [0 0 0]
[IPU1-0]     38.860458 s: sliceCodingParams -> streamFormat       : 0
[IPU1-0]     38.860519 s:

With that paramaeterset it is normal 00 00 00 01:

 pLinkChPrm->format                 = SYSTEM_IVIDEO_H264BP;
        pLinkChPrm->profile                = 66;
        pLinkChPrm->dataLayout             = VENC_FIELD_SEPARATED;
        pLinkChPrm->fieldMergeEncodeEnable = FALSE;
        pLinkChPrm->enableAnalyticinfo     = 0;
        pLinkChPrm->enableWaterMarking     = 0;
        pLinkChPrm->maxBitRate             = -1;
 
        pLinkChPrm->encodingPreset         = SYSTEM_XDM_USER_DEFINED;
        pLinkChPrm->rateControlPreset      = SYSTEM_IVIDEO_LOW_DELAY;
        pLinkChPrm->enableHighSpeed        = FALSE;
        pLinkChPrm->enableSVCExtensionFlag = FALSE;
        pLinkChPrm->numTemporalLayer       = 0;
        pLinkChPrm->IDRFrameInterval       = 1;
 
        pLinkChPrm->overrideInputScanFormat= 0;
        pLinkChPrm->fieldPicEncode         = 0;

 

        pLinkDynPrm->intraFrameInterval    = 4;
        pLinkDynPrm->targetBitRate         = 10*1000*1000;
        pLinkDynPrm->interFrameInterval    = 1;
        pLinkDynPrm->mvAccuracy            = SYSTEM_IVIDENC2_MOTIONVECTOR_QUARTERPEL;
        pLinkDynPrm->inputFrameRate        = 30; 
        pLinkDynPrm->rcAlg                 = 0;
        pLinkDynPrm->qpMin                 = 0;
        pLinkDynPrm->qpMax                 = 51;
        pLinkDynPrm->qpInit                = 25;
        pLinkDynPrm->vbrDuration           = 8;
        pLinkDynPrm->vbrSensitivity        = 0;
        pLinkDynPrm->maxPicSizeRatioI      = 1;
 
      staticParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_USERDEFINED;    
      staticParams->sliceCodingParams.sliceMode = IH264_SLICEMODE_BYTES;
      staticParams->sliceCodingParams.sliceUnitSize = 65000;
      staticParams->sliceCodingParams.streamFormat = IH264_BYTE_STREAM;

In appendix you will find the both frames where you can see the SEIs.SEI.zip

And can you explain our problem before why it is necessary to invalidate the cache that the buffers are refreshed?
Is there any option we chould set in IPC?

Regards and thanks,
Sebastian

0 Sebastian Hauschild over 5 years ago in reply to Sebastian Hauschild

Expert 1070 points

Hi Ram,

another question to SEI.
Here are 3 extracts from 3 buffers

1.Buffer

00 00 00 01 27 42 80 2a 8b 95 00 f0 04 4f cb 00 40
00 00 00 01 28 de 1e 20
00 00 00 01 06 80
00 00 00 01 25 88 80 01 00 9f ff fc 3d 14 00 04 0c 7c 9a 93 93 93 93 93 93 93

2. Buffer

00 00 00 01 06 05 14 e8 83 45 e0 61 ce 11 e1 9c a8 00 02 a5 d5 c5 1b 1c c0 23 0b 80

00 00 00 01 21 9a 00 08 14 0a 0d 89 84 20 16 c1 51 ea 07 83 dc 0e 62 ad 40 f0 7e 07 83

3.Buffer

00 00 00 01 06 05 14 e8 83 45 e0 61 ce 11 e1 9c a8 00 02 a5 d5 c5 1b 18 8a 2d 65 80

00 00 00 01 21 9a 00 10 24 08 c1 e6 1a 13 08 40 bc 26 2e 1f 14 41 50 2c c0 48 61 e1 3c

Why is there a SEI before NON IDR? And is there difference between SEI before IDR and NON IDR?

This is the parameterset:

in UC:
        pLinkChPrm->format                 = SYSTEM_IVIDEO_H264BP;
        pLinkChPrm->profile                = 66;
        pLinkChPrm->dataLayout             = VENC_FIELD_SEPARATED;
        pLinkChPrm->fieldMergeEncodeEnable = FALSE;
        pLinkChPrm->enableAnalyticinfo     = TRUE;
        pLinkChPrm->enableWaterMarking     = TRUE;
        pLinkChPrm->maxBitRate             = -1;
        /* Set to SYSTEM_XDM_USER_DEFINED if SEI/meta data needs to be enabled */
        pLinkChPrm->encodingPreset         = SYSTEM_XDM_USER_DEFINED;
        pLinkChPrm->rateControlPreset      = SYSTEM_IVIDEO_LOW_DELAY;
        pLinkChPrm->enableHighSpeed        = FALSE;
        pLinkChPrm->enableSVCExtensionFlag = FALSE;
        pLinkChPrm->numTemporalLayer       = 0;
        pLinkChPrm->IDRFrameInterval       = 1;
        pLinkChPrm->overrideInputScanFormat= 0;
        pLinkChPrm->fieldPicEncode         = 0;

 

        pLinkDynPrm->intraFrameInterval    = 4;
        pLinkDynPrm->targetBitRate         = 10*1000*1000;
        pLinkDynPrm->interFrameInterval    = 1;
        pLinkDynPrm->mvAccuracy            = SYSTEM_IVIDENC2_MOTIONVECTOR_QUARTERPEL;
        pLinkDynPrm->inputFrameRate        = 30; 
        pLinkDynPrm->rcAlg                 = 0;
        pLinkDynPrm->qpMin                 = 0;
        pLinkDynPrm->qpMax                 = 51;
        pLinkDynPrm->qpInit                = 25;
        pLinkDynPrm->vbrDuration           = 8;
        pLinkDynPrm->vbrSensitivity        = 0;
        pLinkDynPrm->maxPicSizeRatioI      = 1;

 

in Encoder Link:
staticParams->videnc2Params.metadataType[0] = IH264_SEI_USER_DATA_UNREGISTERED; 
inputBufDesc->numMetaPlanes = 1;

Regards and thank you,
Sebastian

0 Ramprasad over 5 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

>>We have now the problem, that the SEI sometimes has another start code (00 00 00 ff) with a special parameterset:

This is strange, we haven't seen this issue. How often you are seeing the issue?

>>And can you explain our problem before why it is necessary to invalidate the cache that the buffers are refreshed?

Can you share me the changes you added, I will check.

>>Why is there a SEI before NON IDR? And is there difference between SEI before IDR and NON IDR?

This also should not happen if naluControlParams are properly set. How often you are observing this?

Please make sure below settings.

staticParams->nalUnitControlParams.naluPresentMaskIntraPicture = 0x0002;

staticParams->nalUnitControlParams.naluPresentMaskNonIntraPicture = 0x0002;

Thanks

Ram

0 Sebastian Hauschild over 5 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

>> This is strange, we haven't seen this issue. How often you are seeing the issue?
with the parameter set above we get always that start code (00 00 00 ff) after PPS.

>>Can you share me the changes you added, I will check.
We call Cache_inv() on the NEW_DATA event of our RTP link, before we are parsing the encoder link buffer
        /* Cache invalidate required as CPU copy is used */
        Cache_inv(
                bitstreamBuf->bufAddr,
                bitstreamBuf->fillLength,
                (UInt16)Cache_Type_ALLD,
                (Bool)TRUE
                );


>> How often you are observing this?
with above parameter set we get always the problem
and these both parameters are already correctly set in enclink_h264_set_static_params:

staticParams->nalUnitControlParams.naluPresentMaskIntraPicture       =    0x0002;
staticParams->nalUnitControlParams.naluPresentMaskNonIntraPicture =    0x0002;

Then the create function enc_link_h264_create use these static params.
but we don't see any method to read the initialized static params and display them. Can you suggest, how to read them out, esp that both parameters

Regards,
Sebastian

0 Ramprasad over 5 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

I still could not understand what parameter setting is causing this SEI wrong start code. Do you see the issue with default encoder setting(without any changes in encLink_h264.c). Does it mean it was normal earlier and after your slice coding setting changes you are seeing the issues?

I am seeing same cache invalidation setting in nullsink also and that too specifically for encoded bitstream data.

Does rtp link involve cpu copy, then it is required to do cache invalidation.

In enclink_h264_set_static_params()

force

staticParams->nalUnitControlParams.naluPresentMaskIntraPicture = 0x0002;
staticParams->nalUnitControlParams.naluPresentMaskNonIntraPicture = 0x0002;

and try.

Thanks

Ram

0 Ramprasad over 5 years ago in reply to Ramprasad

TI__Mastermind 30480 points

Hi Sebastian,

I think I don't have this change

in Encoder Link:
staticParams->videnc2Params.metadataType[0] = IH264_SEI_USER_DATA_UNREGISTERED; 
inputBufDesc->numMetaPlanes = 1;

I will try with this change and check.

Can you share me your encLink_h264.c?

Thanks
Ram

0 Sebastian Hauschild over 5 years ago in reply to Ramprasad

Expert 1070 points

"I still could not understand what parameter setting is causing this SEI wrong start code. Do you see the issue with default encoder setting(without any changes in encLink_h264.c). Does it mean it was normal earlier and after your slice coding setting changes you are seeing the issues?"

>>yes, it was normal earlier, it means we saw correct start code from SEI, eventhough the content was incorrect, only two bytes with 06 80

0 Sebastian Hauschild over 5 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

here the file:encLink_h264 (1).c

Thank you!

Sebastian

0 Ramprasad over 5 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

I see

staticParams->videnc2Params.metadataType[0] = IH264_SEI_USER_DATA_UNREGISTERED;

but

inputBufDesc->metadataPlaneDesc[inputBufDesc->numMetaPlanes].buf = NULL;

This buffer should have pointed to a valid buffer holding SEI data. Are you not seeing any issue with this?

Appendix A says buf should point to a SEI buffer.

I am guessing you started seeing wrong start code only after enabling USER_DEFINED_SEI.

Thanks

Ram

0 Sebastian Hauschild over 5 years ago in reply to Ramprasad

Expert 1070 points

Hi Ram,

even if this is commented out: // staticParams->videnc2Params.metadataType[0] = IH264_SEI_USER_DATA_UNREGISTERED;
the startcode is wrong. We observed that changing the slice mode has that effect with the start code:

00 00 00 FF 06 80
staticParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_USERDEFINED;
dynamicParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_USERDEFINED;

00 00 00 01 06 80
staticParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_DEFAULT;
dynamicParams->sliceCodingParams.sliceCodingPreset = IH264_SLICECODING_DEFAULT;

thank you,

Sebastian

0 Sebastian Hauschild over 5 years ago in reply to Sebastian Hauschild

Expert 1070 points

Hi Ram,

an additional question. (SEI is more casual)
Our Stream is almost stable.
But we have still trouble with the Parser. We want to avoid to scan for the startCode at the begin of each framebuffer.
The parameter SliceUnitSize sets the max. size of one slice. Is the size of each slice allmost close to this parameter?
So that, when SliceUnitSize is for example 8000 we could start with the scan at 7500 and so on?
The last slice is probably smaller but that is calculatable with the framebuffer size.

Thank you Ram!

Sebastian

0 Ramprasad over 5 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

Good to know that your SEI issue is now resolved. You mentioned enabling slice encoding caused it. Now did you identify the rootcause?

I am enabling low-latency based encoding which will help you to find slice start boundary and slice size. I have partially done.

I will update you once it is done. We can have a webex also to make you understand how this work.

Thanks

Ram

0 Sebastian Hauschild over 5 years ago in reply to Ramprasad

Expert 1070 points

Hello Ram,

Oh, that sounds wonderful! When do you think you will have finished the low latency encoding?

No, we haven't solved the SEI issue. Only delayed because it is not so important. We ignore the SEIs at the moment.

Great, thank you!

Sebastian

0 Ramprasad over 5 years ago in reply to Sebastian Hauschild

TI__Mastermind 30480 points

Hi Sebastian,

I am working on it. I will try to finish it tomorrow. I will post you the update.

Thanks

Ram

Processors

Processors forum

TDA2P-ABZ: Encoder H264 Baseline Profile - TDA2P