This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5728: Realtime 1080p60 encoding problems

Part Number: AM5728

We are having trouble sustaining a single stream 60fps rate with 1080p H.264 encoding using VIDENC2_Process(), resulting in dropped frames.

Our measured execution time for VIDENC2_Process (via clock_gettime, monotonic) on a single 1080p frame often exceeds 20ms; since frames are arriving every 16.6ms, the pipeline falls behind, and the VIP driver drops frames (we are using 8 capture buffers; increasing this only delays the start of the frame drops.)

Our pipeline is VIP (capture at 1080p60) -> VPE (convert YUYV to NV12) -> encoder. It works fine up to and including 1080p50.

CPU or memory contention should not be an issue as we are showing only about 15% total CPU utilization while encoding, and are running at the highest OPP as well as using interleaved memory banks.

The VIDENC2_Process calls occur on a dedicated high-priority, realtime-scheduled thread. All DMA buffers are allocated via DRM / omap_bo operations.

We have observed that gstreamer achieves only ~55fps encoded frame rate for 1080p input, so it appears the issue exists there as well (it silently drops frames and simply adjusts the timing of the subsequent ones, which is not an option in our application.) 

How can the HDVICP2 encoder be made to work at 60FPS 1080p ? What other factors affect encoder performance?

Thanks for any assistance,

---Chris

  • Hi Chris,

    Is this issue occurring with particular set of streams with higher data rate and certain encode parameters or you are seeing this issue with all streams?

    It would be good to get the details so that we can give you some next level pointers.

    Regards

    Karthik

  • The issue occurs with all streams and we've experimented with many different parameter sets; they all fail to keep up with 1080p60 encoding with an average Process() call time of 18-19ms, well above the required 16.6ms for 60fps. 

  • Hi Chris,

    Can you please share all the static and dynamic codec parameters? Also the encoder version (it is mentioned in the Release notes). 

    How complex is the input video in you use case? (Are there too many scene change or high motion?)

  • An example set of parameters is attached. However, these parameters are computed based on input format, frame rate, and user input, so we need to know if it is necessary to constrain them in order to achieve the target frame rate.

    An XDM_GETVERSION call against the encoder algorithm instance returns the string:

        H264ENC_TI DEV.500.V.H264AVC.E.IVAHD.02.00.09.01

    We have been testing with different types of 1080p60 video and see drops with all of them; the less complex ones have fewer drops, that's all. We need to be able to encode arbitrary video streams without drops; quality degradation is expected for high motion/high complexity cases, but frame drops are not acceptable. 

    Sample encoder parameter set (static first, then dynamic as indicated in the log below):

    1624301524.723509 com D 1097 h264codec_debug.h:94: IH264ENC_Params:
    1624301524.723586 com D 1097 h264codec_debug.h:221: IVIDENC2_Params:
    1624301524.723591 com D 1097 h264codec_debug.h:224: size: 316
    1624301524.723614 com D 1097 h264codec_debug.h:225: encodingPreset: XDM_USER_DEFINED (3)
    1624301524.723621 com D 1097 h264codec_debug.h:226: rateControlPreset: IVIDEO_USER_DEFINED (5)
    1624301524.723624 com D 1097 h264codec_debug.h:227: maxHeight: 1080
    1624301524.723627 com D 1097 h264codec_debug.h:228: maxWidth: 1920
    1624301524.723632 com D 1097 h264codec_debug.h:229: dataEndianness: XDM_BYTE (1)
    1624301524.723635 com D 1097 h264codec_debug.h:230: maxInterFrameInterval: 1
    1624301524.723637 com D 1097 h264codec_debug.h:231: maxBitRate: -1
    1624301524.723640 com D 1097 h264codec_debug.h:232: minBitRate: 0
    1624301524.723646 com D 1097 h264codec_debug.h:233: inputChromaFormat: XDM_YUV_420SP (9)
    1624301524.723651 com D 1097 h264codec_debug.h:234: inputContentType: IVIDEO_PROGRESSIVE (0)
    1624301524.723656 com D 1097 h264codec_debug.h:235: operatingMode: IVIDEO_ENCODE_ONLY (1)
    1624301524.723662 com D 1097 h264codec_debug.h:236: profile: IH264_MAIN_PROFILE (77)
    1624301524.723665 com D 1097 h264codec_debug.h:237: level: 4.2
    1624301524.723676 com D 1097 h264codec_debug.h:238: inputDataMode: IVIDEO_ENTIREFRAME (3)
    1624301524.723681 com D 1097 h264codec_debug.h:239: outputDataMode: IVIDEO_ENTIREFRAME (3)
    1624301524.723684 com D 1097 h264codec_debug.h:240: numInputDataUnits: 1
    1624301524.723687 com D 1097 h264codec_debug.h:241: numOutputDataUnits: 1
    1624301524.723692 com D 1097 h264codec_debug.h:242: metadataType[0]: IVIDEO_METADATAPLANE_NONE (-1)
    1624301524.723697 com D 1097 h264codec_debug.h:243: metadataType[1]: IVIDEO_METADATAPLANE_NONE (-1)
    1624301524.723703 com D 1097 h264codec_debug.h:244: metadataType[2]: IVIDEO_METADATAPLANE_NONE (-1)
    1624301524.723706 com D 1097 h264codec_debug.h:245:
    1624301524.724699 com D 1097 h264codec_debug.h:322: IH264ENC_RateControlParams:
    1624301524.724719 com D 1097 h264codec_debug.h:325: rateControlParamsPreset: IH264_RATECONTROLPARAMS_USERDEFINED (1)
    1624301524.724725 com D 1097 h264codec_debug.h:326: scalingMatrixPreset: IH264_SCALINGMATRIX_NONE (0)
    1624301524.724731 com D 1097 h264codec_debug.h:327: rcAlgo: IH264_RATECONTROL_PRC (0)
    1624301524.724734 com D 1097 h264codec_debug.h:328: qpI: 19
    1624301524.724737 com D 1097 h264codec_debug.h:329: qpMaxI: 51
    1624301524.724740 com D 1097 h264codec_debug.h:330: qpMinI: 19
    1624301524.724742 com D 1097 h264codec_debug.h:331: qpP: 28
    1624301524.724744 com D 1097 h264codec_debug.h:332: qpMaxP: 51
    1624301524.724747 com D 1097 h264codec_debug.h:333: qpMinP: 17
    1624301524.724749 com D 1097 h264codec_debug.h:334: qpOffsetB: 4
    1624301524.724752 com D 1097 h264codec_debug.h:335: qpMaxB: 51
    1624301524.724754 com D 1097 h264codec_debug.h:336: qpMinB: 17
    1624301524.724756 com D 1097 h264codec_debug.h:337: allowFrameSkip: 0
    1624301524.724759 com D 1097 h264codec_debug.h:338: removeExpensizeCoeff: 0
    1624301524.724761 com D 1097 h264codec_debug.h:339: chromaQPIndexOffset: 0
    1624301524.724765 com D 1097 h264codec_debug.h:340: IPQualityFactor: IH264_QUALITY_FACTOR_1 (0)
    1624301524.724768 com D 1097 h264codec_debug.h:341: initialBufferLevel: 10000000
    1624301524.724771 com D 1097 h264codec_debug.h:342: HRDBufferSize: 10000000
    1624301524.724774 com D 1097 h264codec_debug.h:343: minPicSizeRatioI: 0
    1624301524.724776 com D 1097 h264codec_debug.h:344: maxPicSizeRatioI: 0
    1624301524.724778 com D 1097 h264codec_debug.h:345: minPicSizeRatioP: 0
    1624301524.724781 com D 1097 h264codec_debug.h:346: maxPicSizeRatioP: 0
    1624301524.724783 com D 1097 h264codec_debug.h:347: minPicSizeRatioB: 0
    1624301524.724785 com D 1097 h264codec_debug.h:348: maxPicSizeRatioB: 0
    1624301524.724787 com D 1097 h264codec_debug.h:349: enablePRC: 1
    1624301524.724790 com D 1097 h264codec_debug.h:350: enablePartialFrameSkip: 0
    1624301524.724792 com D 1097 h264codec_debug.h:351: discardSavedBits: 0
    1624301524.724794 com D 1097 h264codec_debug.h:352: reserved: 0
    1624301524.724797 com D 1097 h264codec_debug.h:353: VBRDuration: 8
    1624301524.724799 com D 1097 h264codec_debug.h:354: VBRSensitivity: 0
    1624301524.724802 com D 1097 h264codec_debug.h:355: skipDistributionWindowLength: 5
    1624301524.724804 com D 1097 h264codec_debug.h:356: numSkipInDistributionWindow: 1
    1624301524.724807 com D 1097 h264codec_debug.h:357: enableHRDComplianceMode: 1
    1624301524.724810 com D 1097 h264codec_debug.h:358: frameSkipThMulQ5: 0
    1624301524.724813 com D 1097 h264codec_debug.h:359: vbvUseLevelThQ5: 0
    1624301524.724816 com D 1097 h264codec_debug.h:360: reservedRC[3]: [0, 0, 0]
    1624301524.724819 com D 1097 h264codec_debug.h:361:
    1624301524.724822 com D 1097 h264codec_debug.h:370: IH264ENC_InterCodingParams:
    1624301524.724828 com D 1097 h264codec_debug.h:373: interCodingPreset: IH264_INTERCODING_DEFAULT (0)
    1624301524.724832 com D 1097 h264codec_debug.h:374: searchRangeHorP: 144
    1624301524.724835 com D 1097 h264codec_debug.h:375: searchRangeVerP: 32
    1624301524.724838 com D 1097 h264codec_debug.h:376: searchRangeHorB: 144
    1624301524.724841 com D 1097 h264codec_debug.h:377: searchRangeVerB: 16
    1624301524.724846 com D 1097 h264codec_debug.h:378: interCodingBias: IH264_BIASFACTOR_MEDIUM (1)
    1624301524.724851 com D 1097 h264codec_debug.h:379: skipMVCodingBias: IH264_BIASFACTOR_MEDIUM (1)
    1624301524.724880 com D 1097 h264codec_debug.h:380: minBlockSizeP: IH264_BLOCKSIZE_16x16 (0)
    1624301524.724885 com D 1097 h264codec_debug.h:381: minBlockSizeB: IH264_BLOCKSIZE_16x16 (0)
    1624301524.724891 com D 1097 h264codec_debug.h:382: meAlgoMode: IH264ENC_MOTIONESTMODE_NORMAL (0)
    1624301524.724894 com D 1097 h264codec_debug.h:383:
    1624301524.724897 com D 1097 h264codec_debug.h:392: IH264ENC_IntraCodingParams:
    1624301524.724903 com D 1097 h264codec_debug.h:395: intraCodingPreset: IH264_INTRACODING_DEFAULT (0)
    1624301524.724906 com D 1097 h264codec_debug.h:396: lumaIntra4x4Enable: 0x000
    1624301524.724910 com D 1097 h264codec_debug.h:397: lumaIntra8x8Enable: 0x0ff
    1624301524.724913 com D 1097 h264codec_debug.h:398: lumaIntra16x16Enable: 0x0f
    1624301524.724916 com D 1097 h264codec_debug.h:399: chromaIntra8x8Enable: 0x0f
    1624301524.724922 com D 1097 h264codec_debug.h:400: chromaComponentEnable: IH264_CHROMA_COMPONENT_CR_ONLY (1)
    1624301524.724927 com D 1097 h264codec_debug.h:401: intraRefreshMethod: IH264_INTRAREFRESH_NONE (0)
    1624301524.724930 com D 1097 h264codec_debug.h:402: intraRefreshRate: 0
    1624301524.724933 com D 1097 h264codec_debug.h:403: gdrOverlapRowsBtwFrames: 0
    1624301524.724936 com D 1097 h264codec_debug.h:404: constrainedIntraPredEnable: 0
    1624301524.724941 com D 1097 h264codec_debug.h:405: intraCodingBias: IH264ENC_INTRACODINGBIAS_NORMAL (0)
    1624301524.724944 com D 1097 h264codec_debug.h:406:
    1624301524.726262 com D 1097 h264codec_debug.h:415: IH264ENC_NALUControlParams:
    1624301524.726279 com D 1097 h264codec_debug.h:418: naluControlPreset: IH264_NALU_CONTROL_USERDEFINED (1)
    1624301524.726283 com D 1097 h264codec_debug.h:419: naluPresentMaskStartOfSequence: 0x2180
    1624301524.726287 com D 1097 h264codec_debug.h:420: naluPresentMaskIDRPicture: 0x21a0
    1624301524.726290 com D 1097 h264codec_debug.h:421: naluPresentMaskIntraPicture: 0x0002
    1624301524.726293 com D 1097 h264codec_debug.h:422: naluPresentMaskNonIntraPicture: 0x0002
    1624301524.726296 com D 1097 h264codec_debug.h:423: naluPresentMaskEndOfSequence: 0x0c00
    1624301524.726299 com D 1097 h264codec_debug.h:424:
    1624301524.726302 com D 1097 h264codec_debug.h:433: IH264ENC_SliceCodingParams:
    1624301524.726307 com D 1097 h264codec_debug.h:436: sliceCodingPreset: IH264_SLICECODING_DEFAULT (0)
    1624301524.726313 com D 1097 h264codec_debug.h:437: sliceMode: IH264_SLICEMODE_NONE (0)
    1624301524.726317 com D 1097 h264codec_debug.h:438: sliceUnitSize: 0
    1624301524.726320 com D 1097 h264codec_debug.h:439: sliceStartOffset[3]: 0, 0, 0
    1624301524.726325 com D 1097 h264codec_debug.h:440: streamFormat: IH264_BYTE_STREAM (0)
    1624301524.726328 com D 1097 h264codec_debug.h:441:
    1624301524.726331 com D 1097 h264codec_debug.h:450: IH264ENC_LoopFilterParams:
    1624301524.726336 com D 1097 h264codec_debug.h:453: loopfilterPreset: IH264_LOOPFILTER_DEFAULT (0)
    1624301524.726341 com D 1097 h264codec_debug.h:454: loopfilterDisableIDC: IH264_DISABLE_FILTER_NONE (0)
    1624301524.726344 com D 1097 h264codec_debug.h:455: filterOffsetA: 0
    1624301524.726347 com D 1097 h264codec_debug.h:456: filterOffsetB: 0
    1624301524.726349 com D 1097 h264codec_debug.h:457:
    1624301524.726352 com D 1097 h264codec_debug.h:466: IH264ENC_FMOCodingParams:
    1624301524.726357 com D 1097 h264codec_debug.h:469: fmoCodingPreset: IH264_FMOCODING_NONE (0)
    1624301524.726360 com D 1097 h264codec_debug.h:470: numSliceGroups: 1
    1624301524.726366 com D 1097 h264codec_debug.h:471: sliceGroupMapType: IH264_RASTER_SCAN_SLICE_GRP (4)
    1624301524.726371 com D 1097 h264codec_debug.h:472: sliceGroupChangeDirectionFlag: IH264_RASTER_SCAN (0)
    1624301524.726375 com D 1097 h264codec_debug.h:473: sliceGroupChangeRate: 0
    1624301524.726377 com D 1097 h264codec_debug.h:474: sliceGroupChangeCycle: 0
    1624301524.726380 com D 1097 h264codec_debug.h:475: sliceGroupParams[2]: 0, 0
    1624301524.726383 com D 1097 h264codec_debug.h:476:
    1624301524.726385 com D 1097 h264codec_debug.h:485: IH264ENC_VUICodingParams:
    1624301524.726390 com D 1097 h264codec_debug.h:488: vuiCodingPreset: IH264_VUICODING_USERDEFINED (1)
    1624301524.726393 com D 1097 h264codec_debug.h:489: aspectRatioInfoPresentFlag: 1
    1624301524.726400 com D 1097 h264codec_debug.h:490: aspectRatioIdc: IH264ENC_ASPECTRATIO_EXTENDED (255)
    1624301524.726403 com D 1097 h264codec_debug.h:491: videoSignalTypePresentFlag: 0
    1624301524.726408 com D 1097 h264codec_debug.h:492: videoFormat: IH264ENC_VIDEOFORMAT_COMPONENT (0)
    1624301524.726411 com D 1097 h264codec_debug.h:493: videoFullRangeFlag: 0
    1624301524.726414 com D 1097 h264codec_debug.h:494: timingInfoPresentFlag: 1
    1624301524.726416 com D 1097 h264codec_debug.h:495: hrdParamsPresentFlag: 1
    1624301524.726419 com D 1097 h264codec_debug.h:496: numUnitsInTicks: 1000
    1624301524.726421 com D 1097 h264codec_debug.h:497:
    1624301524.726423 com D 1097 h264codec_debug.h:506: IH264ENC_StereoInfoParams:
    1624301524.726428 com D 1097 h264codec_debug.h:509: stereoInfoPreset: IH264_STEREOINFO_DISABLE (0)
    1624301524.726431 com D 1097 h264codec_debug.h:510: topFieldIsLeftViewFlag: 1
    1624301524.726434 com D 1097 h264codec_debug.h:511: viewSelfContainedFlag: 0
    1624301524.726436 com D 1097 h264codec_debug.h:512:
    1624301524.726438 com D 1097 h264codec_debug.h:521: IH264ENC_FramePackingSEIParams:
    1624301524.726443 com D 1097 h264codec_debug.h:524: framePackingPreset: IH264_FRAMEPACK_SEI_DISABLE (0)
    1624301524.726449 com D 1097 h264codec_debug.h:525: framePackingType: IH264_FRAMEPACK_SIDE_BY_SIDE (3)
    1624301524.726452 com D 1097 h264codec_debug.h:526: frame0PositionX: 0
    1624301524.726455 com D 1097 h264codec_debug.h:527: frame0PositionY: 0
    1624301524.726458 com D 1097 h264codec_debug.h:528: frame1PositionX: 0
    1624301524.726460 com D 1097 h264codec_debug.h:529: frame1PositionY: 0
    1624301524.726463 com D 1097 h264codec_debug.h:530: reservedByte: 0
    1624301524.726465 com D 1097 h264codec_debug.h:531:
    1624301524.726467 com D 1097 h264codec_debug.h:540: IH264ENC_SVCCodingParams:
    1624301524.726472 com D 1097 h264codec_debug.h:543: svcExtensionFlag: IH264_SVC_EXTENSION_FLAG_DISABLE (0)
    1624301524.726475 com D 1097 h264codec_debug.h:544: dependencyID: 0
    1624301524.726478 com D 1097 h264codec_debug.h:545: qualityID: 0
    1624301524.726481 com D 1097 h264codec_debug.h:546: enhancementProfileID: 0
    1624301524.726483 com D 1097 h264codec_debug.h:547: layerIndex: 0
    1624301524.726485 com D 1097 h264codec_debug.h:548: refLayerDQId: 0
    1624301524.726488 com D 1097 h264codec_debug.h:549:
    1624301524.726492 com D 1097 h264codec_debug.h:110: interlaceCodingType: IH264_INTERLACE_FIELDONLY_ARF (3)
    1624301524.726495 com D 1097 h264codec_debug.h:111: bottomFieldIntra: 0
    1624301524.726500 com D 1097 h264codec_debug.h:112: gopStructure: IH264ENC_GOPSTRUCTURE_UNIFORM (1)
    1624301524.726505 com D 1097 h264codec_debug.h:113: entropyCodingMode: IH264_ENTROPYCODING_CABAC (1)
    1624301524.726511 com D 1097 h264codec_debug.h:114: transformBlockSize: IH264_TRANSFORM_ADAPTIVE (2)
    1624301524.726514 com D 1097 h264codec_debug.h:115: log2MaxFNumMinus4: 10
    1624301524.726519 com D 1097 h264codec_debug.h:116: picOrderCountType: IH264_POC_TYPE_0 (0)
    1624301524.726523 com D 1097 h264codec_debug.h:117: enableWatermark: 0
    1624301524.726526 com D 1097 h264codec_debug.h:118: IDRFrameInterval: 1
    1624301524.726529 com D 1097 h264codec_debug.h:119: pConstantMemory: 0
    1624301524.726531 com D 1097 h264codec_debug.h:120: maxIntraFrameInterval: 2147483647
    1624301524.726535 com D 1097 h264codec_debug.h:121: debugTraceLevel: 0
    1624301524.726537 com D 1097 h264codec_debug.h:122: lastNFramesToLog: 0
    1624301524.726540 com D 1097 h264codec_debug.h:123: enableAnalyticinfo: 0
    1624301524.726542 com D 1097 h264codec_debug.h:124: enableGMVSei: 0
    1624301524.726545 com D 1097 h264codec_debug.h:125: constraintSetFlags: 0x00
    1624301524.726547 com D 1097 h264codec_debug.h:126: enableRCDO: 0
    1624301524.726552 com D 1097 h264codec_debug.h:127: enableLongTermRefFrame: IH264ENC_LTRP_NONE (0)
    1624301524.726555 com D 1097 h264codec_debug.h:128: LTRPPeriod: 0
    1624301524.726560 com D 1097 h264codec_debug.h:129: numTemporalLayer: IH264_TEMPORAL_LAYERS_1 (1)
    1624301524.726565 com D 1097 h264codec_debug.h:130: referencePicMarking: IH264_LONG_TERM_PICTURE (1)
    1624301524.726568 com D 1097 h264codec_debug.h:131: reservedParams[3]: 0, 0, 0
    1624301524.726571 com D 1097 h264codec_debug.h:132:
    1624301524.728490 com D 1097 h264codec_debug.h:141: IH264ENC_DynamicParams:
    1624301524.728494 com D 1097 h264codec_debug.h:255: IVIDENC2_DynamicParams:
    1624301524.728496 com D 1097 h264codec_debug.h:258: size: 240
    1624301524.728500 com D 1097 h264codec_debug.h:259: inputHeight: 1080
    1624301524.728502 com D 1097 h264codec_debug.h:260: inputWidth: 1920
    1624301524.728504 com D 1097 h264codec_debug.h:261: refFrameRate: 59940
    1624301524.728506 com D 1097 h264codec_debug.h:262: targetFrameRate: 59940
    1624301524.728508 com D 1097 h264codec_debug.h:263: targetBitRate: 5000000
    1624301524.728510 com D 1097 h264codec_debug.h:264: intraFrameInterval: 60
    1624301524.728516 com D 1097 h264codec_debug.h:265: generateHeader: XDM_ENCODE_AU (0)
    1624301524.728519 com D 1097 h264codec_debug.h:266: captureWidth: 1920
    1624301524.728524 com D 1097 h264codec_debug.h:267: forceFrame: IVIDEO_NA_FRAME (-1)
    1624301524.728526 com D 1097 h264codec_debug.h:268: interFrameInterval: 1
    1624301524.728531 com D 1097 h264codec_debug.h:269: mvAccuracy: IVIDENC2_MOTIONVECTOR_QUARTERPEL (2)
    1624301524.728534 com D 1097 h264codec_debug.h:270: sampleAspectRatioHeight: 1
    1624301524.728536 com D 1097 h264codec_debug.h:271: sampleAspectRatioWidth: 1
    1624301524.728538 com D 1097 h264codec_debug.h:272: ignoreOutbufSizeFlag: 0
    1624301524.728540 com D 1097 h264codec_debug.h:273: putDataFxn: (nil)
    1624301524.728542 com D 1097 h264codec_debug.h:274: putDataHandle: (nil)
    1624301524.728544 com D 1097 h264codec_debug.h:275: getDataFxn: (nil)
    1624301524.728546 com D 1097 h264codec_debug.h:276: getDataHandle: (nil)
    1624301524.728548 com D 1097 h264codec_debug.h:277: getBufferFxn: (nil)
    1624301524.728549 com D 1097 h264codec_debug.h:278: getBufferHandle: (nil)
    1624301524.728551 com D 1097 h264codec_debug.h:279: lateAcquireArg: -1
    1624301524.728553 com D 1097 h264codec_debug.h:280:
    1624301524.728555 com D 1097 h264codec_debug.h:322: IH264ENC_RateControlParams:
    1624301524.728559 com D 1097 h264codec_debug.h:325: rateControlParamsPreset: IH264_RATECONTROLPARAMS_USERDEFINED (1)
    1624301524.728563 com D 1097 h264codec_debug.h:326: scalingMatrixPreset: IH264_SCALINGMATRIX_NONE (0)
    1624301524.728568 com D 1097 h264codec_debug.h:327: rcAlgo: IH264_RATECONTROL_PRC (0)
    1624301524.728570 com D 1097 h264codec_debug.h:328: qpI: 19
    1624301524.728572 com D 1097 h264codec_debug.h:329: qpMaxI: 51
    1624301524.728574 com D 1097 h264codec_debug.h:330: qpMinI: 19
    1624301524.728576 com D 1097 h264codec_debug.h:331: qpP: 28
    1624301524.728578 com D 1097 h264codec_debug.h:332: qpMaxP: 51
    1624301524.728580 com D 1097 h264codec_debug.h:333: qpMinP: 17
    1624301524.728582 com D 1097 h264codec_debug.h:334: qpOffsetB: 4
    1624301524.728584 com D 1097 h264codec_debug.h:335: qpMaxB: 51
    1624301524.728585 com D 1097 h264codec_debug.h:336: qpMinB: 17
    1624301524.728587 com D 1097 h264codec_debug.h:337: allowFrameSkip: 0
    1624301524.728589 com D 1097 h264codec_debug.h:338: removeExpensizeCoeff: 0
    1624301524.728591 com D 1097 h264codec_debug.h:339: chromaQPIndexOffset: 0
    1624301524.728594 com D 1097 h264codec_debug.h:340: IPQualityFactor: IH264_QUALITY_FACTOR_1 (0)
    1624301524.728597 com D 1097 h264codec_debug.h:341: initialBufferLevel: 10000000
    1624301524.728599 com D 1097 h264codec_debug.h:342: HRDBufferSize: 10000000
    1624301524.728601 com D 1097 h264codec_debug.h:343: minPicSizeRatioI: 0
    1624301524.728603 com D 1097 h264codec_debug.h:344: maxPicSizeRatioI: 0
    1624301524.728605 com D 1097 h264codec_debug.h:345: minPicSizeRatioP: 0
    1624301524.728607 com D 1097 h264codec_debug.h:346: maxPicSizeRatioP: 0
    1624301524.728609 com D 1097 h264codec_debug.h:347: minPicSizeRatioB: 0
    1624301524.728610 com D 1097 h264codec_debug.h:348: maxPicSizeRatioB: 0
    1624301524.728612 com D 1097 h264codec_debug.h:349: enablePRC: 1
    1624301524.728614 com D 1097 h264codec_debug.h:350: enablePartialFrameSkip: 0
    1624301524.728616 com D 1097 h264codec_debug.h:351: discardSavedBits: 0
    1624301524.728617 com D 1097 h264codec_debug.h:352: reserved: 0
    1624301524.728619 com D 1097 h264codec_debug.h:353: VBRDuration: 8
    1624301524.728621 com D 1097 h264codec_debug.h:354: VBRSensitivity: 0
    1624301524.728623 com D 1097 h264codec_debug.h:355: skipDistributionWindowLength: 5
    1624301524.728625 com D 1097 h264codec_debug.h:356: numSkipInDistributionWindow: 1
    1624301524.728627 com D 1097 h264codec_debug.h:357: enableHRDComplianceMode: 1
    1624301524.728629 com D 1097 h264codec_debug.h:358: frameSkipThMulQ5: 0
    1624301524.728630 com D 1097 h264codec_debug.h:359: vbvUseLevelThQ5: 0
    1624301524.728633 com D 1097 h264codec_debug.h:360: reservedRC[3]: [0, 0, 0]
    1624301524.728635 com D 1097 h264codec_debug.h:361:
    1624301524.728637 com D 1097 h264codec_debug.h:370: IH264ENC_InterCodingParams:
    1624301524.728641 com D 1097 h264codec_debug.h:373: interCodingPreset: IH264_INTERCODING_DEFAULT (0)
    1624301524.728643 com D 1097 h264codec_debug.h:374: searchRangeHorP: 144
    1624301524.728646 com D 1097 h264codec_debug.h:375: searchRangeVerP: 32
    1624301524.728648 com D 1097 h264codec_debug.h:376: searchRangeHorB: 144
    1624301524.728650 com D 1097 h264codec_debug.h:377: searchRangeVerB: 16
    1624301524.728654 com D 1097 h264codec_debug.h:378: interCodingBias: IH264_BIASFACTOR_MEDIUM (1)
    1624301524.728658 com D 1097 h264codec_debug.h:379: skipMVCodingBias: IH264_BIASFACTOR_MEDIUM (1)
    1624301524.728662 com D 1097 h264codec_debug.h:380: minBlockSizeP: IH264_BLOCKSIZE_16x16 (0)
    1624301524.728666 com D 1097 h264codec_debug.h:381: minBlockSizeB: IH264_BLOCKSIZE_16x16 (0)
    1624301524.728670 com D 1097 h264codec_debug.h:382: meAlgoMode: IH264ENC_MOTIONESTMODE_NORMAL (0)
    1624301524.728673 com D 1097 h264codec_debug.h:383:
    1624301524.728675 com D 1097 h264codec_debug.h:392: IH264ENC_IntraCodingParams:
    1624301524.728679 com D 1097 h264codec_debug.h:395: intraCodingPreset: IH264_INTRACODING_DEFAULT (0)
    1624301524.728682 com D 1097 h264codec_debug.h:396: lumaIntra4x4Enable: 0x000
    1624301524.728685 com D 1097 h264codec_debug.h:397: lumaIntra8x8Enable: 0x0ff
    1624301524.728688 com D 1097 h264codec_debug.h:398: lumaIntra16x16Enable: 0x0f
    1624301524.728690 com D 1097 h264codec_debug.h:399: chromaIntra8x8Enable: 0x0f
    1624301524.728695 com D 1097 h264codec_debug.h:400: chromaComponentEnable: IH264_CHROMA_COMPONENT_CR_ONLY (1)
    1624301524.728699 com D 1097 h264codec_debug.h:401: intraRefreshMethod: IH264_INTRAREFRESH_NONE (0)
    1624301524.728702 com D 1097 h264codec_debug.h:402: intraRefreshRate: 0
    1624301524.728704 com D 1097 h264codec_debug.h:403: gdrOverlapRowsBtwFrames: 0
    1624301524.728706 com D 1097 h264codec_debug.h:404: constrainedIntraPredEnable: 0
    1624301524.728710 com D 1097 h264codec_debug.h:405: intraCodingBias: IH264ENC_INTRACODINGBIAS_NORMAL (0)
    1624301524.728712 com D 1097 h264codec_debug.h:406:
    1624301524.728715 com D 1097 h264codec_debug.h:433: IH264ENC_SliceCodingParams:
    1624301524.728719 com D 1097 h264codec_debug.h:436: sliceCodingPreset: IH264_SLICECODING_DEFAULT (0)
    1624301524.728724 com D 1097 h264codec_debug.h:437: sliceMode: IH264_SLICEMODE_NONE (0)
    1624301524.728726 com D 1097 h264codec_debug.h:438: sliceUnitSize: 0
    1624301524.728729 com D 1097 h264codec_debug.h:439: sliceStartOffset[3]: 0, 0, 0
    1624301524.728733 com D 1097 h264codec_debug.h:440: streamFormat: IH264_BYTE_STREAM (0)
    1624301524.728735 com D 1097 h264codec_debug.h:441:
    1624301524.728737 com D 1097 h264codec_debug.h:150: sliceGroupChangeCycle: 0
    1624301524.728739 com D 1097 h264codec_debug.h:151: searchCenter: {32767, 32767}
    1624301524.728742 com D 1097 h264codec_debug.h:152: enableStaticMBCount: 0
    1624301524.728744 com D 1097 h264codec_debug.h:153: enableROI: 0
    1624301524.728746 com D 1097 h264codec_debug.h:154: reservedDynParams[3]: 0, 0, 0

  • Hi,

    Thank you for sharing the parameters settings.

    I believe you are measuring the time across VIDENC2_process() process call. Can you also share the sample encoded bit stream from your usecase? Would like to analyse the bitstream.

    How is the behaviour with profile: Baseline (66)?

  • Hi,

    Is the issue resolved? If not can you please share the sample encoded stream for analysis?

    Could you get time to check on Baseline profile?

  • This issue is not resolved.

    I have attempted several times to upload a sample video clip, however, the "insert image/video/file" function here uploads about half of it and then goes back to "no file or URL selected" without any error, and the file is not inserted. Is there a size limit? The clip is about 15 seconds long, 24MB. Should large uploads be placed elsewhere? 

    I'll try with a shorter clip and post again here if it works.

    Regarding baseline profile, when we select that, the codec control call returns an error. We will attempt to fix this, however, our use case requires to be able to encode Main and High 1080p60 material in any case.

  • Hi,

    I am not aware about the size limit to upload the video here. Regret the inconvenience. 

    Attaching an sample config file. Please make sure you modify the resolution, framerate and bitrates only. Other parameters please keep them same as in this sample config./cfs-file/__key/communityserver-discussions-components-files/791/1080p.txt

  • Using the parameter values you supplied above, 1080p60 frame encoding times are pushed up into the 25-26ms range, which is about 40% higher than what we were seeing before and 160% of what will be needed for real-time encoding (frames arrive every 16.6ms, and the average encoding time must be less than that to avoid overrun.)

    So, in short, this made the problem considerably worse.

  • Hi,

    Sorry to hear that! I use the vid2nc2test app from here: https://git.ti.com/cgit/glsdk/omapdrmtest

    I used the config parameters as shared above in attachment it showed ~18-19ms.

    I changed the intraCodingPreset = 0. With this change the time measure across process call shows ~14ms.

  • Making this change reduces the encode time on our system from 25-26ms down to 18-19ms we had before, but not down to the 14ms you are quoting.

    So, back to my original question: what other factors affect encoder performance within a VIP -> VPE -> Encoder pipeline (on an otherwise very lightly loaded system), as opposed to the videnc2test standalone application which only reads a file? The VIP and VPE appear to easily keep up with the required frame rate. In addition to encoding parameters, we have experimented with using tiled memory for the NV12 buffers, and adjusting IVA DMA PEG priority, etc. still without sufficient performance benefit to reach the < 16ms target encoding rate.

  • After lots of additional testing using only videnc2test modified to print encoding times, we have found that, for the known-good parameter set:

    1. The timing overrun does not occur when running SDK 06.01.00.08 on the EVM, or when running it on our board with an older SDK (02.00.01.07).
    2. The timing overrun does occur on our board when running SDK 06.01.00.08 (which we want to use.)
    3. The condition which causes the timing overrun results in frame encoding times that are consistently ~4ms greater than when the condition is not present.
    4. 4ms is almost exactly the amount of time it takes to memcpy a single semi-planar 1080-line NV12 input buffer; therefore
    5. It seems likely that the slow condition is produced as a result of an extra memcpy of the input buffer prior to encoding.
    6. The cmemk module and /proc/cmem are present when running on the EVM; however, they are not present when running either the 02.00.01.07 SDK (which is fast) or the 06.01.00.08 SDK (which is slow) on our hardware.
    7. When we add the cmemk module the lsmod reference count for it on our board is 0, as opposed to 2 on the EVM, and the encoding times are not affected.

    This is suggestive that something may be amiss with the codec input buffers in the "slow" situation which causes them to be memcpy()d. We have experimented with TILED vs non-tiled memory buffers, various cache settings, etc. but none of these things affect the encoding time by more than 1ms or so.

    Does the encoder need to allocate CMEM buffers in order to avoid a memcpy? If so, how do we ensure that the omap_bo_new calls will do that?

    What kernel configuration, device tree, or other environmental factors could be the root cause of the slowdown? (We have not altered the default CMA and CMEM allocations of the stock AM5728 DTS. All of our tests are done at the highest CPU OPP set via the userspace scaler.

  • Hi,

    Thank you for the detailed analysis. The standalone experiments(file in file out) did in my setup showed ~14ms as mentioned above.

    I am an codec engineer and will need to check with other relevant teams for the further analysis on top your findings.

  • We have identified and fixed the root cause of the problem, which is a misconfiguration of the encoder clock DPLL. This occurs because of changes in the DTS organization across the SDK releases. The 02.00.01.07 SDK defaulted to a 532MHz clock whereas the 06.01.00.08 SDK defaults to a 388MHz clock, resulting in slower encode times. It seems that somehow the EVM code over-rides the default clock rate, but we were unable to discover how or where; however, modifying the device tree to force the faster rate corrects the issue and we now obtain ~14ms frame encoding times consistently.

  • Hi,

    That's great news!!

    As it was mentioned in the first post of this query of using OPP_HIGH, we had ruled out the CLK freq.

    Thank you for posting your debug analysis here. That would help others with similar issues.

  • Yes, the CPU OPP_HIGH setting was referred to in the initial post, however, the CPU OPP has no effect on the encoder clock, which must be set independently. With the older SDK, there is no choice as to the encoder clock setting, it was always 532MHz/HIGH.