FPS drop in Scalar link

Prashant Andani

Hi,

I am using IPNC RDK 3.8 with AR0331 Sensor.

I am using 1080p@60fps tri Stream usecase. So my all streams will be like 1080p@60fps H264, 480p@30fps H264and 1080p@5fps MJPEG.

If i set 30FPS for 1st Stream through encoder fps control, It is giving 30Fps but if i set 60fps, it is giving only 50-52fps.

I observed that Scalar link is skipping 10-12fps. So, encoder link is getting only 50-52 fps. as per below encoder statistics.

1). 1080p@30fp Statistics:

[m3video] ==================== EncLink_PrintDetails ====================
[m3video]      60092: HDVICP-ID:0
[m3video]              totalAcquire2wait in msec:       916
[m3video]              totalWait2Isr in msec:     36218
[m3video]              totalIsr2Done in msec:        76
[m3video]              totalWait2Done in msec:     36294
[m3video]              totalDone2Release in msec:         0
[m3video]              totalAcquire2Release in msec:     37640
[m3video]              totalAcq2acqDelay in msec:     16993
[m3video]              totalElapsedTime in msec:     54633
[m3video]              numAccessCnt:      3587
[m3video]              IVA-FPS :        66
[m3video]
[m3video] *** ENCODE Statistics ***
[m3video]
[m3video] Elasped Time           : 18 secs
[m3video]
[m3video]
[m3video] CH | In Recv In Skip In User Out Latency
[m3video] Num | FPS     FPS     Skip FPS FPS Min / Max
[m3video] --------------------------------------------
[m3video]    0 |      60       0       30 30.39 41 / 64
[m3video]    1 |      30       0        0 30.33   5 / 18
[m3video]    2 |      60       0       55 5.06 36 / 49
[m3video]
[m3video] Multi Channel Encode Average Submit Batch Size
[m3video] Max Submit Batch Size : 24
[m3video] IVAHD_0 Average Batch Size : 1
[m3video] IVAHD_0 Max achieved Batch Size : 1
[m3video] ==============================================================
[m3vpss ] ==================== CameraLink_PrintDetails ====================
[m3vpss ] 64885: CAMERA: Fields = 5402 (fps = 90, CPU Load = 3)
[m3vpss ] 64885: CAMERA: Num Resets = 0 (Avg 0 ms per reset)
[m3vpss ] =================================================================
[m3vpss ] 65099: CAMERA: Fields = 5419 (fps = 90), Total Resets = 0 (Avg 0 ms per reset)

2). ->1080p@60fp Statistics:

[m3vpss ] ==================== CameraLink_PrintDetails ====================
[m3vpss ] 184492: CAMERA: Fields = 16205 (fps = 90, CPU Load = 41)
[m3vpss ] 184492: CAMERA: Num Resets = 0 (Avg 0 ms per reset)
[m3vpss ] =================================================================
[m3vpss ] 185105: CAMERA: Fields = 5419 (fps = 90), Total Resets = 0 (Avg 0 ms per reset)
[m3video] ==================== EncLink_PrintDetails ====================
[m3video]      188866: HDVICP-ID:0
[m3video]              totalAcquire2wait in msec:      3361
[m3video]              totalWait2Isr in msec:    129917
[m3video]              totalIsr2Done in msec:       289
[m3video]              totalWait2Done in msec:    130206
[m3video]              totalDone2Release in msec:         0
[m3video]              totalAcquire2Release in msec:    135165
[m3video]              totalAcq2acqDelay in msec:     48253
[m3video]              totalElapsedTime in msec:    183400
[m3video]              numAccessCnt:     12569
[m3video]              IVA-FPS :        68
[m3video]
[m3video] *** ENCODE Statistics ***
[m3video]
[m3video] Elasped Time           : 19 secs
[m3video]
[m3video]
[m3video] CH | In Recv In Skip In User Out Latency
[m3video] Num | FPS     FPS     Skip FPS FPS Min / Max
[m3video] --------------------------------------------
[m3video]    0 |      48       0        0 48.42 112 / 156
[m3video]    1 |      30       0        0 30.21 52 / 116
[m3video]    2 |      60       0       55 5.00 97 / 134
[m3video]
[m3video] Multi Channel Encode Average Submit Batch Size
[m3video] Max Submit Batch Size : 24
[m3video] IVAHD_0 Average Batch Size : 1
[m3video] IVAHD_0 Max achieved Batch Size : 1
[m3video] ==============================================================

Thanks and Regards,

Prashant

over 8 years ago

0 Margarita Gashova over 8 years ago

TI__Guru* 80190 points

Hello,

I will notify the IPNC team for help.

BR
Margarita

0 Prashant Andani over 8 years ago in reply to Margarita Gashova

Intellectual 810 points

Hi Margarita,

Waiting for your reply.

Thanks and Regards,
Prashant

0 Margarita Gashova over 8 years ago in reply to Prashant Andani

TI__Guru* 80190 points

Hello,

I notified the IPNC team for help.

Meanwhile you could search in the e2e for similar issue.

BR
Margarita

0 Brijesh Jadav over 8 years ago in reply to Margarita Gashova

TI__Guru**** 398890 points

Can you try at 50fps? it might be difficult to process 1080p stream at 60fps..

Regards,

Brijesh

0 Prashant Andani over 8 years ago in reply to Brijesh Jadav

Intellectual 810 points

Hi Brijesh,

At 50fps it is giving 49fps.
As per my knowledge DM8127 is capable to process and stream 1080p@60fps.

from the below statistics you can see Scaler link is skipping 13 frames.

[m3vpss ] *** [Scalar0 ] SCLR Statistics ***
[m3vpss ]
[m3vpss ] Elasped Time : 10 secs
[m3vpss ] Total Fields Processed : 23666
[m3vpss ] Total Fields FPS : 87 FPS
[m3vpss ]
[m3vpss ]
[m3vpss ] CH | In Recv In Reject In Process Out Skip Out User Out Latency
[m3vpss ] Num | FPS FPS FPS FPS FPS Skip FPS Min / Max
[m3vpss ] ---------------------------------------------------------------------
[m3vpss ] 0 | 65 0 52 52 13 0 23 54

Is there any limitation of scaler link?

BR
Prashant

0 Brijesh Jadav over 8 years ago in reply to Prashant Andani

TI__Guru**** 398890 points

Hi Prashant,

Scalar link is just a very simple, it does not have high overhead to reduce fps.
i think this is coming from somewhere else.
Please check
1, if you are setting input and output fps to same in scalar link.
2, Scalar link is receiving frames at 60fps rate, ie, previous link is sending frames at correct fps..

Regards,
Brijesh

0 Prashant Andani over 7 years ago in reply to Brijesh Jadav

Intellectual 810 points

Hi Brijesh,

I have Appro reference board and it is giving 60fps with same use-case.

1. Yes, I set same Input and Output fps.

2. Previous link (NSF) is giving 60 fps.

[m3vpss ] 2858323: CAMERA: Fields = 257757 (fps = 90, CPU Load = 63)
[m3vpss ] 2858323: CAMERA: Num Resets = 0 (Avg 0 ms per reset)
[m3vpss ]
[m3vpss ] *** [Scalar0   ] SCLR Statistics ***
[m3vpss ]
[m3vpss ] Elasped Time           : 19 secs
[m3vpss ] Total Fields Processed : 142879
[m3vpss ] Total Fields FPS       : 92 FPS
[m3vpss ]
[m3vpss ]
[m3vpss ] CH | In Recv In Reject In Process Out Skip Out User Out Latency
[m3vpss ] Num | FPS     FPS       FPS        FPS    FPS   Skip FPS Min / Max
[m3vpss ] ---------------------------------------------------------------------
[m3vpss ]    0 |      62         0         52 52       10        0 24 52
[m3vpss ]
[m3vpss ]
[m3vpss ] *** [NSF0] NSF Statistics ***
[m3vpss ]
[m3vpss ] Elasped Time           : 19 secs
[m3vpss ] Total Fields Processed : 171838
[m3vpss ] Total Fields FPS       : 68 FPS
[m3vpss ]
[m3vpss ]
[m3vpss ] CH | In Recv In Reject In Process Out User Out Latency
[m3vpss ] Num | FPS     FPS       FPS        FPS Skip FPS Min / Max
[m3vpss ] ---------------------------------------------------------
[m3vpss ]    0 |      62         0         62 62        0   0 / 19

Regards,

Prashant

0 Brijesh Jadav over 7 years ago in reply to Prashant Andani

TI__Guru**** 398890 points

Hi Prashant,

What is scalar input and output frame size?

Rgds,
Brijesh

0 Prashant Andani over 7 years ago in reply to Brijesh Jadav

Intellectual 810 points

Hi Brijesh,

Input and output to scalar link is 1080p YUV420. So, it is 1920X1080X1.5 = 3110400.

Regards,
Prashant

0 Brijesh Jadav over 7 years ago in reply to Prashant Andani

TI__Guru**** 398890 points

so it is around 124MP, should be able to do.. can you try increasing number of output buffers..
please check if lazy loading is enabled, disable it if it is enabled..

Rgds,
Brijesh

0 Prashant Andani over 7 years ago in reply to Brijesh Jadav

Intellectual 810 points

Hi Brijesh,

It was issue of Encoder link.
Encoder link is not capable to encode H264 1080p@60 with 4 Ring buffer pool.
So i use 6 Ring buffer pool and it works for me.

#define UTILS_BITBUF_MAX_ALLOC_POOLS (6) //(4)

Regards,
Prashant.

0 Brijesh Jadav over 7 years ago in reply to Prashant Andani

TI__Guru**** 398890 points

Great, please close the thread..

Rgds,

Brijesh

0 Prashant Andani over 7 years ago in reply to Brijesh Jadav

Intellectual 810 points

Hi Brijesh,

Now I am getting frame drop in Encoder Link.

Case 1: Second stream with 30fps

[m3video] ==================== EncLink_PrintDetails ====================

[m3video] 298753: HDVICP-ID:0

[m3video] totalAcquire2wait in msec: 7907

[m3video] totalWait2Isr in msec: 274477

[m3video] totalIsr2Done in msec: 719

[m3video] totalWait2Done in msec: 275196

[m3video] totalDone2Release in msec: 0

[m3video] totalAcquire2Release in msec: 286030

[m3video] totalAcq2acqDelay in msec: 7066

[m3video] totalElapsedTime in msec: 293096

[m3video] numAccessCnt: 26408

[m3video] IVA-FPS : 90

[m3video]

[m3video] *** ENCODE Statistics ***

[m3video]

[m3video] Elasped Time : 18 secs

[m3video]

[m3video] CH | In Recv In Skip In User Out Latency

[m3video] Num | FPS FPS Skip FPS FPS Min / Max

[m3video] --------------------------------------------

[m3video] 0 | 60 2 0 56.89 126 / 185

[m3video] 1 | 31 0 0 30.78 60 / 127

[m3video] 2 | 62 0 56 5.22 89 / 141

[m3video]

[m3video] Multi Channel Encode Average Submit Batch Size

[m3video] Max Submit Batch Size : 24

[m3video] IVAHD_0 Average Batch Size : 1

[m3video] IVAHD_0 Max achieved Batch Size : 1

[m3video] ==============================================================

case 2: if i reduce output fps of second stream to 10 fps then i will get 60fps.

[m3video] ==================== EncLink_PrintDetails ====================

[m3video] 738610: HDVICP-ID:0

[m3video] totalAcquire2wait in msec: 19639

[m3video] totalWait2Isr in msec: 686827

[m3video] totalIsr2Done in msec: 1795

[m3video] totalWait2Done in msec: 688622

[m3video] totalDone2Release in msec: 0

[m3video] totalAcquire2Release in msec: 715491

[m3video] totalAcq2acqDelay in msec: 17460

[m3video] totalElapsedTime in msec: 732951

[m3video] numAccessCnt: 65560

[m3video] IVA-FPS : 89

[m3video]

[m3video] *** ENCODE Statistics ***

[m3video]

[m3video] Elasped Time : 18 secs

[m3video]

[m3video] CH | In Recv In Skip In User Out Latency

[m3video] Num | FPS FPS Skip FPS FPS Min / Max

[m3video] --------------------------------------------

[m3video] 0 | 62 0 1 61.22 87 / 130

[m3video] 1 | 31 0 20 10.39 13 / 76

[m3video] 2 | 62 0 57 5.22 44 / 91

[m3video]

[m3video] Multi Channel Encode Average Submit Batch Size

[m3video] Max Submit Batch Size : 24

[m3video] IVAHD_0 Average Batch Size : 1

[m3video] IVAHD_0 Max achieved Batch Size : 1

[m3video] ==============================================================

Is there any limitation of encoder link that it can not process 1080p@60 and 480p@30 fps H264 stream?

Regards,

Prashant

0 Prashant Andani over 7 years ago in reply to Prashant Andani

Intellectual 810 points

Hi Brijesh,

Waiting for your reply.

Thanks,

Prashant

0 Brijesh Jadav over 7 years ago in reply to Prashant Andani

TI__Guru**** 398890 points

Sorry, I can help in hdvpss, not in encoder. Please contact your local TI FAE..

Rgds,
Brijesh

0 Margarita Gashova over 7 years ago in reply to Prashant Andani

TI__Guru* 80190 points

Hello,

Prashant Andani said:
Is there any limitation of encoder link that it can not process 1080p@60 and 480p@30 fps H264 stream?

DM8148 has one HDVICP module. HDVICP module is scalable in terms of resolutions and channels supported, so one HDVICP2 could support 1x 1080p60 or 2x 1080p30 or or 4x 720p30.

You could try 1080p30 and 480p30 at the same time in this case you should not observe a drop.

BR
Margarita

0 Gajanan Maske over 7 years ago in reply to Brijesh Jadav

Expert 2285 points

Hi Prashant,

You are using 3 channel for encoding. The real time performance possible based on your configurations & depends on Resolution, Bit-rate, EncodingModes,etc

Can you provide the complete input configuration parameters for the 3 channels??

1080p@60fps H264, 480p@30fps H264and 1080p@5fps MJPEG looks not possible straight away.

Regards

Gajanan

0 Prashant Andani over 7 years ago in reply to Margarita Gashova

Intellectual 810 points

Hi Margarita,

You are right.
it is working fine with 1080p30 and 480p30 but getting frame drop with 1080p60 and 480p30.

Regards,
Prashant

0 Prashant Andani over 7 years ago in reply to Gajanan Maske

Intellectual 810 points

Hi Gajanan,

Below is my Encoder Init Parameters.

/* Video Encoder Link params */

MULTICH_INIT_STRUCT(EncLink_CreateParams, encPrm);

{

EncLink_ChCreateParams *pLinkChPrm;

EncLink_ChDynamicParams *pLinkDynPrm;

VENC_CHN_DYNAMIC_PARAM_S *pDynPrm;

VENC_CHN_PARAMS_S *pChPrm;

for (i = 0; i < VENC_PRIMARY_CHANNELS; i++)

{

pLinkChPrm = &encPrm.chCreateParams[i];

pLinkDynPrm = &pLinkChPrm->defaultDynamicParams;

pChPrm = &gVencModuleContext.vencConfig.encChannelParams[i];

pDynPrm = &pChPrm->dynamicParam;

pLinkChPrm->format = IVIDEO_H264HP;

pLinkChPrm->profile = VENC_CHN_DEFAULT_PROFILE;

pLinkChPrm->dataLayout = IVIDEO_PROGRESSIVE;

pLinkChPrm->fieldMergeEncodeEnable = FALSE;

pLinkChPrm->enableAnalyticinfo = pChPrm->enableAnalyticinfo;

pLinkChPrm->maxBitRate = pChPrm->maxBitRate;

pLinkChPrm->encodingPreset = pChPrm->encodingPreset;

pLinkChPrm->rateControlPreset = IVIDEO_USER_DEFINED; //pChPrm->rcType;

pLinkChPrm->enableHighSpeed = FALSE;

pLinkChPrm->enableWaterMarking = pChPrm->enableWaterMarking;

pLinkChPrm->StreamPreset = 0;

pLinkDynPrm->intraFrameInterval = pDynPrm->intraFrameInterval;

pLinkDynPrm->targetBitRate = pDynPrm->targetBitRate;

pLinkDynPrm->interFrameInterval = 1;

pLinkDynPrm->mvAccuracy = IVIDENC2_MOTIONVECTOR_QUARTERPEL;

pLinkDynPrm->inputFrameRate = pDynPrm->inputFrameRate;

pLinkDynPrm->rcAlg = pDynPrm->rcAlg;

pLinkDynPrm->qpMin = pDynPrm->qpMin;

pLinkDynPrm->qpMax = pDynPrm->qpMax;

pLinkDynPrm->qpInit = pDynPrm->qpInit;

pLinkDynPrm->vbrDuration = pDynPrm->vbrDuration;

pLinkDynPrm->vbrSensitivity = pDynPrm->vbrSensitivity;

encPrm.numBufPerCh[i] = 4;

gVencModuleContext.encFormat[i] = pLinkChPrm->format;

}

/* Video Encoder Framerate */

encPrm.chCreateParams[0].defaultDynamicParams.inputFrameRate = 60; // ENC_LINK_DEFAULT_ALGPARAMS_INPUTFRAMERATE;

encPrm.chCreateParams[1].defaultDynamicParams.inputFrameRate = 30; // ENC_LINK_DEFAULT_ALGPARAMS_INPUTFRAMERATE;

encPrm.vsEnable = g_x_UI_mcfw_config.m_b_vsEnable;

encPrm.isVaUseCase = 0;

for (i = VENC_PRIMARY_CHANNELS; i < (VENC_CHN_MAX - 1); i++)

{

encPrm.chCreateParams[i].format = IVIDEO_MJPEG;

encPrm.chCreateParams[i].profile = 0;

encPrm.chCreateParams[i].dataLayout = IVIDEO_PROGRESSIVE;

encPrm.chCreateParams[i].fieldMergeEncodeEnable = FALSE;

encPrm.chCreateParams[i].defaultDynamicParams.intraFrameInterval = 0;

encPrm.chCreateParams[i].encodingPreset = 0;

encPrm.chCreateParams[i].enableAnalyticinfo = 0;

encPrm.chCreateParams[i].enableWaterMarking = 0;

encPrm.chCreateParams[i].defaultDynamicParams.inputFrameRate = 60;

encPrm.chCreateParams[i].rateControlPreset = 0;

encPrm.chCreateParams[i].defaultDynamicParams.targetBitRate = 100 * 1000;

encPrm.chCreateParams[i].defaultDynamicParams.interFrameInterval = 0;

encPrm.chCreateParams[i].defaultDynamicParams.mvAccuracy = 0;

gVencModuleContext.encFormat[i] = encPrm.chCreateParams[i].format;

encPrm.numBufPerCh[i] = 4;

}

encPrm.inQueParams.prevLinkId = ipcInVideoId;

encPrm.inQueParams.prevLinkQueId = 0;

encPrm.outQueParams.nextLink = gVencModuleContext.ipcBitsOutRTOSId;

Input Parameter of Encoder.

1). 1st Channel H264

Frame Rate: 60

BitRate: 8M

2). 2nd Channel H264

Frame Rate: 30

BitRate: 2M

3). 3rd Channel MJPEG

Frame Rate: 5

Regards,

Prashant

0 Gajanan Maske over 7 years ago in reply to Prashant Andani

Expert 2285 points

Hi Prashant,

Try with below settings for H.264 Encoder settings -
1 - .encodingPreset == XDM_USER_DEFINED, enableHighSpeed = TRUE

((staticParams->videnc2Params.encodingPreset == XDM_USER_DEFINED) &&
(algCreateParams->enableHighSpeed == TRUE))

OR
2 - encodingPreset = XDM_HIGH_SPEED

With the above changes, the encoder performance will increase.

Regards
Gajanan

0 Prashant Andani over 7 years ago in reply to Gajanan Maske

Intellectual 810 points

Hi Gajanan,

Thanks for your quick response.

I tried both options but getting same issue.

[m3video] ==================== EncLink_PrintDetails ====================
[m3video] 474907: HDVICP-ID:0
[m3video] totalAcquire2wait in msec: 10561
[m3video] totalWait2Isr in msec: 440749
[m3video] totalIsr2Done in msec: 1090
[m3video] totalWait2Done in msec: 441839
[m3video] totalDone2Release in msec: 0
[m3video] totalAcquire2Release in msec: 457019
[m3video] totalAcq2acqDelay in msec: 10588
[m3video] totalElapsedTime in msec: 467591
[m3video] numAccessCnt: 41734
[m3video] IVA-FPS : 89
[m3video]
[m3video] *** ENCODE Statistics ***
[m3video]
[m3video] Elasped Time : 17 secs
[m3video]
[m3video]
[m3video] CH | In Recv In Skip In User Out Latency
[m3video] Num | FPS FPS Skip FPS FPS Min / Max
[m3video] --------------------------------------------
[m3video] 0 | 61.29 3 1 57.41 121 / 179
[m3video] 1 | 31.65 0 0 31.24 65 / 133
[m3video] 2 | 63.35 0 57 5.29 98 / 149
[m3video]
[m3video] ==============================================================

Regards,
Prashant

0 Gajanan Maske over 7 years ago in reply to Prashant Andani

Expert 2285 points

Hi Prashant,

What are the frequencies of HDVICP2, ARM, HDVPSS,ISS,DDR. If you are operatings this cores at full speed then there is no scope of any more improvement.
And also had you make sure that high-speed is enabled & really get into effect ??

Regards
Gajanan

0 Prashant Andani over 7 years ago in reply to Gajanan Maske

Intellectual 810 points

Hi Gajanan,

I am using TMS320DM8127SCYED2 Processor chip
ARM clk: 1000MHz
DDR clk: 533MHz
L3 clk: 220MHz
HDVICP2 clk: 450MHz
ISS clk: 400MHz

Yes, I put debug print in below if condition.
((staticParams->videnc2Params.encodingPreset == XDM_USER_DEFINED) &&
(algCreateParams->enableHighSpeed == TRUE))

0 Gajanan Maske over 7 years ago in reply to Prashant Andani

Expert 2285 points

Hi Prashant,

You are using the all the cores with maximum possibility. So I don't see any scope of further improvement. Modify your use-case according to the performance achieved.

Regards
Gajanan

0 Prashant Andani over 7 years ago in reply to Gajanan Maske

Intellectual 810 points

Hi Gajanan,

My Observation

case 1) If i Set 1080p@60fps H264, 480p@30fps H264 and 1080p@5fps MJPEG, it gives 56-57 fps for first channel.

case 2) If i Set 1080p@60fps H264, 480p@10fps H264 and 1080p@5fps MJPEG, it gives 60 fps for first channel.

case 3) If i Set 1080p@60fps MJPEG, 480p@30fps H264 and 1080p@5fps MJPEG, it gives 60 fps for first channel.

Is there any limitation of HDVICP2 that it can not process upto total 70FPS as per my clock frequencies?

Regards,

Prashant

0 Gajanan Maske over 7 years ago in reply to Prashant Andani

Expert 2285 points

Hi Prashant,

If you go through the datasheet, TI safe claims of HDVICP2 supports for 1080p@60fps real time.

What your use case achieving is more than that.

H.264 Codec is more complex than the MJPEG Codec. H.264 Codec provides better compression or Quality compared to MJPEG.

So due to less complexity, performance of MJPEG is better than H.264.

Regards

Gajanan

0 Margarita Gashova over 7 years ago in reply to Prashant Andani

TI__Guru* 80190 points

Hello,

Please check my post for the limitation of DM8148.

DM8168 has 3x HDVICP modules.

Margarita

Processors

Processors forum

FPS drop in Scalar link