System overload on HRDBuffer size > 300000

Andreas Werner

Other Parts Discussed in Thread: DM385

Hi,

I have a question regarding the HRDBuffer Size.

The H264 Encoder User Guide say that the HRDBufferSize shall be 0.5 * targetBitrate for CBR which is what I Need.

The Video Input Format is 1280x720@60Hz and the Encoded bit rate is 30Mbit/s.
The Video is captured, encoded to h264 and transmitted using sockets to a Decoder.

If I set the HRDBuffersize > 1500000, the socket send function returns with EAGAIN (resource temporarly not available).
I have debugged this error and found that it only occurs if a Frame with a size of > 320000 Bytes is received. If the size of the Frames
are smaller, everything is working correctly.

What I did next is, to just throw away those big Frame to be sure that there is no problem with my transmit function. Throw away means,
not allocating memory in Linux for the frame and to not put the frame im my send Queue.

The result of this test was, that there is still the error message from the socket. (though the frame is actually not send)

So the assumption is that if such a big Frame is received, the System gets "overloaded" and the Interfaces such as the sockets gets blocked for a Long time.

It would be interesting if somebody of you has the same Problems like me. Another Point would be, if there are any limitations with only 256MB of System RAM.

over 9 years ago

0 Andreas Werner over 9 years ago

Intellectual 325 points

If the error occured, there is sometimes also a lockup on the m3 with the following Messages

[m3video] ENC : IVAHDID : 0 ENCLINK:ERROR in EncLink_SubmitBatch.Status[-1]
[m3video] 48749:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[473]::INTERNAL 1
[m3video] ALGPROCESS FAILED
[m3video] Number of Entries in Process List : 1
[m3video] Extended error for entry 0 : 0x10400
[m3video] 48749:WARN
[m3video] ENCLINK:ERROR in Enclink_h264EncodeFrameBatch.Status[-1] for IVAHD_0
[m3video] 48749:WARN
[m3video] ENC : IVAHDID : 0 ENCLINK:ERROR in EncLink_SubmitBatch.Status[-1]
[m3video] 48765:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[473]::INTERNAL 1
[m3video] ALGPROCESS FAILED
[m3video] Number of Entries in Process List : 1
[m3video] Extended error for entry 0 : 0x10400
[m3video] 48765:WARN
[m3video] ENCLINK:ERROR in Enclink_h264EncodeFrameBatch.Status[-1] for IVAHD_0
[m3video] 48765:WARN
[m3video] ENC : IVAHDID : 0 ENCLINK:ERROR in EncLink_SubmitBatch.Status[-1]
[m3video] 48781:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[473]::INTERNAL 1
[m3video] ALGPROCESS FAILED
[m3video] Number of Entries in Process List : 1
[m3video] Extended error for entry 0 : 0x10400
[m3video] 48781:WARN
[m3video] ENCLINK:ERROR in Enclink_h264EncodeFrameBatch.Status[-1] for IVAHD_0
[m3video] 48781:WARN
[m3video] ENC : IVAHDID : 0 ENCLINK:ERROR in EncLink_SubmitBatch.Status[-1]
[m3video] 48797:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[473]::INTERNAL 1
[m3video] ALGPROCESS FAILED
[m3video] Number of Entries in Process List : 1
[m3video] Extended error for entry 0 : 0x10400
[m3video] 48797:WARN
[m3video] ENCLINK:ERROR in Enclink_h264EncodeFrameBatch.Status[-1] for IVAHD_0
[m3video] 48797:WARN
[m3video] ENC : IVAHDID : 0 ENCLINK:ERROR in EncLink_SubmitBatch.Status[-1]
[m3video] 48813:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[473]::INTERNAL 1
[m3video] ALGPROCESS FAILED
[m3video] Number of Entries in Process List : 1
[m3video] Extended error for entry 0 : 0x10400
[m3video] 48813:WARN
[m3video] ENCLINK:ERROR in Enclink_h264EncodeFrameBatch.Status[-1] for IVAHD_0
[m3video] 48813:WARN
[m3video] ENC : IVAHDID : 0 ENCLINK:ERROR in EncLink_SubmitBatch.Status[-1]
[m3video] Unhandled Exception:
[m3video] Exception occurred in ThreadType_Task
[m3video] handle: 0x26cb2db4.
[m3video] stack base: 0x302000.
[m3video] stack size: 0x2000.
[m3video] R0 = 0x269f8f80 R8 = 0x00000000
[m3video] R1 = 0x269fc3c0 R9 = 0x269f8f80
[m3video] R2 = 0x2000ce39 R10 = 0x269fb400
[m3video] R3 = 0x00303ae0 R11 = 0x269f8f80
[m3video] R4 = 0x00000001 R12 = 0x8674c331
[m3video] R5 = 0x86769400 SP(R13) = 0x00303ad8
[m3video] R6 = 0x269fc3c0 LR(R14) = 0x2000cba1
[m3video] R7 = 0xbc4dd480 PC(R15) = 0x86769400
[m3video] PSR = 0x60000000
[m3video] ICSR = 0x0440f803
[m3video] MMFSR = 0x00
[m3video] BFSR = 0x00
[m3video] UFSR = 0x0002
[m3video] HFSR = 0x40000000
[m3video] DFSR = 0x00000000
[m3video] MMAR = 0xe000ed34
[m3video] BFAR = 0xe000ed38
[m3video] AFSR = 0x00000000
[m3video] Terminating Execution...

0 Margarita Gashova over 9 years ago

TI__Guru* 80190 points

Hello,

What is the software release that you are using here?
Are you observe the same problem when you use targetbitrate=8Mbps for example?

BR
Margarita

0 Andreas Werner over 9 years ago in reply to Margarita Gashova

Intellectual 325 points

Hi,
i am currently using the DVR_RDK 4.0.

I have not tested it with a targetbitrate=8Mbps, but i observed the error with 10Mbps.

Do you have an idea what this error could be?

Regards
Andy

0 Andreas Werner over 9 years ago in reply to Margarita Gashova

Intellectual 325 points

I have also tested it with 8Mbps and other Bitrate, i get always the same failure.

Do you have any new about this?

0 Margarita Gashova over 9 years ago in reply to Andreas Werner

TI__Guru* 80190 points

Hello,

I am not aware with DVRRDK.
Your use case is .... encode -> decode ...., right?
If yes, what about if you use only encode? Are you observe the same problem.

You could check this error Error 0x10400
It seems bits 10 and 16 are set.
Bit 10 is insufficient input data.

This error is typically applicable for decoders. This is set when the input data provided is not sufficient to produce of one frame of data. This can be also be set for encoders when the number of valid samples in the input frame is not sufficient to process a frame.

BR
Margarita

0 Andreas Werner over 9 years ago in reply to Margarita Gashova

Intellectual 325 points

Hi,
my use case is as following
capture -> encode -> send via Ethernet

The Decoder is connected via Ethernet to the DM385.

I think the question here is not really related to the DVR_RDK, it is more related to the HRDBuffer size. At the Moment I cannot set the HRDBuffer to 0.5 *targetBitRate, which I Need for CBR. If I set it to this value, I somtimes receive Frames > 300000bytes which leads to this error or to a blocking socket.

Is there any configuration which I Need to set to? Maby the 256MB System RAM is the root cause?

Regards
Andy

0 Gajanan Maske over 9 years ago in reply to Andreas Werner

Expert 2325 points

Hi Andy,

Can you let me know the limit exceeding the >300000bytes is for I/IDR frame. In that case, below settings need to be changed in Encoder.

maxPicSizeRatioI = 120 (default is 640). This will reduce the size of the I/IDR frame.

Regards

Gajanan

0 Andreas Werner over 9 years ago in reply to Gajanan Maske

Intellectual 325 points

Hi Gajanan,
thanks for the reply.

I have checked the Frames and found that most of them are I/IDR Frames. I set the maxPicSizeRatioI = 120 but there are still Frame (not I/IDR) which exeed 300000bytes. I have tried to also set the maxPicSizeRatioP, but there are still such big Frames.

I have also observed that it seems to depend on the intraFrameInterval. If i set this to 60 the big Frames occurs immediately, if I set the intraFrameInterval to 5, it takes more time until I receive such a big Frame.

Regards
Andy

0 Gajanan Maske over 9 years ago in reply to Andreas Werner

Expert 2325 points

Hi Andy,

Try with below 2 experiments to control the framesize. Make sure rateControlParamsPreset = 1 (user defined)-

1 - Allowed range for maxPicSizeRatioI values are 2 to 30 & 33 to 960. Try with minimum value 2.

2 - If still not able to control the size of I/IDR frame, then increase the qpMinI (Range from 51 - 1).

Regards
Gajanan

0 Andreas Werner over 9 years ago in reply to Gajanan Maske

Intellectual 325 points

Hi Gajanan,
i have the following Settings:

rateControlParamsPreset = 1
rateControlPreset = 5
maxPicSizeRatioI = 2
targetBitRate=30000000
intraFrameInterval=60
qpMin=10
qpMax=36

It seems to be that I can control the I-Frames size, it does not exceed 145000bytes. I tried to also set maxPicSizeRationP = 2 to also control the P Frames but there are still a few Frames which exceeds 300000bytes.

Is there another way to control the P-Frames size?

Regards
Andy

0 Gajanan Maske over 9 years ago in reply to Andreas Werner

Expert 2325 points

Hi Andreas,

It would be difficult to maintain the framesize within limits due to High Bit-rate settings(30Mbps). Based on your settings on an best case 125,000 bytes should be used for every frame. But whenever there will be more activity for any frame, the bits consumption will be more for that frame (i.e >300,000 bytes). So I recommend to maintain the MaxPicSizeRatioI & MaxPicSizeRatioIP accordingly. There is scope of incresing the value of MaxPicSizeRatioI (since not excedding the 145000 bytes. Let is go till 250000 so that less budget of bytes need to be used for P frames). This values should be find by experimentation since it depends on the sequence type, vary with Bitrate, resolution. And also configure below parameters -

IF CBR = 0.5 * TargetBitrate = 15000000

IF VBR = 2 * TargetBitrate = 60000000

Regards

Gajanan

0 Andreas Werner over 9 years ago in reply to Gajanan Maske

Intellectual 325 points

Hi Gajanan,

ok i understand this, but why does so Problem occur it the Frame size is to big? What is the root cause? I mean how can I say that it will work for every

Resolution and different Settings if i find a good MaxPicSizeRation Setting?

I Need to Support may resolutions and Bitrate Settings from 1Mbit/s up to 30Mbit/s.

Regards

Andy

0 Andreas Werner over 9 years ago in reply to Gajanan Maske

Intellectual 325 points

Hi,
i have added some more debug prints. Maybe this could help.

The following is currently set in the Encoder, the Input Video is a static Picture.

This Looks like, that the Encoder tries to meet the minBitRate with inserting a filler Frame, but the size of the
filler Frame is about 400000bytes.
Is this the correct behaviour of the Encoder?

Input Video Format: 1920x1080@60
targetBitRate=15000000
maxBitRate=15000000
minBitRate=8000000
intraFrameInterval=60
maxPicSizeRatioI=2
maxPicSizeRatioP=0
rateControlPreset=IVIDEO_USER_DEFINED
rcAlg=IH264_RATECONTROL_PRC_LOW_DELAY

[ 266.389572] GOP Starting...
[ 266.389612] Frame size: 30716
[ 266.421595] Frame size: 28747
[ 266.453644] Frame size: 21736
[ 266.485672] Frame size: 18017
[ 266.517717] Frame size: 17913
[ 266.549765] Frame size: 17600
[ 266.581802] Frame size: 17756
[ 266.613840] Frame size: 17558
[ 266.645885] Frame size: 17805
[ 266.677933] Frame size: 17469
[ 266.709974] Frame size: 17529
[ 266.742031] Frame size: 17490
[ 266.774070] Frame size: 17583
[ 266.806109] Frame size: 17335
[ 266.838153] Frame size: 17461
[ 266.870196] Frame size: 17337
[ 266.902239] Frame size: 17853
[ 266.934282] Frame size: 17447
[ 266.966322] Frame size: 17638
[ 267.014384] Frame size: 17715
[ 267.046424] Frame size: 17867
[ 267.080226] Frame size: 17685
[ 267.112245] Frame size: 17547
[ 267.144280] Frame size: 17522
[ 267.176320] Frame size: 17526
[ 267.208367] Frame size: 17525
[ 267.240414] Frame size: 17525
[ 267.272459] Frame size: 17521
[ 267.304500] Frame size: 17526
____________BigBuf
___________P Frame
[ 267.336799] Frame size: 461052
[ 267.368628] Frame size: 17525
[ 267.416680] Frame size: 17523
[ 267.448733] Frame size: 17524
[ 267.480771] Frame size: 17522
[ 267.512815] Frame size: 17528
[ 267.544867] Frame size: 17521
[ 267.576905] Frame size: 17524
[ 267.608946] Frame size: 17525
[ 267.640996] Frame size: 17525
[ 267.673042] Frame size: 17521
[ 267.705080] Frame size: 17526
[ 267.737121] Frame size: 17522
[ 267.769167] Frame size: 17525
[ 267.817225] Frame size: 17523
[ 267.849268] Frame size: 17524
[ 267.881330] Frame size: 17522
[ 267.913355] Frame size: 17528
[ 267.945397] Frame size: 17521
[ 267.977441] Frame size: 17524
[ 268.009482] Frame size: 17525
[ 268.041524] Frame size: 17525
[ 268.073564] Frame size: 17521
[ 268.105611] Frame size: 17526
[ 268.137651] Frame size: 17522
[ 268.169694] Frame size: 17525
[ 268.217763] Frame size: 17523
[ 268.249803] Frame size: 17524
[ 268.281844] Frame size: 17522
[ 268.313894] Frame size: 17528
____________BigBuf
___________P Frame
[ 268.346193] Frame size: 491807

0 Gajanan Maske over 9 years ago in reply to Andreas Werner

Expert 2325 points

Hi Andreas,

It will be always recommended to not set minBitrate which will add filler bytes to achieve. So if minbitrate is not constraint for you, just set the Minbitrate = 0. Then you will not fact the issue of crossing the framesize above limit & you can revert the Maxpicsizeratio I/P as well.

Regards
Gajanan

0 Andreas Werner over 9 years ago in reply to Gajanan Maske

Intellectual 325 points

Hi Gajanan,

If I understand it correctly, I Need to set the minBitRate because I have a requirement to Support CBR. I Need to have constant Bitrate for moving Pictures as well as for static Pictures.

With this requirement, minBitrate=0 is not an Option for me.

Please correct my if I am wrong.

Regards
Andy

0 Gajanan Maske over 9 years ago in reply to Andreas Werner

Expert 2325 points

Hi Andreas,

I can understand that with CBR, you should have constant bitrate with/without moving pictures. But is it some Hardline/Constraint based on network or Decoder. You can also switch to VBR if you dont have any other constraints.
If I understood correctly, you have used CBR due to higher limit (i.e bytes should not cross > 300000 bytes). But if you don't have problem with lower limit (Minbytes should be X) ,you can use it.
You just name it modified CBR.

Regards
Gajanan

0 Andreas Werner over 9 years ago in reply to Gajanan Maske

Intellectual 325 points

Hi,
the Decoder requires CBR. And CBR is also supported by the Encoder.

I have not switched to CBR due to the 300000bytes issue. This issue needs to be solved, and I Need to understand why there is such a "Magic" 300000byte behaviour.

I have no limitation for the Frame size. The Limitation is currently only given by the Encoder it self, because it behavious really strange if the Frame size exceeds the 300000bytes border. (socket gets blocked for > 1sec).

Regards
Andy

0 Gajanan Maske over 9 years ago in reply to Andreas Werner

Expert 2325 points

Hi Andreas,

As per my understanding, to decoder CBR/VBR should not matter. Its features of Encoder to use VBR/CBR. Can you explain if you set Encoder to VBR, how decoder will be affected ??

Regards
Gajanan

Processors

Processors forum

System overload on HRDBuffer size > 300000