Performance issue with dual encoding and scaling.

Stephane,

Could you please clarify how do you measure the framerate and at what point in the processing chain you do this?

Also, do you experience the same drop of framerate when you encode only one channel?

In case you do, it would be great to try it without scaling - just capture and encode to see if frame rate drops in this scenario as well.

Thanks,

Michael

The stats are gathered by counting the number of times the FillBufferDone callback of components get called over a period time.

I do it for all components (capture, deint, videnc...). The output of my program looks like this in the mentioned case:

(irrelevant cmd line syntax here fyi: -i defines input port and format, -o means output, -b is bitrate in kbps... rest is TS streaming stuff)

./omax_test -i 0:720p -o 0:480p -b 8000 -i 1:1080p -o 1:720p -b 8000 -h 10.64.1.101 -p 1234 -p 3456

Output

----------------------------------------
VFCC#1   Captured Frames     = 3,504,159
VFCC#1   Captured Frame Rate = 42 FPS
VFCC#2   Captured Frames     = 3,255,820
VFCC#2   Captured Frame Rate = 39 FPS
DEINT#1  Output Frames       = 3,504,155
DEINT#1  Frame Rate          = 42 FPS
DEINT#2  Output Frames       = 3,255,816
DEINT#2  Frame Rate          = 39 FPS
VIDENC#1 Encoded Frames      = 3,504,155
VIDENC#1 Encoded Bytes       = 2,567,940,968
VIDENC#1 Frame Rate          = 42 FPS
VIDENC#1 BitRate             = 5,734 kbps
VIDENC#2 Encoded Frames      = 3,255,815
VIDENC#2 Encoded Bytes       = 2,723,895,633
VIDENC#2 Frame Rate          = 38 FPS
VIDENC#2 BitRate             = 5,118 kbps
----------------------------------------
VFCC#1   Captured Frames     = 3,504,368
VFCC#1   Captured Frame Rate = 41 FPS
VFCC#2   Captured Frames     = 3,256,015
VFCC#2   Captured Frame Rate = 39 FPS
DEINT#1  Output Frames       = 3,504,365
DEINT#1  Frame Rate          = 41 FPS
DEINT#2  Output Frames       = 3,256,012
DEINT#2  Frame Rate          = 39 FPS
VIDENC#1 Encoded Frames      = 3,504,363
VIDENC#1 Encoded Bytes       = 2,571,447,079
VIDENC#1 Frame Rate          = 41 FPS
VIDENC#1 BitRate             = 5,450 kbps
VIDENC#2 Encoded Frames      = 3,256,010
VIDENC#2 Encoded Bytes       = 2,727,148,005
VIDENC#2 Frame Rate          = 39 FPS
VIDENC#2 BitRate             = 5,247 kbps
----------------------------------------
VFCC#1   Captured Frames     = 3,504,579
VFCC#1   Captured Frame Rate = 42 FPS
VFCC#2   Captured Frames     = 3,256,208
VFCC#2   Captured Frame Rate = 38 FPS
DEINT#1  Output Frames       = 3,504,575
DEINT#1  Frame Rate          = 42 FPS
DEINT#2  Output Frames       = 3,256,205
DEINT#2  Frame Rate          = 38 FPS
VIDENC#1 Encoded Frames      = 3,504,574
VIDENC#1 Encoded Bytes       = 2,574,922,043
VIDENC#1 Frame Rate          = 42 FPS
VIDENC#1 BitRate             = 5,669 kbps
VIDENC#2 Encoded Frames      = 3,256,203
VIDENC#2 Encoded Bytes       = 2,730,377,210
VIDENC#2 Frame Rate          = 39 FPS
VIDENC#2 BitRate             = 5,143 kbps
----------------------------------------

Encoding and scaling a single input:

./omax_test -i 0:720p -o 0:480p -b 8000  -h 10.64.1.101 -p 1234

----------------------------------------
VFCC#1   Captured Frames     = 297
VFCC#1   Captured Frame Rate = 60 FPS
DEINT#1  Output Frames       = 296
DEINT#1  Frame Rate          = 60 FPS
VIDENC#1 Encoded Frames      = 296
VIDENC#1 Encoded Bytes       = 4,860,981
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,972 kbps
----------------------------------------
VFCC#1   Captured Frames     = 598
VFCC#1   Captured Frame Rate = 60 FPS
DEINT#1  Output Frames       = 597
DEINT#1  Frame Rate          = 60 FPS
VIDENC#1 Encoded Frames      = 596
VIDENC#1 Encoded Bytes       = 9,869,390
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 8,002 kbps

Works fine and frame rate stays at 60.

Hi,

Both the encoders will be allocated the same HDVICP accelerator by the scheduler running on VIDEO-M3. This is because of the resolutions that are being encoded - 720p and 480p. You can confirm this from the LoggerSMDump output (http://processors.wiki.ti.com/index.php/OMX_Viewing_Media_Controller_Traces) and checking for the string "Allocated IVAHD" in the logs.

Once you confirm this, you can try allocating this to different HDVICPs by creating the VENC component for 1080p resolution and then actually encoding @ 720p and 480p respectively. This way, the 2 VENCs should be assigned different HDVICPs. You can again confirm this from the logs.

If this resolves the issue, we need to see why with the same HDVICP you are unable to do 2 encodes @60fps - 720p & 480p.

Regards,

Anirban

Ok so I do see in the traces that IVAHD 1 is the only encoder getting used.

If I change my test to first encode 1080p natively from one input and then encode 480p (downscaled from other input at 720p):

./omax_test -i 1:1080p -o 1:1080p -b 8000 -i 0:720p -o 0:480p -b 8000 -h 10.64.1.101 -p 1234 -p 3456

The performance is better but not perfect:

----------------------------------------
VFCC#1   Captured Frames     = 586
VFCC#1   Captured Frame Rate = 59 FPS
VFCC#2   Captured Frames     = 520
VFCC#2   Captured Frame Rate = 52 FPS
DEINT#1  Output Frames       = 519
DEINT#1  Frame Rate          = 51 FPS
VIDENC#1 Encoded Frames      = 582
VIDENC#1 Encoded Bytes       = 9,616,657
VIDENC#1 Frame Rate          = 59 FPS
VIDENC#1 BitRate             = 7,845 kbps
VIDENC#2 Encoded Frames      = 517
VIDENC#2 Encoded Bytes       = 8,539,548
VIDENC#2 Frame Rate          = 51 FPS
VIDENC#2 BitRate             = 6,720 kbps
----------------------------------------
VFCC#1   Captured Frames     = 882
VFCC#1   Captured Frame Rate = 59 FPS
VFCC#2   Captured Frames     = 784
VFCC#2   Captured Frame Rate = 52 FPS
DEINT#1  Output Frames       = 780
DEINT#1  Frame Rate          = 52 FPS
VIDENC#1 Encoded Frames      = 881
VIDENC#1 Encoded Bytes       = 14,604,571
VIDENC#1 Frame Rate          = 59 FPS
VIDENC#1 BitRate             = 7,896 kbps
VIDENC#2 Encoded Frames      = 778
VIDENC#2 Encoded Bytes       = 12,891,711
VIDENC#2 Frame Rate          = 52 FPS
VIDENC#2 BitRate             = 7,026 kbps

First encoder chain is very close to 60 but second is around 51-52 frames per sec. I did see in the trace that IVAHD 1 and IVAHD 2 were being used in that case.

Still it doesn't seem to me that this is simply related to the IVAHD capacity, this is what I mean; If I just use one input at 720p and encode it natively and also encode it downscaled through a DEINT at 480p, the performance is fine (frame rates at 59/60...). In the traces only one IVAHD is being used and it seems to be able to cope OK.

./omax_test -i 0:720p -o 0:720p -b 8000 -o 0:480p -b 8000 -h 10.64.1.101 -p 1234 -p 3456

----------------------------------------
VFCC#1   Captured Frames     = 883
VFCC#1   Captured Frame Rate = 59 FPS
DEINT#1  Output Frames       = 881
DEINT#1  Frame Rate          = 59 FPS
VIDENC#1 Encoded Frames      = 881
VIDENC#1 Encoded Bytes       = 14,613,775
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,968 kbps
VIDENC#2 Encoded Frames      = 878
VIDENC#2 Encoded Bytes       = 14,559,872
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,972 kbps
----------------------------------------
VFCC#1   Captured Frames     = 1,177
VFCC#1   Captured Frame Rate = 59 FPS
DEINT#1  Output Frames       = 1,175
DEINT#1  Frame Rate          = 59 FPS
VIDENC#1 Encoded Frames      = 1,175
VIDENC#1 Encoded Bytes       = 19,532,082
VIDENC#1 Frame Rate          = 59 FPS
VIDENC#1 BitRate             = 7,897 kbps
VIDENC#2 Encoded Frames      = 1,173
VIDENC#2 Encoded Bytes       = 19,465,364
VIDENC#2 Frame Rate          = 59 FPS
VIDENC#2 BitRate             = 7,932 kbps

So there seems to be something else at work here.

FYI I will be on business travel for a few days so I won't be able to respond to any further comments before then.

Thanks

Hi,

Yes, I agree that it does not look like a HDVICP issue. But what is strange is that there is an improvement in performance with 2 HDVICPs, although in this experiment you are not scaling on the 1080p encoding path. Can you provide the following information and try out a few more things?

1. Are you using the VC daughter card? How are you capturing 2 inputs?

2. What is the A8 load in the original chain that you have - 1080p capture with 720p encode and 720p capture with 480p encode running simultaneously?

3. How have you handled the second output port of DEI-H and DEI-M? Are you passing a dummy buffer and doing nothing with it? Have you configured this port for colour conversion (420 to 422) and down-scaling?

4. Can you try running this chain with the following modifications and provide the data?

(a) No down-scaling with 2 HDVICPs: 1080p capture with 1080p encode simultaneously with 720p capture with 720p encode

(b) No-down-scaling with 1 HDVICP: 720p capture with 720p encode for both channels

Regards,

Anirban

I think the following answers all your questions:

1). I’m using both component inputs on the expansion IO daughter card.

2). Cpu load is typically around 20% for all cases with little variance.

3). For the DEINT I’m using the second port for the 4:2:0 output to the encoder (OMX_VFPC_OUTPUT_PORT_START_INDEX + 1). The first port is setup for 4:2:2 with the same format as the second port. I just recycle the filled buffers on the first port that I am not really using (but still have to enable or else nothing works).

 4. Can you try running this chain with the following modifications and provide the data?

(a) No down-scaling with 2 HDVICPs: 1080p capture with 1080p encode simultaneously with 720p capture with 720p encode

root@dm816x-evm:/home/ssthilaire# ./omax_test -i 1:1080p -o 1:1080p -b 8000 -i 0:720p -o 0:720p -b 8000 -h 10.64.1.101 -p 1234 -p 3456
Will encode input 1:1920x1080p60 to an output of 1920x1080p60 at 8000 kbps.
Encoded stream will be sent to 10.64.1.101 at port 1234.
Encoded stream will be not be stored to disk.
Will also encode input 0:1280x720p60 to an output of 1280x720p60 at 8000 kbps.
Encoded stream will be sent to 10.64.1.101 at port 3456.
Encoded stream will be not be stored to disk.
Program will run indefinitely until CTRL-C is entered.
Press any key to begin or x to exit.

----------------------------------------
VFCC#1   Captured Frames     = 297
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 297
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 294
VIDENC#1 Encoded Bytes       = 4,895,229
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,719 kbps
VIDENC#2 Encoded Frames      = 295
VIDENC#2 Encoded Bytes       = 4,830,205
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,989 kbps
----------------------------------------
VFCC#1   Captured Frames     = 597
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 597
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 595
VIDENC#1 Encoded Bytes       = 9,851,798
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,940 kbps
VIDENC#2 Encoded Frames      = 595
VIDENC#2 Encoded Bytes       = 9,833,027
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,977 kbps
----------------------------------------
VFCC#1   Captured Frames     = 898
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 898
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 896
VIDENC#1 Encoded Bytes       = 14,856,528
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,840 kbps
VIDENC#2 Encoded Frames      = 896
VIDENC#2 Encoded Bytes       = 14,853,820
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 8,020 kbps
----------------------------------------
VFCC#1   Captured Frames     = 1,198
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 1,197
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 1,196
VIDENC#1 Encoded Bytes       = 19,874,038
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 8,002 kbps
VIDENC#2 Encoded Frames      = 1,196
VIDENC#2 Encoded Bytes       = 19,852,063
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,986 kbps
----------------------------------------
VFCC#1   Captured Frames     = 1,498
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 1,498
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 1,497
VIDENC#1 Encoded Bytes       = 24,890,900
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 8,011 kbps
VIDENC#2 Encoded Frames      = 1,497
VIDENC#2 Encoded Bytes       = 24,867,841
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,969 kbps
----------------------------------------
VFCC#1   Captured Frames     = 1,798
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 1,799
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 1,797
VIDENC#1 Encoded Bytes       = 30,146,967
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 9,227 kbps
VIDENC#2 Encoded Frames      = 1,797
VIDENC#2 Encoded Bytes       = 29,866,356
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,995 kbps

(b) No-down-scaling with 1 HDVICP: 720p capture with 720p encode for both channels

root@dm816x-evm:/home/ssthilaire# ./omax_test -i 1:720p -o 1:720p -b 8000 -i 0:720p -o 0:720p -b 8000 -h 10.64.1.101 -p 1234 -p 3456
Will encode input 1:1280x720p60 to an output of 1280x720p60 at 8000 kbps.
Encoded stream will be sent to 10.64.1.101 at port 1234.
Encoded stream will be not be stored to disk.
Will also encode input 0:1280x720p60 to an output of 1280x720p60 at 8000 kbps.
Encoded stream will be sent to 10.64.1.101 at port 3456.
Encoded stream will be not be stored to disk.
Program will run indefinitely until CTRL-C is entered.
Press any key to begin or x to exit.

----------------------------------------
VFCC#1   Captured Frames     = 299
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 298
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 298
VIDENC#1 Encoded Bytes       = 4,899,991
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,998 kbps
VIDENC#2 Encoded Frames      = 296
VIDENC#2 Encoded Bytes       = 4,880,290
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,926 kbps
----------------------------------------
VFCC#1   Captured Frames     = 599
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 598
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 599
VIDENC#1 Encoded Bytes       = 9,911,349
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,989 kbps
VIDENC#2 Encoded Frames      = 597
VIDENC#2 Encoded Bytes       = 9,877,333
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,975 kbps
----------------------------------------
VFCC#1   Captured Frames     = 900
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 899
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 899
VIDENC#1 Encoded Bytes       = 14,905,133
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 8,124 kbps
VIDENC#2 Encoded Frames      = 897
VIDENC#2 Encoded Bytes       = 14,877,408
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 8,275 kbps
----------------------------------------
VFCC#1   Captured Frames     = 1,201
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 1,199
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 1,200
VIDENC#1 Encoded Bytes       = 19,946,662
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 8,000 kbps
VIDENC#2 Encoded Frames      = 1,197
VIDENC#2 Encoded Bytes       = 19,875,714
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,968 kbps
----------------------------------------
VFCC#1   Captured Frames     = 1,502
VFCC#1   Captured Frame Rate = 60 FPS
VFCC#2   Captured Frames     = 1,500
VFCC#2   Captured Frame Rate = 60 FPS
VIDENC#1 Encoded Frames      = 1,501
VIDENC#1 Encoded Bytes       = 24,918,331
VIDENC#1 Frame Rate          = 60 FPS
VIDENC#1 BitRate             = 7,997 kbps
VIDENC#2 Encoded Frames      = 1,498
VIDENC#2 Encoded Bytes       = 24,917,100
VIDENC#2 Frame Rate          = 60 FPS
VIDENC#2 BitRate             = 7,996 kbps

Basically for both your requested test cases, performance seems fine.

Regards

One more thing. We can record the h.264 output to a file with our program and in cases where the output frame rate is not consistent with the input (when doing dual encoding and scaling) compliance issues will be reported by the Tektronix MTS4EA analyzer.

Here are two cases where we get a lot of compliance issues reported:

Example1: We encode one input video, native 1080p signal, at 1080p on a first OMX VENC encoder, plus a downscaled version of this input signal at 720p on a second OMX VENC encoder, and we record the generated output files to disk. Both encoder components frame rates are near 45 fps. After H.264 analysis they are incorrect with regards to H.264 syntax.

Example 2: We encode one input natively at 1080p and we encode a second signal downscaled to 480p from native 720p60 video signal this also generates H.264 compliance errors on both recorded files.

Hello Stephane,

I am trying to construct the same pipeline as you were for simultaneous dual capture through: VFCC -> VFPC -> VENC components.

My concern is that on one of my card's inputs the data has to be captured as RGB24 and by the manual, the capture VFCC component does not accept RGB24 (it does YUV 422).

Would you remember in which colour format was data that was captured by VFCC in your case (initially, before it possibly got colour-converted by VFCC to feed the DEI component)? Was it RGB24 by an chance?

Thank you,

Przemek

No it was not, we were actually using YCbCr 4:2:2... which is of no help. Sorry.

Steph

Hi,

Thanks for replying anyway.

Przemek