AM5718: ducati encoding causing artifacts on CSI2 mipi capture data

Steven Springman

Part Number: AM5718

Hi,

We have a custom AM5718 board capturing a 1080p30 parallel signal on vin1a interface and a 4k10 signal on the mipi/CSI2 interface. It seems that encoding the 1080p30 input via the ducati gst plugin causes capture artifacts/tearing on the 4k10 capture. All software is based on latest PSDK v06_03_00_106.

1080p30 capture/encode uses the following pipeline based on multimedia examples from the PSDK:

Fullscreen

1
2
3
gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1920,height=1080,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 device=/dev/video1 ! video/x-raw, framerate=30/1 ! queue ! ducatih264enc inter-interval=1 intra-interval=30 max-bitrate=20000 profile=main bitrate=2048 ! \
h264parse ! video/x-h264, alignment=au, framerate=30/1, parsed=true, stream-format=avc ! fakesink sync=true
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1920,height=1080,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 device=/dev/video1 ! video/x-raw, framerate=30/1 ! queue ! ducatih264enc inter-interval=1 intra-interval=30 max-bitrate=20000 profile=main bitrate=2048 ! \
h264parse ! video/x-h264, alignment=au, framerate=30/1, parsed=true, stream-format=avc ! fakesink sync=true

Then, capturing the 4k image using yavta:

Fullscreen

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
~# yavta -c20 -fUYVY -F/tmp/vout_3840x2160_uyvy.yuv -s3840x2160 /dev/video0 
Device /dev/video0 opened.
Device `cal' on `platform:cal-000' is a video output (without mplanes) device.
Video format set: UYVY (59565955) 3840x2160 (stride 7680) field none buffer size 16588800
Video format: UYVY (59565955) 3840x2160 (stride 7680) field none buffer size 16588800
8 buffers requested.
length: 16588800 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x75e66000.
length: 16588800 offset: 16588800 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x74e94000.
length: 16588800 offset: 33177600 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x73ec2000.
length: 16588800 offset: 49766400 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0x72ef0000.
length: 16588800 offset: 66355200 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0x71f1e000.
length: 16588800 offset: 82944000 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0x70f4c000.
length: 16588800 offset: 99532800 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0x6ff7a000.
length: 16588800 offset: 116121600 timestamp type/source: mono/EoF
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

~# yavta -c20 -fUYVY -F/tmp/vout_3840x2160_uyvy.yuv -s3840x2160 /dev/video0 
Device /dev/video0 opened.
Device `cal' on `platform:cal-000' is a video output (without mplanes) device.
Video format set: UYVY (59565955) 3840x2160 (stride 7680) field none buffer size 16588800
Video format: UYVY (59565955) 3840x2160 (stride 7680) field none buffer size 16588800
8 buffers requested.
length: 16588800 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x75e66000.
length: 16588800 offset: 16588800 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x74e94000.
length: 16588800 offset: 33177600 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x73ec2000.
length: 16588800 offset: 49766400 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0x72ef0000.
length: 16588800 offset: 66355200 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0x71f1e000.
length: 16588800 offset: 82944000 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0x70f4c000.
length: 16588800 offset: 99532800 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0x6ff7a000.
length: 16588800 offset: 116121600 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0x6efa8000.
0 (0) [-] none 0 16588800 B 30307.231711 30307.231743 3.768 fps ts mono/EoF
1 (1) [-] none 1 16588800 B 30307.298304 30308.453068 15.017 fps ts mono/EoF
2 (2) [-] none 2 16588800 B 30307.398513 30309.578408 9.979 fps ts mono/EoF
3 (3) [-] none 3 16588800 B 30307.495741 30310.719035 10.285 fps ts mono/EoF
4 (4) [-] none 4 16588800 B 30307.570030 30311.855455 13.461 fps ts mono/EoF
5 (5) [-] none 5 16588800 B 30307.696110 30312.992119 7.931 fps ts mono/EoF
6 (6) [-] none 6 16588800 B 30307.798217 30314.156335 9.794 fps ts mono/EoF
7 (7) [-] none 7 16588800 B 30308.497451 30315.384029 1.430 fps ts mono/EoF
8 (0) [-] none 8 16588800 B 30309.673843 30316.538102 0.850 fps ts mono/EoF
9 (1) [-] none 9 16588800 B 30310.794927 30317.659110 0.892 fps ts mono/EoF
10 (2) [-] none 10 16588800 B 30311.893722 30318.754583 0.910 fps ts mono/EoF
11 (3) [-] none 11 16588800 B 30313.092405 30319.852317 0.834 fps ts mono/EoF
12 (4) [-] none 12 16588800 B 30314.191201 30320.958442 0.910 fps ts mono/EoF
13 (5) [-] none 13 16588800 B 30315.489775 30322.090539 0.770 fps ts mono/EoF
14 (6) [-] none 14 16588800 B 30316.588569 30323.249807 0.910 fps ts mono/EoF
15 (7) [-] none 15 16588800 B 30317.787252 30324.467732 0.834 fps ts mono/EoF
16 (0) [-] none 16 16588800 B 30318.758788 30325.617410 1.029 fps ts mono/EoF
17 (1) [-] none 17 16588800 B 30319.858452 30326.731003 0.909 fps ts mono/EoF
18 (2) [-] none 18 16588800 B 30321.083638 30327.865031 0.816 fps ts mono/EoF
19 (3) [-] none 19 16588800 B 30323.281225 30328.984948 0.455 fps ts mono/EoF
Captured 20 frames in 22.018650 seconds (0.908321 fps, 15067953.162703 B/s).
8 buffers released.

Most frames have either green lines or tearing in captured image:

Capturing with v4l2-ctl or gstreamer produces the same result. When the gst pipeline is off or when it is replaced with a fakesink, 4k capture looks perfect. Gst pipeline with fakesink instead of ducati encoder:

Fullscreen

1
2
gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1920,height=1080,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 device=/dev/video1 ! video/x-raw, framerate=30/1 ! queue ! fakesink sync=true
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1920,height=1080,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 device=/dev/video1 ! video/x-raw, framerate=30/1 ! queue ! fakesink sync=true

We also have hdmi output in our pipeline that tends to make the issue worse:

Fullscreen

1
2
3
4
gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1920,height=1080,framerate=(fraction)30/1' ! \
tee name=src ! queue ! kmssink plane-id=38 async=false sync=false src. ! vpe num-input-buffers=8 device=/dev/video1 ! video/x-raw, framerate=30/1 ! \
queue ! ducatih264enc inter-interval=1 intra-interval=30 max-bitrate=20000 profile=main bitrate=2048 ! h264parse ! \
video/x-h264, alignment=au, framerate=30/1, parsed=true, stream-format=avc ! fakesink sync=true
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1920,height=1080,framerate=(fraction)30/1' ! \
tee name=src ! queue ! kmssink plane-id=38 async=false sync=false src. ! vpe num-input-buffers=8 device=/dev/video1 ! video/x-raw, framerate=30/1 ! \
queue ! ducatih264enc inter-interval=1 intra-interval=30 max-bitrate=20000 profile=main bitrate=2048 ! h264parse ! \
video/x-h264, alignment=au, framerate=30/1, parsed=true, stream-format=avc ! fakesink sync=true

Both the ducati encoder plugin and kmssink allocate dma-buf memory from the DRM driver, while all other buffers are allocated from v4l2. Is there some contention between the DRM and v4l2 buffers? What would cause the artifacts we are seeing?

Thanks,

Steve

over 3 years ago