SK-AM62P-LP: Slow Cameras Updates

Sergio Rossi

Part Number: SK-AM62P-LP
Other Parts Discussed in Thread: AM62P

Tool/software:

Hi,

We've implemented a 4 cameras decoder driver but when we run our pipeline we get 100% in each core, as shown bellow

our current pipeline is shown bellow

GST_DEBUG=3 gst-launch-1.0 -v \
v4l2src device=/dev/video2 ! queue max-size-buffers=8 max-size-bytes=0 max-size-time=0 ! \
video/x-raw,format=UYVY,width=1280,height=720,framerate=10/1 ! \
videoscale ! video/x-raw,format=UYVY,width=640,height=360 ! queue ! compositor.sink_0 \
v4l2src device=/dev/video3 ! queue max-size-buffers=8 max-size-bytes=0 max-size-time=0 ! \
video/x-raw,format=UYVY,width=1280,height=720,framerate=10/1 ! \
videoscale ! video/x-raw,format=UYVY,width=640,height=360 ! queue ! compositor.sink_1 \
v4l2src device=/dev/video4 ! queue max-size-buffers=8 max-size-bytes=0 max-size-time=0 ! \
video/x-raw,format=UYVY,width=1280,height=720,framerate=10/1 ! \
videoscale ! video/x-raw,format=UYVY,width=640,height=360 ! queue ! compositor.sink_2 \
v4l2src device=/dev/video5 ! queue max-size-buffers=8 max-size-bytes=0 max-size-time=0 ! \
video/x-raw,format=UYVY,width=1280,height=720,framerate=10/1 ! \
videoscale ! video/x-raw,format=UYVY,width=640,height=360 ! queue ! compositor.sink_3 \
compositor name=compositor \
sink_0::xpos=0   sink_0::ypos=0 \
sink_1::xpos=640 sink_1::ypos=0 \
sink_2::xpos=0   sink_2::ypos=360 \
sink_3::xpos=640 sink_3::ypos=360 ! \
queue ! video/x-raw,width=1280,height=720,framerate=120/1 ! queue ! kmssink

is there anything that we can do to improve it?

1 month ago

0 Jianzhong Xu 1 month ago

TI__Guru 52335 points

Hi Sergio,

First of all, are you getting correct camera data?

If that's the case, I think the "videoscale" plugin is probably the most CPU consuming thing. Please optimize your pipeline.

Also, please try something like below:

gst-launch-1.0 \
v4l2src device=/dev/video-ov5640-cam0 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720, format=YUY2 ! ticolorconvert ! queue ! mosaic.sink_0 \
v4l2src device=/dev/video-ov5640-cam1 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720, format=YUY2 ! ticolorconvert ! queue ! mosaic.sink_1 \
v4l2src device=/dev/video-ov5640-cam2 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720, format=YUY2 ! ticolorconvert ! queue ! mosaic.sink_2 \
v4l2src device=/dev/video-ov5640-cam3 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720, format=YUY2 ! ticolorconvert ! queue ! mosaic.sink_3 \
timosaic name=mosaic \
sink_0::startx=300 sink_0::starty=0 sink_0::width=640 sink_0::height=480 \
sink_1::startx=980 sink_1::starty=0 sink_1::width=640 sink_1::height=480  \
sink_2::startx=300 sink_2::starty=500 sink_2::width=640 sink_2::height=480 \
sink_3::startx=980 sink_3::starty=500 sink_3::width=640 sink_3::height=480 ! \
video/x-raw, width=1920, height=1080, format=NV12 ! queue ! kmssink driver-name=tidss sync=false force-modesetting=true

Regards,

Jianzhong

0 Sergio Rossi 1 month ago in reply to Jianzhong Xu

Prodigy 245 points

Jianzhong Xu said:
First of all, are you getting correct camera data?

yes, but we sense a little delay on the image

We made some changes on the pipeline as you mentioned

gst-launch-1.0 \
v4l2src device=/dev/video2 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720,framerate=30/1, format=UYVY ! ticolorconvert ! queue ! mosaic.sink_0 \
v4l2src device=/dev/video3 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720,framerate=30/1, format=UYVY ! ticolorconvert ! queue ! mosaic.sink_1 \
v4l2src device=/dev/video4 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720,framerate=30/1 ,format=UYVY ! ticolorconvert ! queue ! mosaic.sink_2 \
v4l2src device=/dev/video5 ! queue max-size-buffers=1 leaky=2 ! video/x-raw, width=1280,height=720,framerate=30/1 ,format=UYVY ! ticolorconvert ! queue ! mosaic.sink_3 \
timosaic name=mosaic \
sink_0::startx=0 sink_0::starty=0 sink_0::width=640 sink_0::height=360 \
sink_1::startx=640 sink_1::starty=0 sink_1::width=640 sink_1::height=360  \
sink_2::startx=0 sink_2::starty=360 sink_2::width=640 sink_2::height=360 \
sink_3::startx=640 sink_3::starty=360 sink_3::width=640 sink_3::height=360 ! \
video/x-raw, width=1280, height=720, framerate=60/1, format=NV12 ! queue ! kmssink driver-name=tidss sync=false force-modesetting=true

but we still got 100% in each core

0 Sergio Rossi 1 month ago in reply to Jianzhong Xu

Prodigy 245 points

I was running more tests and I got this results

if I run only the camera on "dev/video2" with this pipeline " gst-launch-1.0 -v v4l2src device=/dev/video3 ! video/x-raw,width=1280,height=720,format=UYVY ! kmssink " my output is bellow

this fault didin't appear on single "dev/video3" and single "dev/video4".

but when I ran it in mosaic mode this error doesn't appear

+1 Jianzhong Xu 1 month ago in reply to Sergio Rossi

TI__Guru 52335 points

Hello Sergio,

Does you dev/video2 correspond to 30102000.ticsi2rx context 0? If that's the case, this is a known issue with context 0 when there are multiple cameras connected.

Please try to skip context 0 when you set up the routes using the script I shared earlier.

Regards,

Jianzhong

0 Sergio Rossi 1 month ago in reply to Jianzhong Xu

Prodigy 245 points

Hello Jianzhong,

Jianzhong Xu said:
Does you dev/video2 correspond to 30102000.ticsi2rx context 0?

yes

we made the following changes on the script

insmod /lib/modules/6.6.32-ti-01287-g6de6e418c80e-dirty/kernel/drivers/media/platform/ti/nvp6324/nvp6324.ko 

# Set CDNS CSI Bridge
media-ctl -R '"cdns_csi2rx.30101000.csi-bridge" [0/1 -> 1/0 [1], 0/2 -> 1/1 [1], 0/3 -> 1/2 [1], 0/4 -> 1/3 [1]]'
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/1 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/2 [fmt:UYVY8_1X16/1280x720 field:none]'
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/3 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/4 [fmt:UYVY8_1X16/1280x720 field:none]'

# Set j721e-csi2rx routes
media-ctl -R '"30102000.ticsi2rx" [0/1 -> 1/0 [1], 0/2 -> 2/0 [1], 0/3 -> 3/0 [1], 0/4 -> 4/0 [1]]'
media-ctl -V '"30102000.ticsi2rx":0/1 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"30102000.ticsi2rx":0/2 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"30102000.ticsi2rx":0/3 [fmt:UYVY8_1X16/1280x720 field:none]'
media-ctl -V '"30102000.ticsi2rx":0/4 [fmt:UYVY8_1X16/1280x720 field:none]'

# Set video node formats
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 1" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 2" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 3" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 4" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 5" -v width=1280,height=720,pixelformat=UYVY

but our dev/video2 remains in context 0

root@am62pxx-evm:~# media-ctl -p
Media controller API version 6.6.32

Media device information
------------------------
driver          j721e-csi2rx
model           TI-CSI2RX
serial          
bus info        platform:30102000.ticsi2rx
hw revision     0x1
driver version  6.6.32

Device topology
- entity 1: 30102000.ticsi2rx (7 pads, 7 links, 4 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	routes:
		0/1 -> 1/0 [ACTIVE]
		0/2 -> 2/0 [ACTIVE]
		0/3 -> 3/0 [ACTIVE]
		0/4 -> 4/0 [ACTIVE]
	pad0: Sink
		[stream:1 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:2 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:3 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:4 fmt:UYVY8_1X16/1280x720 field:none]
		<- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[stream:0 fmt:UYVY8_1X16/1280x720 field:none]
		-> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
	pad2: Source
		[stream:0 fmt:UYVY8_1X16/1280x720 field:none]
		-> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
	pad3: Source
		[stream:0 fmt:UYVY8_1X16/1280x720 field:none]
		-> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
	pad4: Source
		[stream:0 fmt:UYVY8_1X16/1280x720 field:none]
		-> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
	pad5: Source
		-> "30102000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
	pad6: Source
		-> "30102000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]

- entity 9: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links, 4 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
	routes:
		0/1 -> 1/0 [ACTIVE]
		0/2 -> 1/1 [ACTIVE]
		0/3 -> 1/2 [ACTIVE]
		0/4 -> 1/3 [ACTIVE]
	pad0: Sink
		[stream:1 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:2 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:3 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:4 fmt:UYVY8_1X16/1280x720 field:none]
		<- "nvp6324_mipi 4-0033":0 [ENABLED,IMMUTABLE]
	pad1: Source
		[stream:0 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:1 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:2 fmt:UYVY8_1X16/1280x720 field:none]
		[stream:3 fmt:UYVY8_1X16/1280x720 field:none]
		-> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
	pad2: Source
	pad3: Source
	pad4: Source

- entity 15: nvp6324_mipi 4-0033 (1 pad, 1 link, 4 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev2
	routes:
		0/0 -> 0/0 [ACTIVE]
		0/0 -> 0/1 [ACTIVE]
		0/0 -> 0/2 [ACTIVE]
		0/0 -> 0/3 [ACTIVE]
	pad0: Source
		[stream:0 fmt:UYVY8_1X16/1280x720@1/30 field:none colorspace:srgb]
		[stream:1 fmt:UYVY8_1X16/1280x720@1/30 field:none colorspace:srgb]
		[stream:2 fmt:UYVY8_1X16/1280x720@1/30 field:none colorspace:srgb]
		[stream:3 fmt:UYVY8_1X16/1280x720@1/30 field:none colorspace:srgb]
		-> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]

- entity 21: 30102000.ticsi2rx context 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Sink
		<- "30102000.ticsi2rx":1 [ENABLED,IMMUTABLE]

- entity 27: 30102000.ticsi2rx context 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Sink
		<- "30102000.ticsi2rx":2 [ENABLED,IMMUTABLE]

- entity 33: 30102000.ticsi2rx context 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Sink
		<- "30102000.ticsi2rx":3 [ENABLED,IMMUTABLE]

- entity 39: 30102000.ticsi2rx context 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Sink
		<- "30102000.ticsi2rx":4 [ENABLED,IMMUTABLE]

- entity 45: 30102000.ticsi2rx context 4 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Sink
		<- "30102000.ticsi2rx":5 [ENABLED,IMMUTABLE]

- entity 51: 30102000.ticsi2rx context 5 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video7
	pad0: Sink
		<- "30102000.ticsi2rx":6 [ENABLED,IMMUTABLE]

Is there something that we are missing?

And about the problem with 100% of CPU usage with the mosaic, we discovery that our problem is due to video scaling, is there a way to use hardware acceleration with the pipeline?

Regards,
Sergio

0 Jianzhong Xu 1 month ago in reply to Sergio Rossi

TI__Guru 52335 points

Sergio Rossi said:
Is there something that we are missing?

Please refer to this script: https://github.com/TexasInstruments/edgeai-gst-apps/blob/main/scripts/setup_cameras_v3link.sh. Follow setup_imx219() and set up for your sensors.

Sergio Rossi said:
And about the problem with 100% of CPU usage with the mosaic, we discovery that our problem is due to video scaling, is there a way to use hardware acceleration with the pipeline?

You should use the GPU on AM62P for displaying the images.

Regards,

Jianzhong

0 Sergio Rossi 1 month ago in reply to Jianzhong Xu

Prodigy 245 points

Jianzhong Xu said:
You should use the GPU on AM62P for displaying the images.

How do I do that?

+1 Sergio Rossi 1 month ago in reply to Jianzhong Xu

Prodigy 245 points

Jianzhong Xu said:
Please refer to this script: https://github.com/TexasInstruments/edgeai-gst-apps/blob/main/scripts/setup_cameras_v3link.sh. Follow setup_imx219() and set up for your sensors.

Thank you, that script helped us to reroute our sensors, now we are using this commands to start our driver

#carrega o driver:
insmod /lib/modules/6.6.32-ti-01287-g6de6e418c80e-dirty/kernel/drivers/media/platform/ti/nvp6324/nvp6324.ko 

# Set CDNS CSI Bridge
media-ctl -R '"cdns_csi2rx.30101000.csi-bridge" [0/0 -> 1/0 [1], 0/1 -> 1/1 [1], 0/2 -> 1/2 [1], 0/3 -> 1/3 [1]]'
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/0 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/1 [fmt:UYVY8_1X16/1280x720 field:none]'
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/2 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/3 [fmt:UYVY8_1X16/1280x720 field:none]'

# Set video node formats
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 0" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 1" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 2" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 3" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 4" -v width=1280,height=720,pixelformat=UYVY
v4l2-ctl -z platform:30102000.ticsi2rx -d "30102000.ticsi2rx context 5" -v width=1280,height=720,pixelformat=UYVY

#Set j721e-csi2rx routes dev/video3, dev/video4, dev/video5, dev/video6
media-ctl -R '"30102000.ticsi2rx" [0/0 -> 2/0 [1], 0/1 -> 3/0 [1], 0/2 -> 4/0 [1], 0/3 -> 5/0 [1]]'
media-ctl -V '"30102000.ticsi2rx":0/0 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"30102000.ticsi2rx":0/1 [fmt:UYVY8_1X16/1280x720 field:none]' 
media-ctl -V '"30102000.ticsi2rx":0/2 [fmt:UYVY8_1X16/1280x720 field:none]'
media-ctl -V '"30102000.ticsi2rx":0/3 [fmt:UYVY8_1X16/1280x720 field:none]'

we're researching GPU acceleration and we saw that on AM62pXX it was removed dependencies to TIOVX

# Remove edgeai-tiovx-modules dependency for ARM only devices
DEPENDS:remove:am62xx = "edgeai-tiovx-modules"
DEPENDS:remove:am62pxx = "edgeai-tiovx-modules"
RDEPENDS:${PN}:remove:am62xx = "edgeai-tiovx-modules"
RDEPENDS:${PN}:remove:am62pxx = "edgeai-tiovx-modules"
RDEPENDS:${PN}-source:remove:am62xx = "edgeai-tiovx-modules-dev"
RDEPENDS:${PN}-source:remove:am62pxx = "edgeai-tiovx-modules-dev"

COMPATIBLE_MACHINE = "j721e|j721s2|j784s4|j722s|am62axx|am62xx|am62pxx"

Is there any reason for that removal?

Regards,

Sergio

0 Jianzhong Xu 1 month ago in reply to Sergio Rossi

TI__Guru 52335 points

Hi Sergio,

Can you close this thread and open a new one regarding how to use GPU? Your new inquiry will be routed to the GPU expert.

Thank you.

Jianzhong

Processors

Processors forum

SK-AM62P-LP: Slow Cameras Updates