PROCESSOR-SDK-AM68A: Interface two USB cameras on TDA4VL-Q1 EVM

Hi Nikhil, 

Thanks for providing the link.

I am confused about where do I include the patch file?

Is it under "ti-processor-sdk-linux-<version>/board-support/linux-5.10.162+gitAUTOINC+76b3e88d56-g76b3e88d56/patches"? Will it automatically recompile and build the image?

Currently, I am using a prebuilt 'tisdk-edgeai-image-j721s2-evm" image and using mksdboot.sh to build the image on SD card.

Thanks,

Ahmed

Hi,

Please apply the patch here in this folder

ti-processor-sdk-linux-<version>/board-support/linux-5.10.162+gitAUTOINC+76b3e88d56-g76b3e88d56/

Then build the commands mentioned in the documentation in the below path 

ti-processor-sdk-linux-<version>/

After this uvcvideo.ko would be generated in below path.

ti-processor-sdk-linux-<version>/board-support/linux-5.10.162+gitAUTOINC+76b3e88d56-g76b3e88d56//drivers/media/usb/uvc/uvcvideo.ko

Replace only this file in your SD Card which was earlier created using mksdboot.sh.

Regards,

Nikhil

Hi Nikhil, 

Nikhil Dasan said:

ti-processor-sdk-linux-<version>/board-support/linux-5.10.162+gitAUTOINC+76b3e88d56-g76b3e88d56//drivers/media/usb/uvc/uvcvideo.ko

The "uvcvideo.ko" is successfully generated on this path.

Nikhil Dasan said:

Replace only this file in your SD Card which was earlier created using mksdboot.sh.

The previous "uvcvideo.ko" file is located in "/lib/modules/5.10.162-g76b3e88d56/kernel/drivers/media/usb/uvc" and replace with the newly generated one.

After this step, as specified in the documentation, I am supposed to run these two steps:

# Remove uvcvideo module and install modified version using below commands

rmmmod uvcvideo
insmod uvcvideo.ko

# Set the desired BW cap as shown below

echo 1200 >  /sys/module/uvcvideo/parameters/bandwidth_cap

Can you specify from which directory I should run these commands?

Thanks,

Ahmed

Hi Ahmed

rmmmod uvcvideo
insmod uvcvideo.ko

This is run on the SK board. But since you have replaced .ko file in the SD Card, the linux would pick this up upon boot.

To confirm if the driver is present, after the boot please do "lsmod" on the board.

Please check if the uvc driver is present there.

Regards,

Nikhil

The uvcvideo module is removed from "/sys/module" path when I entered this command, 

rmmmod uvcvideo

After moving to ""/lib/modules/5.10.162-g76b3e88d56/kernel/drivers/media/usb/uvc" directory, I ran

insmod uvcvideo.ko

and received this error,

insmod: ERROR: could not insert module /uvcvideo.ko: Unknown symbol in module

For the debugging purpose, the contents of the patch file is like this,

Subject: [PATCH] media: uvcvideo: Add bandwidth_cap module param
From: Andrew Murray <amurray@xxxxxxxxxxxxxxxxxxxx>
Date: Fri, 21 Aug 2020 23:00:38 +0100
Cc: linux-media@xxxxxxxxxxxxxxx

Many UVC devices report larger values for dwMaxPayloadTransferSize than
appear to be required. This results in less bandwidth available for
other devices.

This problem is commonly observed when attempting to stream from multiple
UVC cameras with the host controller returning -ENOSPC and sometimes a
warning (XHCI controllers: "Not enough bandwidth for new device state.").

For uncompressed video, the UVC_QUIRK_FIX_BANDWIDTH works around this issue
by overriding the device provided dwMaxPayloadTransferSize with a
calculation of the actual bandwidth requirements from the requested frame
size and rate. However for compressed video formats it's not practical to
estimate the bandwidth required as the kernel doesn't have enough
information.

Let's provide a pragmatic solution by allowing the user to impose an upper
threshold to the amount of bandwidth each UVC device can reserve. If the
parameter isn't used then no threshold is imposed.

Signed-off-by: Andrew Murray <amurray@xxxxxxxxxxxxxxxxxxxx>
---
 drivers/media/usb/uvc/uvc_driver.c | 3 +++
 drivers/media/usb/uvc/uvc_video.c  | 8 ++++++++
 drivers/media/usb/uvc/uvcvideo.h   | 1 +
 3 files changed, 12 insertions(+)

diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 431d86e1c94b..d5ecac7fc264 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -33,6 +33,7 @@ unsigned int uvc_no_drop_param;
 static unsigned int uvc_quirks_param = -1;
 unsigned int uvc_trace_param;
 unsigned int uvc_timeout_param = UVC_CTRL_STREAMING_TIMEOUT;
+unsigned int uvc_bandwidth_cap_param;
 
 /* ------------------------------------------------------------------------
  * Video formats
@@ -2389,6 +2390,8 @@ module_param_named(trace, uvc_trace_param, uint, S_IRUGO|S_IWUSR);
 MODULE_PARM_DESC(trace, "Trace level bitmask");
 module_param_named(timeout, uvc_timeout_param, uint, S_IRUGO|S_IWUSR);
 MODULE_PARM_DESC(timeout, "Streaming control requests timeout");
+module_param_named(bandwidth_cap, uvc_bandwidth_cap_param, uint, S_IRUGO|S_IWUSR);
+MODULE_PARM_DESC(bandwidth_cap, "Maximum bandwidth per device");
 
 /* ------------------------------------------------------------------------
  * Driver initialization and cleanup
diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index a65d5353a441..74a0dc0614cf 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -196,6 +196,14 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming *stream,
 
 		ctrl->dwMaxPayloadTransferSize = bandwidth;
 	}
+
+	if (uvc_bandwidth_cap_param &&
+	    ctrl->dwMaxPayloadTransferSize > uvc_bandwidth_cap_param) {
+		uvc_trace(UVC_TRACE_VIDEO,
+			"Bandwidth capped from %u to %u B/frame.\n",
+			ctrl->dwMaxPayloadTransferSize, uvc_bandwidth_cap_param);
+		ctrl->dwMaxPayloadTransferSize = uvc_bandwidth_cap_param;
+	}
 }
 
 static size_t uvc_video_ctrl_size(struct uvc_streaming *stream)
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 6ab972c643e3..c7d9220c9a7a 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -718,6 +718,7 @@ extern unsigned int uvc_no_drop_param;
 extern unsigned int uvc_trace_param;
 extern unsigned int uvc_timeout_param;
 extern unsigned int uvc_hw_timestamps_param;
+extern unsigned int uvc_bandwidth_cap_param;
 
 #define uvc_trace(flag, msg...) \
 	do { \
-- 
2.17.1

Thanks,

Ahmed

Hi Nikhil,

Nikhil Dasan said:

To confirm if the driver is present, after the boot please do "lsmod" on the board.

There is no "uvcvideo" module present if I enter "lsmod" on the board.

I also checked on this "/sys/module/" path, and found no "uvcvideo" module.

Previously, there was one, but after patching (replacing the uvcvideo.ko) and rebooting the board, the module is vanished. 

Thanks,

Ahmed

Hi Nikhil,

A QUICK UPDATE!

The uvcvideo module comes up after I interface usb camera with the board. With lsmod, I see the "uvcvideo". Sorry for the confusion.

However, after replacing the "uvcvideo.ko" file and booting up the board, I cannot set the "bandwidth_cap" parameter,

echo 1200 >  /sys/module/uvcvideo/parameters/bandwidth_cap

I manually tried creating the bandwidth_cap paramter in "/sys/module/uvcvideo/parameters/" directory, but received "permission denied" error that cannot be solved with sudo access.

Thanks,

Ahmed

Hi Ahmed,

The patch is a Linux openSource patch.

This was based on the below blog 

Multiple UVC cameras on Linux: an unexpected challenge – The Good Penguin

Could you please check the same?

Regards,

Nikhil

Hi Nikhil,

Yes, I checked that blog.

After I used the patch here mentioned in this documentation, I build the image. After booting up the board, as you mentioned,

Nikhil Dasan said:

rmmmod uvcvideo
insmod uvcvideo.ko

This is run on the SK board. But since you have replaced .ko file in the SD Card, the linux would pick this up upon boot.

Therefore, I proceeded to the next step where I am supposed to set the "bandwidth_cap" parameter under, "/sys/module/uvcvideo/parameters/" path, like this,

echo 1200 >  /sys/module/uvcvideo/parameters/bandwidth_cap

However, even after patching the image, there is no "bandwidth_cap" parameter under, "/sys/module/uvcvideo/parameters/" path.

Multiple UVC cameras on Linux: an unexpected challenge – The Good Penguin does the same thing in the final steps, but I am not able to do so because there is no "bandwidth_cap" parameter.

Thanks,

Ahmed

Hi Ahmed,

Have you connected the camera to the USB Type C port of the EVM?

Do you see bandwidth_cap upon reboot and connecting the camera?

Regards,

Nikhil

Hi Nikhil,

Nikhil Dasan said:

Have you connected the camera to the USB Type C port of the EVM?

Yes, I have connected the two cameras using the USB Type C port on the EVM by using belkin 7 in 1 hub. 

I have also tried the USB 2.0 stacked ports on the EVM.

Right now, I have enabled the USB 2.0 stacked ports using this command, "gpioset gpiochip0 0=1". This is not a concrete solution to enable the USB 2.0 stacked port, and I am discussing this issue to enable the port by patching the "k3-j721s2-common-proc-board.dtb" file.

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1302971/processor-sdk-j721s2-enable-usb-2-0-hub-on-j721excp01evm/4947757#4947757

Nikhil Dasan said:

Do you see bandwidth_cap upon reboot and connecting the camera?

No, I do not see the bandwidth_cap after connecting the cameras.

Thanks,

Ahmed

Hi Nikhil,

I have success in enabling the USB 2.0 stack ports.

Following up on the previous post,

Nikhil Dasan said:

Do you see bandwidth_cap upon reboot and connecting the camera?

Yes, I can see "bandwidth_cap" parameter, and I set it using this command.

echo 1200 >  /sys/module/uvcvideo/parameters/bandwidth_cap

After this, I do not receive "Not enough bandwidth for new device state" error.

However, I still cannot read two cameras. Below is the code I am using to read both cameras,

import cv2

cap = cv2.VideoCapture(2)
cap1 = cv2.VideoCapture(4)

cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

cap1.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap1.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

while True:

	success, frame = cap.read()
	success1, frame1 = cap1.read()	

	if not success:
		break
	if not success1:
		break

	print(frame.shape, frame1.shape)

cap.release()
cap1.release()

Thanks,

Ahmed

Hi Nikhil,

I just confirmed that even though after using the patch,

When I read two cameras simutaneously from USB 2.0 stack ports on the EVM, I still face "VIDIOC_STREAMON: No space left on device" error. This means that there is still bandwidth issue according to this "https://www.thegoodpenguin.co.uk/blog/multiple-uvc-cameras-on-linux/", which means that the patch is not working on the TI EVM. Am I correct?

Thanks,

Ahmed

Hi Nikhil,

Do you have any update on this thread?

Thanks,

Ahmed

Hi Ahmed,

What is the resolution you are using
Based on the resolution you need to calculate the
BW that you set

1200 might not be enough for your resolution, you might
have to increase

BW calculation is explained in this link
Multiple UVC cameras on Linux: an unexpected challenge – The Good Penguin

Regards
Rahul T R

Hi Rahul,

I am using the resolution of 360x640 at 30FPS and 3072 Bytes/frame.

[   50.407207] uvcvideo: Trying format 0x56595559 (YUYV): 640x360.
[   50.407214] uvcvideo: Using default frame interval 33333.3 us (30.0 fps).
[   50.488352] uvcvideo: Trying format 0x56595559 (YUYV): 640x360.
[   50.488360] uvcvideo: Using default frame interval 33333.3 us (30.0 fps).
[   50.568002] uvcvideo: Setting frame interval to 1/30 (333333).
[   50.650322] uvcvideo: Control 0x00980927 not found.
[   50.650728] uvcvideo: Control 0x00980927 not found.
[   50.660868] uvcvideo: Device requested 3072 B/frame bandwidth.
[   50.660874] uvcvideo: Selecting alternate setting 1 (3072 B/frame bandwidth).
[   50.884791] uvcvideo: Allocated 5 URB buffers of 32x3072 bytes each.

According to the calculation provided in Multiple UVC cameras on Linux: an unexpected challenge – The Good Penguin, I need 165.8 Mbps speed.

For that, I think 1200 should work fine. Am I right?

Thanks,

Ahmed