• State TI Thinks Resolved
  • Locked Locked
  • Replies 14 replies
  • Answers 5 answers
  • Subscribers 100 subscribers
  • Views 4248 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM572X: Dual USB RTP streaming with GStreamer

Pallab Sarkar
Prodigy 165 points
Other Parts Discussed in Thread: BEAGLEBOARD-X15, TUSB8041

Hi E2E Community,

I am trying to use two usb camera record and rtp stream parallel  BeagleBoard-x15
Usb cameras used :- logitech c270

I am using PSDK http://www.ti.com/tool/PROCESSOR-SDK-AM57X

---------------------------------------------------------------------------------------------

v4l2-ctl --list-devices

UVC Camera (046d:0825) (usb-xhci-hcd.1.auto-1.1):

/dev/video2

UVC Camera (046d:0825) (usb-xhci-hcd.1.auto-1.2):
/dev/video1

---------------------------------------------------------------------------------------------

Command used 

---------------------------

gst-launch-1.0 -v v4l2src device=/dev/video1 ! videoconvert ! videoscale ! video/x-raw,format=I420,width=800,height=600,framerate=25/1 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.2.15 port=5000

gst-launch-1.0 -v v4l2src device=/dev/video2 ! videoconvert ! videoscale ! video/x-raw,format=I420,width=800,height=600,framerate=25/1 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.2.15 port=5004

and gets the below Error

===========================================================================================================================
root@am57xx-evm:~# gst-launch-1.0 -v v4l2src device=/dev/video2 ! videoconvert ! videoscale ! video/x-raw,format=I420,width=800,height=600,framerate=25/1 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.2.16 port=5000
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)752\,\ height\=\(int\)416\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)1:4:7:1\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = "video/x-raw\,\ width\=\(int\)752\,\ height\=\(int\)416\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:src: caps = "video/x-raw\,\ width\=\(int\)800\,\ height\=\(int\)600\,\ pixel-aspect-ratio\=\(fraction\)141/104\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\,\ width\=\(int\)800\,\ height\=\(int\)600\,\ pixel-aspect-ratio\=\(fraction\)141/104\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstJpegEnc:jpegenc0.GstPad:sink: caps = "video/x-raw\,\ width\=\(int\)800\,\ height\=\(int\)600\,\ pixel-aspect-ratio\=\(fraction\)141/104\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\,\ width\=\(int\)800\,\ height\=\(int\)600\,\ pixel-aspect-ratio\=\(fraction\)141/104\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps = "video/x-raw\,\ width\=\(int\)752\,\ height\=\(int\)416\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)752\,\ height\=\(int\)416\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)1:4:7:1\,\ framerate\=\(fraction\)25/1"
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not read from resource.
Additional debug info:
../../../gst-plugins-good-1.6.3/sys/v4l2/gstv4l2bufferpool.c(1055): gst_v4l2_buffer_pool_poll (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
poll error 1: No space left on device (28)
Execution ended after 0:00:01.546144281
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
==========================================================================================================================

Same scenario works well on other board setup and my ubuntu dev machine !!!!!

Please share me any pointers on how to make it working !!!

Regards

Pallab Sarkar

  • 0
    TI__Guru**** 393215 points
    The software team have been notified. They will respond here.
  • 0 in reply to Biser Gatchev-XID
    TI__Guru* 80190 points

    Hello,

    Could you try to run two pipelines like:

    gst-launch-1.0 -v v4l2src device=/dev/videoX ! 'caps' ! fakesink silent=true

    ?

    Pallab Sarkar said:
    ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not read from resource.
    Additional debug info:
    ../../../gst-plugins-good-1.6.3/sys/v4l2/gstv4l2bufferpool.c(1055): gst_v4l2_buffer_pool_poll (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
    poll error 1: No space left on device (28)


    Since I know the possible reason for this error could be that there is not enough bandwidth on your usb controller to have simultaneous capture.
    You could try to plugin the camera on another controller.


    Hope this helps.

    BR
    Margarita

  • 0 in reply to Margarita Gashova
    Prodigy 165 points

    Hi Margarita,

    Thanks for your inputs , Below is the analysis based on your inputs

    1. Tried adding fakesink in place of udpsink , but it did not helped to resolve the issue.

    2. Regarding trying with usb controller , Beageleboard x15 has 4 usb port ( one single usb , two usb stack together and one esata usb) tried all combinations  but did not helped.

    Also did experiment to reduce the input video resolution of cameras(two check if its really bandwidth issue)

    v4l2-ctl -d<device no> --set-fmt-video=width=640,height=480

    Still the issue persists.

    If its really a usb controller bandwidth issue then If a single usb camera HD input works then two cam with 640x480 should have worked.

    Just for comparison similar two usb cam setup works well with raspi etc board setup. Is this could be a issue with gstreamer v4l2 Ti implementaion ?

  • 0
    TI__Guru**** 168721 points

    Hi,

    Pallab Sarkar said:
    ../../../gst-plugins-good-1.6.3/sys/v4l2/gstv4l2bufferpool.c(1055): gst_v4l2_buffer_pool_poll (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
    poll error 1: No space left on device (28)

    This message means usb has no enough bandwidth.

    Pallab Sarkar said:

    gst-launch-1.0 -v v4l2src device=/dev/video1 ! videoconvert ! videoscale ! video/x-raw,format=I420,width=800,height=600,framerate=25/1 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.2.15 port=5000

    gst-launch-1.0 -v v4l2src device=/dev/video2 ! videoconvert ! videoscale ! video/x-raw,format=I420,width=800,height=600,framerate=25/1 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.2.15 port=5004

    First try run just one gst pipeline to ensure the raw yuv420 800x600@25fps is supported, then reduce either resolution or framerate down to the point which the second pipeline runs.

    If you really want to get 800x600@25fps from both cameras, you have to give up the raw format, but use mjpeg or h.264 whichever your camera supports. You used 'jpegenc' element, why you use x-raw anyway?

  • 0 in reply to Pallab Sarkar
    TI__Guru**** 168721 points

    Pallab Sarkar said:

    Also did experiment to reduce the input video resolution of cameras(two check if its really bandwidth issue)

    v4l2-ctl -d<device no> --set-fmt-video=width=640,height=480

    Still the issue persists.

    I don't think gst takes the config from v4l2-ctl setting. You have to change the resolution or framerate in your gst pipeline.

  • 0 in reply to Bin Liu
    Prodigy 165 points
    Hi Liu and Margarita,

    I have tried to reduce the usb bandwidth consumption ,

    Below is the best what is working with two cam setup as of now

    ======================================================================

    Camera 1 Setup
    ===============
    on x15(streaming server)
    ---------------------------------

    $v4l2-ctl -d1 --set-fmt-video=width=176,height=144

    $gst-launch-1.0 -v v4l2src device=/dev/video1 ! videoconvert ! videoscale ! video/x-raw,width=176,height=144 ! avenc_mpeg4 ! rtpmp4vpay config-interval=3 ! udpsink host=192.168.2.15 port=5200

    on ubuntu(streaming client)
    ---------------------------------
    $gst-launch-1.0 -v udpsrc port=5200 caps = "application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)MP4V-ES\,\ profile-level-id\=\(string\)1\,\ config\=\(string\)000001b001000001b58913000001000000012000c48d8800cd3204709443000001b24c61766335362e312e30\,\ payload\=\(int\)96\,\ ssrc\=\(uint\)2873740600\,\ timestamp-offset\=\(uint\)391825150\,\ seqnum-offset\=\(uint\)2980" ! rtpmp4vdepay ! avdec_mpeg4 ! autovideosink


    Camera 2 Setup
    ===============
    on x15(streaming server)
    ---------------------------------
    $v4l2-ctl -d2 --set-fmt-video=width=176,height=144

    gst-launch-1.0 -v v4l2src device=/dev/video2 ! videoconvert ! videoscale ! video/x-raw,width=176,height=144 ! avenc_mpeg4 ! rtpmp4vpay config-interval=3 ! udpsink host=192.168.2.15 port=5204

    on ubuntu(streaming client)
    ---------------------------------
    $gst-launch-1.0 -v udpsrc port=5204 caps = "application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)MP4V-ES\,\ profile-level-id\=\(string\)1\,\ config\=\(string\)000001b001000001b58913000001000000012000c48d8800cd3204709443000001b24c61766335362e312e30\,\ payload\=\(int\)96\,\ ssrc\=\(uint\)2873740600\,\ timestamp-offset\=\(uint\)391825150\,\ seqnum-offset\=\(uint\)2980" ! rtpmp4vdepay ! avdec_mpeg4 ! autovideosink
    ========================================================================



    As I tried to move up into resolution at max one pipeline at 640x480 and other at 176x144 is working.
    Still I need to see how to optimize so that can achieve parallel HD streaming
    Thanks you for help in this as I am new to TI boards its is really helpful!!

    Regards
    Pallab Sarkar
  • 0 in reply to Bin Liu
    Prodigy 165 points

    Hi Liu,

    As you suggested I tried using jpeg(compressed) directly which is outputted from webcam . I can see improvement in the resolution

    gst-launch-1.0 v4l2src device=/dev/video1 ! image/jpeg,width=640,height=480,framerate=15/1  !  rtpjpegpay !  udpsink host=192.168.2.15 port=5200

    gst-launch-1.0 v4l2src device=/dev/video2 ! image/jpeg,width=1024,height=576,framerate=15/1 !  rtpjpegpay !  udpsink host=192.168.2.15 port=5000

    So now one cam can work at 1024x576, and another 640x480.

    Now my point how to achieve better bandwidth so that dual(or quad) webcam HD streaming can be done.

    I have tried to connect the usb cams to differnt usb ports so that both cams should be on diffrent usb host controller bus.

    But every combination puts the bot cams to same bus

    root@am57xx-evm:~# lsusb -t

    /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M

       |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M

    /:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M

       |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M

           |__ Port 1: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

           |__ Port 2: Dev 4, If 0, Class=Video, Driver=uvcvideo, 480M

           |__ Port 2: Dev 4, If 1, Class=Video, Driver=uvcvideo, 480M

           |__ Port 2: Dev 4, If 2, Class=Audio, Driver=snd-usb-audio, 480M

           |__ Port 2: Dev 4, If 3, Class=Audio, Driver=snd-usb-audio, 480M

           |__ Port 3: Dev 5, If 0, Class=Video, Driver=uvcvideo, 480M

           |__ Port 3: Dev 5, If 1, Class=Video, Driver=uvcvideo, 480M

           |__ Port 3: Dev 5, If 2, Class=Audio, Driver=snd-usb-audio, 480M

           |__ Port 3: Dev 5, If 3, Class=Audio, Driver=snd-usb-audio, 480M

           |__ Port 4: Dev 6, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

    root@am57xx-evm:~# lsusb

    Bus 001 Device 006: ID 0461:4d81 Primax Electronics, Ltd Dell N889 Optical Mouse

    Bus 001 Device 005: ID 046d:0825 Logitech, Inc. Webcam C270

    Bus 001 Device 004: ID 046d:0825 Logitech, Inc. Webcam C270

    Bus 001 Device 003: ID 413c:2106 Dell Computer Corp. Dell QuietKey Keyboard

    Bus 002 Device 002: ID 0451:8140 Texas Instruments, Inc.

    Bus 002 Device 001: ID 1d6b:0003 Linux Foundation

    Bus 001 Device 002: ID 0451:8142 Texas Instruments, Inc. TUSB8041 4-Port Hub

    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation  

    By any means I am not able to access the other USB3.0 bus (Bus 02.Port 1: Dev 1, class="root_hub", Driver=xhci-hcd/1p, 5000M)

    Is their any method to access this bus, so that I can use both cams with HD resolution .

    Regards

    Pallab Sarkar

  • 0 in reply to Pallab Sarkar
    TI__Guru**** 168721 points

    Hi,

    Pallab Sarkar said:

    I have tried to connect the usb cams to differnt usb ports so that both cams should be on diffrent usb host controller bus.

    But every combination puts the bot cams to same bus

    Please check the X15 board schematics, all the usb ports with type-A receptacle are connected to the same usb controller with a hub.

    The second usb controller is connected to the micro-AB receptacle, you would need a micro-A-to-type-A adapter to connect a camera to this port.

    Please also note that by default the second usb controller is configured to otg mode, so you would need to load a usb gadget driver for it to functional. I typically do 'modprobe g_zero'.

    Pallab Sarkar said:

    By any means I am not able to access the other USB3.0 bus (Bus 02.Port 1: Dev 1, class="root_hub", Driver=xhci-hcd/1p, 5000M)

    Is their any method to access this bus, so that I can use both cams with HD resolution .

    You have to attach USB3.0 devices to use the USB3.0 bus. USB2.0 devices wouldn't be connected to USB3.0 bus.

  • 0 in reply to Bin Liu
    TI__Guru**** 168721 points
    Hi,

    Sorry, some information in my post above is not correct, I forgot that the second usb port on X15 board is set to device-only mode, you won't be able to attach any usb device to it.
  • 0 in reply to Bin Liu
    Expert 1920 points

    Hi Bin,

    Thank you very much for the help so far.

    We would like to understand why would a USB 3.0 bus get saturated even when we try to enforce "MJPEG" on both webcams? Individually, 2 USB 2.0 webcams shouldn't even find it possible to saturate a USB 3.0 bus, isn't it? Also, forcing MJPEG on both, we should see around 60Mbps maximum data rate combined at 720p30.

    The math doesn't seem to add up. Could you please provide some pointers such that we can try figuring out what is wrong? We understand that we may have to dig up some of the code ourselves, based on your pointers.

  • 0 in reply to Anup Rajput
    TI__Guru**** 168721 points
    Anup,

    First of all, the USB2.0 devices don't use the USB3.0 bus, even if they are attached to the USB3.0 port, they are still only use the USB2.0 bus.

    You would need a usb bus analyzer to understand what limits the throughput. My hands are full now for other tasks, please give me some time to look at the MJPEG case with 2 cameras. I didn't examine it before. I will keep you posted.
  • 0 in reply to Bin Liu
    Expert 1920 points
    Hi Bin,

    As we have gathered from other threads, there is an issue with the USB drivers accessing webcams, for which you had a "sort of" working patch for the v4.4 kernels, which wasn't very stable. Is there a more stable patch available now?

    Also were you able to try 2 simultaneous MJPEG streams through USB webcams?
  • 0 in reply to Anup Rajput
    TI__Guru**** 168721 points

    Anup,

    I believe the patch you mentioned is the one I did for AM335x, it is for AM335x only. AM335x uses a different USB controller from the one on AM57x, so the referred patch is not applicable to AM57x.

    I haven't got a chance for the MJPEG usecase yet. But what I found is that AM57x USB high-speed host can schedule up to 5 Isoch transfers per SOF, so as long as the two webcams are configured with such resolution & fps which use no more than 5 Isoch transfers per SOF, the video streaming will run.

    I use USB bus analyzer to check the USB bus traffic.

  • 0 in reply to Pallab Sarkar
    Expert 2230 points

    Hi pallab,

      You have said below pipeline works good for you.

    "gst-launch-1.0 v4l2src device=/dev/video1 ! image/jpeg,width=640,height=480,framerate=15/1 ! rtpjpegpay ! udpsink host=192.168.2.15 port=5200"

    Have you done any changes in VIP driver to take image/jpeg as input of v4l2src?

    Regards,
    Prerak