• State TI Thinks Resolved
  • Locked Locked
  • Replies 8 replies
  • Answers 1 answer
  • Subscribers 100 subscribers
  • Views 784 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5728: Streaming video using gstreamer

Vishal Maheshwari
Expert 1030 points
Part Number: AM5728


Tool/software:

We are using a custom board based on AM5728 SOC with ov5640 camera integrated in it and are  successfully able to stream video between two of our custom boards with sender camera resolution 640 x 480 using gstreamer utility. But, when we switch to camera resolution 1280 x 720, there is a considerable lag in streaming video arriving at  receiver end.

The following commands were used to send and receive video.


Sender (640 x 480):-

gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=640,height=480,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
udpsink host=192.168.8.188 port=5000

Sender (1280 x 720):-

gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
'video/x-raw, format=(string)NV12, width=(int)1280, height=(int)1232' ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
udpsink host=192.168.8.188 port=5000

Receiver:-

gst-launch-1.0 -v udpsrc port=5000 ! \
'application/x-rtp,media=(string)video, clock-rate=(int)90000, payload=(int)96' ! \
rtph264depay ! \
h264parse ! \
ducatih264dec ! \
vpe ! \
'video/x-raw,format=(string)NV12, width=640,height=480' ! \
videoscale sharpen=1 ! \
kmssink sync=false

We have used ducatih264 encoder and decoder which runs on IVA-HD (as per TI) as can be verified from the commands above. 

1. What could be the reason behind the lag even when IVA-HD is used?
2. How can we confirm that IVA-HD is actually getting used?
3. How can the lag be reduced?

  • 0
    TI__Mastermind 35436 points

    Hello Vishal,

    We no longer have a subject matter expert for this IP so please be aware that our support for applications on this IP will be limited. The majority of Multimedia including the IVA-HD support has been now de-scoped from our standard product. 

    Please allow some time for us to look at your query and see how we can help.

    Can you please confirm what SDK you are using and whether you have custom boards or TI boards?

    -Josue 

  • 0
    TI__Mastermind 19670 points

    Hi Vishal,

    Can you confirm that you are changing the camera parameters to the desired resolution before running the GStreamer pipeline that corresponds to that resolution? This can be accomplished with the media-ctl utility.

    For example, the following command would set OV5640 to UYVY format and 1280x720 resolution at 30 fps:

    media-ctl --set-v4l2 "'ov5640 4-003c':0 [fmt:UYVY8_2X8/1280x720@1/30 field:none]"

    Vishal Maheshwari said:
    'video/x-raw, format=(string)NV12, width=(int)1280, height=(int)1232' ! \

    Could you explain the purpose for this line in your 1280x720 pipeline that is not present in the 640x480 pipeline? Why use 1280 and 1232 for the width and height here?

    Thank you,

    Fabiana

  • 0 in reply to Josue Zamitiz-Ayala
    Expert 1030 points

    Like I mentioned we are having a custom board based on TI AM5728 SOC, the SDK version being used is 6.03. 

  • 0 in reply to Fabiana Jaimes
    Expert 1030 points

    Yes, we have enabled the support of 1280x720 resolution in the camera driver.

    Although running the command -> 

    media-ctl --set-v4l2 "'ov5640 4-003c':0 [fmt:UYVY8_2X8/1280x720@1/30 field:none]" gives the following error :- 

    Failed to enumerate /dev/media0 (-2) 


    Fabiana Jaimes said:
    Could you explain the purpose for this line in your 1280x720 pipeline that is not present in the 640x480 pipeline? Why use 1280 and 1232 for the width and height here?

    Yeah, I rectified the command 

    Vishal Maheshwari said:
    video/x-raw, format=(string)NV12, width=(int)1280, height=(int)1232' ! \

    Yeah, this command (for setting display resolution) is not needed at sender's side but it is needed at client side. Now 1280 x 720 resolution is also working fine but there are frame drops at regular intervals. 


    How could frame drops be avoided?
    Thanks!

  • 0 in reply to Vishal Maheshwari
    TI__Mastermind 19670 points

    Could you share the contents of your u-boot environment? Which device tree overlays do you currently have enabled?

    Thank you,

    Fabiana

  • 0 in reply to Fabiana Jaimes
    Expert 1030 points

    We are not able to get u-boot related information as of now.

    Meanwhile, we had asked you this query previously:-

    How can we confirm that IVA-HD is actually getting used?

    Based on this query above, we have this elaborated query below:-

    Can you confirm whether IVAHD performance and usage can be monitored through Code Composer Studio when it is used as a plugin (ducatih264enc) in a GStreamer pipeline? For example, see this pipeline below using ducatih264enc plugin.

    gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
    'video/x-raw,format=(string)YUY2,width=640,height=480,framerate=(fraction)30/1' ! \
    vpe num-input-buffers=8 ! \
    queue ! ducatih264enc intra-interval=1 ! \
    h264parse ! \
    rtph264pay mtu=200 ! \
    udpsink host=192.168.8.188 port=5000


    If it is possible to monitor using CCS then can you send the procedure ?

  • 0 in reply to Vishal Maheshwari
    TI__Mastermind 19670 points

    Hi Vishal,

    Your question is being looked into by our multimedia expert. Please allow some time for a response.

    Thank you,

    Fabiana

  • 0 in reply to Vishal Maheshwari
    TI__Mastermind 18850 points

    Hi Vishal, 

    I am not familiar with the IVA-HD CODEC and am not sure about how to use CCS to monitor the decoder. According to the documentation (here), ducatih264enc is the hardware encoder, so by specifying this in the pipeline you are using the IVA-HD IP. I believe you could monitor how many frames are being encoded by using a Python script that parses GStreamer debug logs with the below steps.

    1) Copy this script to the board: parse_gst_tracers.py

    2) Run the Gstreamer command with "GST_DEBUG_FILE=gst_trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element):v4l2" time" before gst-launch-1.0

    3) Run the python script on gst_trace.log: python3 parse_gst_tracers.py gst_trace.log

    BR,
    Sarabesh S.