AM5728: Sending encoded video and audio via RTP

dhiraj pawar

Part Number: AM5728
Other Parts Discussed in Thread: TVP5147

Hi,

We have a requirement in which we need to transfer video and audio(associated with video) from camera over the network and vice versa. We want to use 'AM5728' as our processor.
AM5728 will take video and audio as input separately, video will be compressed with H.264 in AM5728.

Now our question is
i) how audio (associated with video) will be compressed in AM5728?
ii) how both these separately compressed video and audio are encoded and send to RTP (network), keeping audio synchronized to video? We came across various plug-ins.
Is there any specific plug-in for this for AM5728?

over 7 years ago

0 Margarita Gashova over 7 years ago

TI__Guru* 80190 points

Hello,

1. There are gstreamer software audio encoders.
2. This could be archived by using gstreamer muxer elements. The muxer element merges audio and video in container like mov, avi, mp4, ts etc.
3. There are gst streaming elements like udpsink etc.

BR
Margarita

0 Margarita Gashova over 7 years ago in reply to Margarita Gashova

TI__Guru* 80190 points

In additional, please check this guide:
processors.wiki.ti.com/.../Processor_Training:_Multimedia

BR
Margarita

0 dhiraj pawar over 7 years ago in reply to Margarita Gashova

Prodigy 170 points

Thanks for your quick reply mam.

i) Is my understanding about this as shown below correct?

While sending:
(video + audio) to AM5728 -------> H.264 compression for video + AAC or other compression for audio -------> gstreamer muxer----------->udpsink towards RTP(network)

While receiving:
udpsrc from RTP(network)--------> gstreamer demuxer --------> H.264 decoder(video)+AAC decoder(audio) ---------->HDMI transmitter in AM5728

ii) you said "The muxer element merges audio and video in container".Will Muxer element store some timing information about them so that when we demux them at receiver end and decode them, they will be in sync? or some other way is present which takes care about this synchronization.

0 Margarita Gashova over 7 years ago in reply to dhiraj pawar

TI__Guru* 80190 points

Hello,

1.
Yes. You need the parser elements in the pipeline also.

2.
Should be.
But you could check this article also:
gstreamer.freedesktop.org/.../clocks.html

Please if this answers your questions click the "This resolved my issue" button, to close the thread.

BR
Margarita

0 dhiraj pawar over 7 years ago in reply to Margarita Gashova

Prodigy 170 points

Hello,

you said "You need the parser elements in the pipeline also."
Parsers only cut the stream of data into buffers. For what exactly will it be used and where will be its placement in pipeline?

0 Margarita Gashova over 7 years ago in reply to dhiraj pawar

TI__Guru* 80190 points

Hello,

After the encoders before the muxer.
For example:

.... ! ducatih264enc ! h264parse ! qtmux ! ...

BR
Margarita

0 Prabhakar Lad over 7 years ago

Guru 11260 points

Hello,

If I am not wrong the am5728 doesn’t have a audio encoding hardware so you need to use software encoding avenc_aac plugin from gstreamer.

For sending:

gst-launch-1.0 -v v4l2src device=/dev/video1 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! vpe num-input-buffers=8 ! 'video/x-raw,format=NV12,width=1920,height=1080' ! queue ! ducatih264enc bitrate=6000 ! h264parse ! queue ! mpegts. alsasrc ! queue ! avenc_aac ! compliance=-2 ! aacparse ! mpegts. mpegtsmux name=mpegts ! rtpmp2tpay ! udpsink host=192.168.1.186 port=10000

Note: the client ip and port 192.168.1.186 port=10000 can be changed according to where you want to send, you can also send it on a multicast ip as well. Once you run the above pipeline it will prints the caps for udpsink0.GstPad:sink, note them as you need this on the client side.

Receiver:

On receiver side run the following pipeline, make sure you put in the caps printed by udpsink where I have added caps="application/x-rtp\,.."

gst-launch-1.0 -e -v udpsrc port=10000 caps="application/x-rtp\,.." ! rtpjitterbuffer ! rtpmp2tdepay ! tsdemux name=demux demux. ! queue ! h264parse ! avdec_h264 ! videoconvert ! autovideosink demux. ! queue ! aacparse ! avdec_aac ! audioconvert ! autoaudiosink

Cheers,
--Prabhakar Lad

0 dhiraj pawar over 7 years ago in reply to Prabhakar Lad

Prodigy 170 points

hi,

" For sending: pipeline

gst-launch-1.0 -v v4l2src device=/dev/video1 io-mode=4 ! 'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! vpe num-input-buffers=8 ! 'video/x-raw,format=NV12,width=1920,height=1080' ! queue ! ducatih264enc bitrate=6000 ! h264parse ! queue ! mpegts. alsasrc ! queue ! avenc_aac ! compliance=-2 ! aacparse ! mpegts. mpegtsmux name=mpegts ! rtpmp2tpay ! udpsink host=192.168.1.186 port=10000 "

i) Can we give s-video,component,composite video signals(coming from camera or any source) directly to AM5728 with some plugins ? or

first we have to use external 'video decoder' (Ex,TVP5147) which will convert them into YCrCb 4:2:2 bits and that is given to AM5728. What I have to write in above pipeline for Capturing that signals ?

ii) And about the audio associated with this videos , does AM5728 take this AUDIO directly or we have to use audio codec also externally before AM5728 ?