TDA4AH-Q1: Mixing multiple videos using Gstreamer

Hi Suresh,

One way would be to use mosaic node in vision_apps to combine multiple images and give it as one to codec, that you are following up in another thread,

I am looping in gstreamer expert to comment on possibilities from gstreamer plugins.

Regards,
Gokul

Hi Suresh,

There is a GStreamer plugin named tiovxmosaic which does this: https://github.com/TexasInstruments/edgeai-gst-plugins/wiki/tiovxmosaic 

Example dataflows and pipelines can be found here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-am69a/latest/exports/edgeai-docs/common/edgeai_dataflows.html 

Best,
Jared

Hi Jared McArthur, Thanks for responding. I have followed the examples, but still i have issues.. posting the details.

To add more details: 

i wanted to mix more than two video sources ( tried with static pattern, jpg, and appsrc) with gstreamer in C code. when i tried to mix the videos ( jpg, static pattern and appsrc) using linux shell commands i am able to get a mixed output video. followed the commands below:

st-launch-1.0 -e compositor name=mix sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=320 sink_1::ypos=0 sink_2::xpos=0 sink_2::ypos=240 sink_3::xpos=320 sink_3::ypos=240 ! videoconvert ! v4l2h264enc ! h264parse ! mp4mux! filesink location=mixed_video.mp4 videotestsrc pattern=0 ! video/x-raw,width=320,height=240 ! mix.sink_0 videotestsrc pattern=1 ! video/x-raw,width=320,height=240 ! mix.sink_1 videotestsrc pattern=2 ! video/x-raw,width=320,height=240 ! mix.sink_2 videotestsrc pattern=3 ! video/x-raw,width=320,height=240 ! mix.sink_3

gst-launch-1.0 -e compositor name=mix sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=320 sink_1::ypos=0 sink_2::xpos=0 sink_2::ypos=240 sink_3::xpos=320 sink_3::ypos=240 ! videoconvert ! v4l2h264enc ! h264parse ! mp4mux! filesink location=mixed_video.mp4 uridecodebin uri=file:///opt/vision_apps/image.jpg ! videoscale ! video/x-raw, width=320, height=240 ! imagefreeze ! mix.sink_0 videotestsrc pattern=1 ! video/x-raw,width=320,height=240 ! mix.sink_1 videotestsrc pattern=2 ! video/x-raw,width=320,height=240 ! mix.sink_2 videotestsrc pattern=3 ! video/x-raw,width=320,height=240 ! mix.sink_3

gst-launch-1.0 -e compositor name=mix sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=320 sink_1::ypos=0 sink_2::xpos=0 sink_2::ypos=240 sink_3::xpos=320 sink_3::ypos=240 ! videoconvert ! v4l2h264enc ! h264parse ! mp4mux! filesink location=mixed_video.mp4 filesrc location=image.jpg ! videoscale ! video/x-raw, width=320, height=240 ! imagefreeze ! mix.sink_0 videotestsrc pattern=1 ! video/x-raw,width=320,height=240 ! mix.sink_1 videotestsrc pattern=2 ! video/x-raw,width=320,height=240 ! mix.sink_2 videotestsrc pattern=3 ! video/x-raw,width=320,height=240 ! mix.sink_3

When i tried with multicam codec example application code available in vision_apps in Linux-rtos SDK 9.02 using wrapper functions provided by TI to mix video using gstreamer, i am getting an error " gst_wrapper: myAppSrc not an AppSrc element "

Snippets of code , execution logs are attached.  Please support to mix videos using gstreamer.

Hi Suresh,

It looks like you are using a compositor instead of an appsrc.

Best,
Jared

Hi Jared McArthur,

i am using compositor to mix multiple  videosources (tried with static pattern,jpg files as source) 

Hi Suresh,

Yes, but it looks like you're checking whether the compositor is an appsrc, and it's a compositor, not an appsrc.

Best,
Jared

Hi Jared McArthur,

I want to mix multiple video inputs for that i purpose, i have used compositor with appsrc. Please suggest a suitable command options to mix videos from  different sources and encode mixed video with h264 in TDA4H. 

Hi Suresh,

You can use the tiovxmosaic plugin.

If the pipeline works in the command line, it will work in the C code. You are simply checking something that is false, which is causing the C code to fail.

Best,
Jared