Gstreamer plugin binding to OMX layer ?

Ashish Kumar Mishra

Hi All ,

1) With the Gstreamer0.10 of EZSDK we are facing the problem of MUX ( AVI MUX ) not changing the state when issued command of

gst_element_set_state (mux, GST_STATE_NULL);

Can group members provide any feed back on same ?

2) To find the probable pointers ( i am not having exposure to OMX or Gstreamer plugin programming ) i tried looking at OMX sample code

of capture-encode . But i could not find any relevant pointers here , hence i feel for this i should look at Gstreamer layer & not OMX.

Could group members please provide some pointers or let me know if i am missing any aspect.

over 9 years ago

0 Margarita Gashova over 9 years ago

TI__Guru* 80190 points

Hello,

Are you see this issue only with avimux ?
What about if you use mpegtsmux for example or else?
I am asking because if with mpegtsmux is working or other it could be a problem with the element itself. I have doubts in this but you could try.
One more question: could you share the pipeline which you are using in your application?

The OMX capture encode demo it not using any muxers. It encodes the stream and save elementary stream in file(no container).

BR
Margarita

0 Ashish Kumar Mishra over 9 years ago in reply to Margarita Gashova

Expert 1550 points

Hi Margarita ,

Please find the bare minimum sample code for the same , i am finding short of options to figure the solution for this.

Same code can be easily complied to :-

#define VERSION 1 ----> Gstreamer 1.6 ( Works )

#define VERSION 0 ----> Gstreamer 0.10 ( MUX Hangs )

Please do let me know your view on the code.

Fullscreen 7635.sample_code.c Download

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <gst/gst.h>

#define TIMER_DURATION 10 
#define VERSION 0

GstElement 		*pipeline,
			*src, 
			*timeoverlay,
			*clockoverlay,
			*q1, 
			*q2, 
			*filesink,
			*mux,
			*audiosrc,
			*capsfilter,
			*tempfilesink;

GstCaps 		*acaps;

GMainLoop 		*loop,
			*mainloop;

GstPad 			*muxvideosinkpad,
			*muxaudiosinkpad,
			*muxsrcpad,
			*filesinkpad,
			*q1sinkpad,
			*q2sinkpad;

static gulong 		probe_id; // probe ID

GstBus 			*bus;

GError 			*err = NULL;



gchar  			*sinkname;
char 			buffer[128];
volatile unsigned int 	attempt=0;
unsigned int 		timer_count=0;
unsigned int 		dynamic_filesink= 1;  
gboolean		return_bool = FALSE,
			chunk = TRUE;
unsigned long		return_gulong =0;

typedef enum {
	  NO_NEW_FILE,	        // Keep current file destination
	  NEW_FILE,		// Switch file destination
} NewFileStatus;

static NewFileStatus newfile = NO_NEW_FILE; // Switch File Flag

#if ( VERSION == 1) 
static GstPadProbeReturn eos_cb (GstPad * pad, GstPadProbeInfo * info, gpointer user_data)
{
  g_print("\t [ APP-BLOCK ]: EOS_PROBE [ START ] ===================\n");
  if (GST_EVENT_TYPE (GST_PAD_PROBE_INFO_DATA (info)) != GST_EVENT_EOS)
    return GST_PAD_PROBE_DROP;

   gst_pad_remove_probe (pad, GST_PAD_PROBE_INFO_ID (info));

#else
static gboolean eos_cb (GstPad * pad, GstEvent * event, gpointer user_data)
{
  g_print("\t [ APP-BLOCK ]: EOS_PROBE [ START ] ===================\n");
  if (GST_EVENT_TYPE (event) != GST_EVENT_EOS){
    return TRUE;
  }

  //Remove the probe first
  gst_pad_remove_event_probe (pad, probe_id);
#endif


  gst_element_release_request_pad(mux, muxvideosinkpad);
  g_print("\t [ APP-BLOCK ]: About to set ELEMENTS as NULL \n");
  gst_element_set_state (mux, GST_STATE_NULL);  // Gst-0.10 hangs at this instruction 
  gst_element_set_state (filesink, GST_STATE_NULL);        
  
  // Remove unlinks automatically
  gst_bin_remove (GST_BIN (pipeline), mux);
  gst_bin_remove (GST_BIN (pipeline), filesink);
  
  GstElement *mux0   = gst_element_factory_make  ("avimux",NULL);
  GstElement *fsink0 = gst_element_factory_make("filesink", NULL);
  mux 		= mux0;
  filesink 	= fsink0;
  if(!mux0 || !fsink0) {
	g_print("[ERROR]: Missing elements\n");
  }
 
  gst_bin_add (GST_BIN (pipeline), mux);
  gst_bin_add (GST_BIN (pipeline), filesink);

#if (VERSION == 1)   
  sprintf( buffer, "test_%u.avi" , attempt);
#else
  sprintf( buffer, "test_%d.avi" , attempt);
#endif 

  g_object_set(G_OBJECT (filesink),"location", buffer, NULL);

  muxsrcpad   = gst_element_get_static_pad (mux,"src");
#if (VERSION == 1)
  muxvideosinkpad = gst_element_get_request_pad (mux, "video_%u");
#else
  muxvideosinkpad = gst_element_get_request_pad (mux, "video_%d");
#endif

  if (!gst_element_link_many (q1, mux,NULL )) {
	    g_print ("[APP]: Failed to link DYNAMIC Pad's of elements");
	    return -3;
  }

  filesinkpad = gst_element_get_static_pad (filesink,"sink");
  if (!gst_element_link_many (mux,filesink, NULL)) {
	    g_print ("[APP]: Failed to link STATIC_2 Pad's of elements");
	    return -3;
  } 

  g_print ("Moving to PLAYING\n");
  gst_element_set_state (mux, GST_STATE_PLAYING);
  gst_element_set_state (filesink, GST_STATE_PLAYING);

  newfile = NO_NEW_FILE;
  chunk = TRUE;

  g_print("\t [ APP-BLOCK ]: EOS_PROBE [ STOP ] ===================\n"); 

#if ( VERSION == 1) 
 return GST_PAD_PROBE_OK;
#else
  return TRUE;
#endif


} 



#if ( VERSION == 1 ) 
static GstPadProbeReturn pad_probe_cb (GstPad * pad, GstPadProbeInfo * info,gpointer user_data)
#else 
static gboolean pad_probe_cb (GstPad * pad, GstBuffer * buffer, gpointer user_data)
#endif
{
  unsigned int sleep_count =0;

  g_print("\t [ APP-BLOCK ]: VIDEO_PAD_BROBE [ START ] ===================\n");
#if ( VERSION == 1) 
    gst_pad_remove_probe (pad, GST_PAD_PROBE_INFO_ID (info));
#else
   gst_pad_remove_buffer_probe (pad, probe_id);
#endif 

   g_object_set (G_OBJECT(src), "pattern",attempt, NULL);

#if ( VERSION == 1 )    
    gst_pad_add_probe  (filesinkpad,GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM, eos_cb, user_data, NULL);
#else
   probe_id = gst_pad_add_event_probe (filesinkpad, G_CALLBACK(eos_cb), user_data);
#endif 
   gst_pad_send_event (muxvideosinkpad, gst_event_new_eos ());


  // Wait til the EOS have been processed  
  while(newfile != NO_NEW_FILE){
	g_print("In sleep %d \n",sleep_count);
	sleep_count=sleep_count+1;
	sleep(1);
  }

   g_print("\t [ APP-BLOCK ]: VIDEO_PAD_BROBE [ STOP  ] ===================\n");
#if ( VERSION == 1) 
 return GST_PAD_PROBE_OK;
#else
  return TRUE;
#endif

} // End brace of video_pad_probe_cb


static gboolean timeout_cb (gpointer user_data)
{

  attempt = attempt+1;
  timer_count=timer_count+1;

  if ( chunk == TRUE ) {
	chunk = FALSE;
	g_print("\t [ APP-TIMER ]: IN-TIMER %d [ BEFORE-BLOCKING ] =========\n",timer_count);
	newfile = NEW_FILE;

	#if ( VERSION == 1 )
	gst_pad_add_probe (q1sinkpad, GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM,pad_probe_cb, user_data, NULL);
	#else 
 	probe_id = gst_pad_add_buffer_probe (q1sinkpad, pad_probe_cb, user_data);
	#endif
  }

  g_print("\t [ APP-TIMER ]: IN-TIMER %d =========\n",timer_count);
  if ( timer_count == 12 ){
 	return FALSE;
  }

  return TRUE;
} // End brace of "timeout_cb"






static gboolean bus_cb (GstBus * bus, GstMessage * msg, gpointer user_data)
{

  switch (GST_MESSAGE_TYPE (msg)) {

    case GST_MESSAGE_ERROR:{
	GError *err = NULL;
	gchar *dbg;
	gst_message_parse_error (msg, &err, &dbg);
	gst_object_default_error (msg->src, err, dbg);
	g_error_free (err);
	g_free (dbg);
	g_main_loop_quit(mainloop);
    break;
    }

 
 
    case GST_MESSAGE_STATE_CHANGED: {
	GstState old_state, new_state;
	gst_message_parse_state_changed (msg, &old_state, &new_state, NULL);
	g_print ("[ APP - MESSAGE ]: Element %s changed state from %s to %s.\n",
	      				GST_OBJECT_NAME (msg->src),     
					gst_element_state_get_name (old_state),
			       		gst_element_state_get_name (new_state));
    break;
    }

    case GST_MESSAGE_WARNING:{
	GError *err = NULL;
        gchar *name, *debug = NULL;
        name = gst_object_get_path_string (msg->src);
        gst_message_parse_warning (msg, &err, &debug);
        g_print ("\n[ APP - MESSAGE ]: ERROR: from element %s: %s\n", name, 
								err->message);
        if (debug != NULL)
           g_print ("%s \n\n",debug);
  
        g_error_free (err);
        g_free (debug);
        g_free (name);
    break;
    }

    case GST_MESSAGE_EOS:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_EOS\n");
    break;
    }

    case GST_MESSAGE_INFO:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_INFO\n");
    break;
    }

    case GST_MESSAGE_TAG:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_TAG\n");
    break;
    }
  
    case GST_MESSAGE_BUFFERING:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_BUFFERING\n");
    break;
    }
 
    case GST_MESSAGE_STEP_DONE:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_STEP_DONE\n");
    break;
    }
 
    case GST_MESSAGE_CLOCK_PROVIDE:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_CLOCK_PROVIDE\n");
    break;
    }
 
    case GST_MESSAGE_CLOCK_LOST:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_CLOCK_LOST\n");
    break;
    }
 
    case GST_MESSAGE_NEW_CLOCK:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_NEW_CLOCK\n");
    break;
    }
 
    case GST_MESSAGE_APPLICATION:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_APPLICATION\n");
    break;
    }
 
   case GST_MESSAGE_ELEMENT:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_ELEMENT\n");
   break;
   }
 
   case GST_MESSAGE_SEGMENT_START:{
 	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_SEGMENT_START\n");
   break;
   }
 
   case GST_MESSAGE_SEGMENT_DONE:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_SEGMENT_DONE\n");
   break;
   }
 
    case GST_MESSAGE_LATENCY:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_LATENCY\n");
    break;
    }
 
    case GST_MESSAGE_ASYNC_START:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_ASYNC_START\n");
    break;
    }
 
    case GST_MESSAGE_ASYNC_DONE:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_ASYNC_DONE\n");
    break;
    }
 
    case GST_MESSAGE_REQUEST_STATE:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_REQUEST_STATE\n");
    break;
    }
 
    case GST_MESSAGE_STEP_START:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_STEP_START\n");
    break;
    }
 
    case GST_MESSAGE_QOS:{
	g_print ("[ APP - MESSAGE ]: GST_MESSAGE_QOS\n");
    break;
    }
  
 
    default:
      break;
  }

  return TRUE;

}//End brace of SWITCH condition


 
int main (int argc, char **argv)
{


  gst_init (&argc, &argv);
 
  // 0 - Creating the PIPELINE
  pipeline = gst_pipeline_new ("pipeline");
  
  // 1 - Creating the SRC & TIMEOVERLAY
  src = gst_element_factory_make ("videotestsrc", NULL);
  g_object_set (G_OBJECT(src), "pattern",attempt, NULL);
  g_object_set (G_OBJECT(src), "do-timestamp",attempt, NULL);
  g_object_set (G_OBJECT(src), "is-live", TRUE, NULL);  

  clockoverlay = gst_element_factory_make ("clockoverlay", NULL ) ;
  g_object_set (G_OBJECT (clockoverlay), "time-format","%d-%b-%Y / %H:%M:%S",NULL);
 
  timeoverlay = gst_element_factory_make ("timeoverlay", NULL ) ;

  // 2 - Creating the QUEUE 
  q1 = gst_element_factory_make ("queue", NULL);
  q2 = gst_element_factory_make ("queue", NULL);


  // 3 - Creating the MUX 
  mux = gst_element_factory_make("avimux",NULL);

  // 4 - Creating the FILESINK
  filesink = gst_element_factory_make ("filesink", NULL);
  sprintf( buffer, "test_%d.avi" , attempt);
  g_print("\n\t [APP]: Created new file of %s \n",buffer);
  g_object_set(G_OBJECT(filesink),"location", buffer, NULL);


  // 7 - Adding all the elements in to BIN
  gst_bin_add_many (GST_BIN (pipeline),mux,filesink, NULL);  
  gst_bin_add_many (GST_BIN (pipeline), src, clockoverlay ,timeoverlay, q1,NULL);  


  // 8 - Linking all static pad
   if (!gst_element_link_many (src, clockoverlay,timeoverlay , q1, NULL )) {
	    g_print ("[APP]: Failed to link src, clockoverlay,timeoverlay \n");
	    return -3;
  } 
  
  if (!gst_element_link_many (mux,filesink, NULL)) {
	    g_print ("[APP]: Failed to link mux,filesink ");
	    return -3;
  } 


  // 9 - Linking DYNAMIC PAD
#if ( VERSION == 1 )
  muxvideosinkpad = gst_element_get_request_pad (mux, "video_%u");
#else
  muxvideosinkpad = gst_element_get_request_pad (mux, "video_%d");
#endif

  sinkname = gst_pad_get_name (muxvideosinkpad);
  g_print ("\t [APP]: A new pad %s was created\n\n", sinkname);
  if (!gst_element_link_many (q1, mux,NULL )) {
	    g_print ("[APP]: Failed to link DYNAMIC Pad's of q1, mux");
	    return -3;
  }
  g_free (sinkname);


  // 10 - Getting the blocking PAD 
  muxsrcpad   = gst_element_get_static_pad (mux,"src");
  filesinkpad = gst_element_get_static_pad (filesink,"sink");
  q1sinkpad   = gst_element_get_static_pad (q1,"sink");
  if (q1sinkpad == NULL){
	g_print("Q1SINK pad is NULL \n"); 
        return -4;
  }

  // 11 - Starting the PIPELINE 
  if (gst_element_set_state (pipeline,GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
	    g_error ("[APP]: Failed to go into PLAYING state");
	    return -6;
  }

  mainloop = g_main_loop_new (NULL, FALSE);
  gst_bus_add_watch (GST_ELEMENT_BUS (pipeline), bus_cb, mainloop);
  g_timeout_add_seconds (TIMER_DURATION, timeout_cb, mainloop);
  g_main_loop_run (mainloop);  
  
  // 10 - Stopping the PIPELINE
  gst_element_set_state (pipeline, GST_STATE_NULL);
  gst_object_unref (pipeline);
    return 0;
}

0 Margarita Gashova over 9 years ago in reply to Ashish Kumar Mishra

TI__Guru* 80190 points

Hello,

I will take a look at the code.
On the board you have both gstreamer plugins 0.10 and 1.x, right?

BR
Margarita

0 Margarita Gashova over 9 years ago in reply to Margarita Gashova

TI__Guru* 80190 points

Hello,

Could you provide answers on these questions also:

"Are you see this issue only with avimux ?
What about if you use mpegtsmux for example or else?
I am asking because if with mpegtsmux is working or other it could be a problem with the element itself. I have doubts in this but you could try. "

BR
Margarita

0 Ashish Kumar Mishra over 9 years ago in reply to Margarita Gashova

Expert 1550 points

Hi Margarita ,

1. No i am not having gstreamer1.0 on board .
I have tested the code on my desktop which had gst1.0. On board i am having gst0.10. Is gst1.0 available for dm816x?

2. I have seen similar problem when i changed mux to matroskamux ( from avimux) .
I havent tried with mpegtsmux.

0 Margarita Gashova over 9 years ago in reply to Ashish Kumar Mishra

TI__Guru* 80190 points

Hello,

Ashish Mishra1 said:
1. No i am not having gstreamer1.0 on board .
I have tested the code on my desktop which had gst1.0. On board i am having gst0.10. Is gst1.0 available for dm816x?

In EZSDK we have 0.10 version. Since I know some e2e community members implemented 1.x in EZSDK successful.

Could you compile this application for gst 0.10 and check it on the board.

I have doubts that when you build it, it is built for 0.10.

As I said in previous thread the version 1.x and 0.x are different.

Ashish Mishra1 said:
2. I have seen similar problem when i changed mux to matroskamux ( from avimux) .
I havent tried with mpegtsmux.

mpegtsmux was just an example. Since you observe the same issue with matroskamux, it is not the element itself.

BR
Margarita

0 Ashish Kumar Mishra over 9 years ago in reply to Margarita Gashova

Expert 1550 points

Hi Margarita ,
I have tested this code on DM816x Board & the behaviour is "MUX" doesnt responds to state change signal ( change to NULL)

0 Margarita Gashova over 9 years ago in reply to Ashish Kumar Mishra

TI__Guru* 80190 points

Hello,

Your use case is xxxx !....mux ... ! filesink and you want to unlink the mux and the filesink when the pipeline is in the playing state, right?
If yes:
I do not see in the code where you unlink the mux with the previous element(xxxx) before to send eos to the mux and filesink.

Here is example :

.----------. .----------. .----------.
| element1 | | element2 | | element3 |
... src -> sink src -> sink ...
'----------' '----------' '----------'
.----------.
| element4 |
sink src
'----------'

The purpose is to replace element2 with element4 in the PLAYING
pipeline.

1) block element1 src pad. This can be done async.
2) wait for block to happen. at that point nothing is flowing between
element1 and element2 and nothing will flow until unblocked.
3) unlink element1 and element2
4) optional step: make sure data is flushed out of element2:
4a) pad event probe on element2 src
4b) send EOS to element2, this makes sure that element2 flushes
out the last bits of data it holds.
4c) wait for EOS to appear in the probe, drop the EOS.
4d) remove the EOS pad event probe.
5) unlink element2 and element3
5a) optionally element2 can now be set to NULL and/or removed from the
pipeline.
6) link element4 and element3
7) link element1 and element4
8) make sure element4 is in the same state as the rest of the elements. The
element should at least be PAUSED.
9) unblock element1 src

The same flow can be used to replace an element in a PAUSED pipeline. Only
special care has to be taken when performing step 2) which has to be done
async or it might deadlock. In the async callback one can then perform the
steps from 3). In a playing pipeline one can of course use the async block
as well, so that there is a generic method for both PAUSED and PLAYING.

The same flow works as well for any chain of multiple elements and might
be implemented with a helper function in the future.

Issues
~~~~~~

When an EOS event has passed a pad and the pad is set to blocked, the block will
never happen because no data is going to flow anymore. One possibility is to
keep track of the pad's EOS state and make the block succeed immediatly. This is
not yet implemenented.

When dynamically reconnecting pads, some events (like NEWSEGMENT, EOS,
TAGS, ...) are not yet retransmitted to the newly connected element. It's
unclear if this can be done by core automatically by caching those events and
resending them on a relink. It might also be possible that this needs a
GstFlowReturn value from the event function, in which case the implementation
must be delayed for after 0.11, when we can break API/ABI.

This is for 0.10 version.

You could allocate this information about dynamic replace elements in playing state in part-block.txt file.
You could also check the code which I shared with you in previous thread about the algorithm.

If I have missed the line where you unlink the element before with the mux element let me know before the eos to mux.

BR
Margarita

0 Ashish Kumar Mishra over 9 years ago in reply to Margarita Gashova

Expert 1550 points

Hi Margarita ,

Firstly , i am thankful to you that you are putting your valuable time and sharing inputs to help me solve this problem .

w.r.t your above mentioned pointers , i would re-align the code , test & share the outcome .

0 Margarita Gashova over 9 years ago in reply to Ashish Kumar Mishra

TI__Guru* 80190 points

Hello,

Let me know the result.

Best Regards,
Margarita

0 Margarita Gashova over 9 years ago in reply to Margarita Gashova

TI__Guru* 80190 points

Hello,

You could also check the sample code which I shared it is following the almost the same algorithm.

...! queue ! filesink

BR
Margarita

Processors

Processors forum

Gstreamer plugin binding to OMX layer ?