Linux/DRA746: Linux/DRA746: Waylansink:Video(mp4) playback hangs when playing with subtitle files

Hello Ram,

Do, you have any further updates or suggestion how to debug it further.

Regards,
Ikshwaku

gst-launch-1.0 filesrc location=/tmp/USB1-1.4-sda1/2-MPEG4\(640360_30\)MP3\(128_44\).mp4 ! mp4parser ! mpeg4videoparse ! ducatimpeg4decvpe !  'video/x-raw, format=(string)NV12' ! textoverlay name=txt ! waylandsink filesrc location=/tmp/USB1-1.4-sda1/2-MPEG4\(640360_30\)MP3\(128_44\).sub ! subparse ! txt.

after adding vpe capfilters not change in behavior video freeze and getting killed and getting oom .

by adding  software decoder(avdec_mpeg4) not able to play

gst-launch-1.0 filesrc location=/tmp/USB1-1.4-sda1/2-MPEG4\(640360_30\)MP3\(128_44\).mp4 ! qtdemux ! mpeg4videoparse ! avdec_mpeg4 ! textoverlay name=txt ! waylandsink filesrc location=/tmp/USB1-1.4-sda1/2-MPEG4\(640360_30\)MP3\(128_44\).sub ! subparse ! txt.

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

Redistribute latency...

[destroyed object]: error 0: invalid format 0x32315559

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

surface                : 90 created

wl_display@1: error 0: invalid object 3

Error communicating with wayland: Invalid argument

Error communicating with wayland: Invalid argument

Hi Ram,

Could you help here the debug further and fix this issue?

Regards,

Vikash

diff --git a/src/gstvpe.c b/src/gstvpe.c
index a5ed1ff..efe9fcc 100644
--- a/src/gstvpe.c
+++ b/src/gstvpe.c
@@ -41,7 +41,6 @@
 #define MIN(a,b)     (((a) < (b)) ? (a) : (b))
 #endif
 
-
 static void gst_vpe_class_init (GstVpeClass * klass);
 static void gst_vpe_init (GstVpe * self, gpointer klass);
 static void gst_vpe_base_init (gpointer gclass);
@@ -166,7 +165,8 @@ static gboolean
 gst_vpe_set_output_caps (GstVpe * self)
 {
   GstCaps *outcaps;
-  GstStructure *s, *out_s;
+//  GstStructure *s, *out_s;
+  GstStructure *s, *out_s = NULL;
   gint fps_n, fps_d;
   gint par_width, par_height;
   const gchar *fmt = NULL;
@@ -184,8 +184,12 @@ gst_vpe_set_output_caps (GstVpe * self)
           && gst_caps_is_strictly_equal (outcaps, self->output_caps))) {
     GST_DEBUG_OBJECT (self, "Downstream allowed caps: %s",
         gst_caps_to_string (outcaps));
-    out_s = gst_caps_get_structure (outcaps, 0);
-    fmt = gst_structure_get_string (out_s, "format");
+//    out_s = gst_caps_get_structure (outcaps, 0);
+//    fmt = gst_structure_get_string (out_s, "format");
+    if(gst_caps_get_size(outcaps) > 0 ) {
+	out_s = gst_caps_get_structure (outcaps, 0);
+	fmt = gst_structure_get_string (out_s, "format");
+    }
 
     if (out_s &&
         gst_structure_get_int (out_s, "width", &self->output_width) &&
@@ -210,8 +214,10 @@ gst_vpe_set_output_caps (GstVpe * self)
           2 : self->input_crop.c.height;
       self->output_width = self->input_crop.c.width;
     } else {
-      self->output_height = self->input_height;
-      self->output_width = self->input_width;
+//      self->output_height = self->input_height;
+//      self->output_width = self->input_width;
+      self->output_height = self->input_crop.c.height;
+      self->output_width = self->input_crop.c.width;
     }
     self->output_fourcc = GST_MAKE_FOURCC ('N', 'V', '1', '2');
   }

I have used this pipeline:

gst-launch-1.0 playbin uri=file:///tmp/USB1-1.4-sda1/02_DivX6\(AVI\)_with_SRT_subtitle\(German\).AVI video-sink=waylandsink

please find the logs and vpe changes as attachment.

One difference i have observed:

1. with vpe changes pipeline output shows 

root@mmt2020-a880:~# gst-launch-1.0 playbin uri=file:///tmp/USB1-1.4-sda1/02_DivX6\(AVI\)_with_SRT_subtitle\(German\).AVI video-sink=waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...

*************************************************
Ittiam AVI Demuxer plugged in
*************************************************

***********************************************************
ITTIAM MP3 PLUGIN
***********************************************************

2. without vpe changes pipeline output:

root@mmt2020-a880:~# gst-launch-1.0 playbin uri=file:///tmp/USB1-1.4-sda1/02_DivX6\(AVI\)_with_SRT_subtitle\(German\).AVI video-sink=waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...

*************************************************
Ittiam AVI Demuxer plugged in
*************************************************

***********************************************************
ITTIAM MP3 PLUGIN
***********************************************************
surface : 90 created
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
SetDestinationRectangle: surface ID (90), Width (800), Height (480)
SetSourceRectangle : surface ID (90), Width (800), Height (480)
SetVisibility : surface ID (90), ILM_TRUE
layerAddSurface : surface ID (90) is added to layer ID (1000)
New clock: GstAudioSinkClock

So, basically surface is not created when we are playing with vpe changes.

Hi Ikshwaku,

I am not able to get any clue why it is not working for you. I don't see any logs for vpe plugin.

Please revert the earlier patch ,apply the attached one and check the behavior.

diff --git a/src/gstvpe.c b/src/gstvpe.c
index a5ed1ff..de0fbd9 100644
--- a/src/gstvpe.c
+++ b/src/gstvpe.c
@@ -209,6 +209,9 @@ gst_vpe_set_output_caps (GstVpe * self)
           (self->interlaced) ? self->input_crop.c.height *
           2 : self->input_crop.c.height;
       self->output_width = self->input_crop.c.width;
+    } else if(self->input_crop.c.width) {
+      self->output_height = self->input_crop.c.height;
+      self->output_width = self->input_crop.c.width;
     } else {
       self->output_height = self->input_height;
       self->output_width = self->input_width;

Note: The patch is going to work for only resolution below 1920x1080, it will not work for your 2560x1440 since vpe can support width upto 2048 only.

Ram

Hi Ram,

Could you please help here to resolve this issue? We are not able to play with VPE patch yet.

@Ikshwaku,

Could you share the exact commands you tried to test this?

Regards,

Vikash

Hi Ikshwaku,

Looks like vpe is configured and used from M4 . Can you confirm this?

If it's yes, then you will not be able to use vpe from A15.

Hello Ram,

I have compiled my kernel by commenting DISABLE_COMPLETE(vpe); in our DTS file. But still after that I am not able to see /dev/video0 .

Do you have any idea why I am not able to see it. Do I need to change any other configs also?

Regards,
Ikshwaku

Hello Ram,

Please find the dmesg logs as attachment.

7737.dmesg.zip

We are using different dts files.

Do we need to enable any defconfig for vpe.

Thanks,

Ikshwaku

One more update:

On our target this path "/lib/modules/4.4.xxxxxxxxxxxxxxxxxx/kernel/drivers/media" doesn't exists.

Thanks,
Ikshwaku