This thread has been locked.
If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.
Tool/software: Linux
from customer:
I recently found a critical error in Linux created using your SDK for AM5728 EVM. It is easily reproducible. When I use your in-build GStreamer with several cameras via LAN and with external monitor connected to EVM, then after several tenths of minutes your AM5728 EVM die. The camera was Axis P3905-R. As Attachment I am sending you "test-frag.sh" script which caused that error (after some time). Also I am providing "AM5728 massages.txt" which is recording of /var/log/massages from AM5728 recorded via putty.
Does this issue only occur when running multiple streams and not for single instance of decoding and displaying using playbin?
Did you try creating the gstreamer pipeline by using individual elements and not using playbin? What happens if you use fakesink or use h264avdec plugin rather using waylandsink or ducatih264dec?
What is meant by getting focus of a window here " I can sometimes even force this stuck by simply getting focus of a window with particular camera stream (this usually stucks)." ?
Manisha,
thank you for your support. Firstly, please see answers to your questions with additional comments from customer. Secondly we have a new question regarding H264 video decoding.
1) Regarding your questions
a) Does this issue only occur when running multiple streams and not for a single instance of decoding and displaying using playbin?
No, it does not. I observed this issue for a single instance as well.
b) Did you try creating the gstreamer pipeline by using individual elements and not using playbin?
Yes I did, but I found out some new information within gstreamer elements. Please see below.
c) What happens if you use fakesink or use h264avdec plugin rather using waylandsink or ducatih264dec?
Fakesink does not help, because there are no gstreamer errors as I said. Just an image get stuck. avdec_h264 works properly as far as I can tell, but ther is a big CPU overhead.
d) Can you indicate what do you mean by getting focus of a window, as per "I can sometimes even force this stuck by simply getting focus of a window with particular camera stream (this usually stucks)." ?
Just clicking on a video window or dragging this window to a different position on a screen.
Regarding b):
I found out that the result is the same in case of simple connection of ducatih264dec source to waylandsink sink. However, when I use VPE postprocessing with the same resolution as in the request for camera stream (1024x768) then it works properly as far as I can tell. Further, I used verbose mode within the same gstreamer pipelines. I found out that in first case it passes a "wrong" size/resolution to waylandsink (1152x864) although final video window seems to have right dimensions (1024x768). Example:
Common variable for a shorter form> GST_COMMON='gst-launch-1.0 rtspsrc location="rtsp://192.168.35.212/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=1024x768&fps=25" ! rtph264depay ! h264parse'
********************************
Wrong behaviour:
$GST_COMMON ! ducatih264dec ! waylandsink
/GstPipeline:pipeline0/GstDucatiH264Dec:ducatih264dec0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ level\=\(string\)4.1\,\ profile\=\(string\)main\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ width\=\(int\)1024\,\ height\=\(int\)768\,\ framerate\=\(fraction\)0/1\,\ parsed\=\(boolean\)true"
/GstPipeline:pipeline0/GstDucatiH264Dec:ducatih264dec0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1152\,\ height\=\(int\)864\,\ framerate\=\(fraction\)0/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ drm_mem\=\(boolean\)true\,\ max-ref-frames\=\(int\)12"
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1152\,\ height\=\(int\)864\,\ framerate\=\(fraction\)0/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ drm_mem\=\(boolean\)true"
********************************
"Correct" behaviour:
$GST_COMMON ! ducatih264dec ! vpe ! 'video/x-raw, width=1024, height=768' ! waylandsink
/GstPipeline:pipeline0/GstDucatiH264Dec:ducatih264dec0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ level\=\(string\)4.1\,\ profile\=\(string\)main\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ width\=\(int\)1024\,\ height\=\(int\)768\,\ framerate\=\(fraction\)0/1\,\ parsed\=\(boolean\)true"
/GstPipeline:pipeline0/GstDucatiH264Dec:ducatih264dec0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1152\,\ height\=\(int\)864\,\ framerate\=\(fraction\)0/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ drm_mem\=\(boolean\)true\,\ max-ref-frames\=\(int\)12"
/GstPipeline:pipeline0/GstVpe:vpe0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1152\,\ height\=\(int\)864\,\ framerate\=\(fraction\)0/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ drm_mem\=\(boolean\)true"
/GstPipeline:pipeline0/GstVpe:vpe0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1024\,\ height\=\(int\)768\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1024\,\ height\=\(int\)768\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)0/1"
********************************
Customer questions - Why would I ever need to call VPE postprocessing when there is no need to scale video? And in case it is necessary, what are limitations? Now it looks more like a bug.
2) Regarding H264 decoding
We use Debian "jessie" armhf and I want to compile gst-ducati with software from repositories by myself. I installed others gst-plugins from standard repositories. I downloaded source code of ipcdev, libdce2 and gst-plugin-ducati from TI git repositories. After several hacks I was able to compile, link and finally launch gstreamer with gst-ducati-plugin. I used this pipeline:
gst-launch-1.0 rtspsrc location="rtsp://192.168.35.212/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=640x480&fps=24" ! rtph264depay ! h264parse ! ducatih264dec ! vpe ! 'video/x-raw, format=YUY2, width=640, height=480' ! xvimagesink
The problem is that this pipeline seems to work just once per 10 or 20 launching approximately. Usually it fails with these messages (set GST_DEBUG variable):
--------------------------------- start ----------------------------------------
Setting pipeline to PAUSED ...
>> dce_init
Open omapdrm device
>> dce_ipc_init
open(/dev/rpmsg-dce) -> 0x150780
>> Engine_open Params::name = ivahd_vidsvr size = 12
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.35.212/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=640x480&fps=24
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:00.613931977 499 0xb500ed80 WARN rtpjitterbuffer rtpjitterbuffer.c:185:rtp_jitter_buffer_set_clock_rate: Clock rate changed from 0 to 90000
0:00:00.625204614 499 0xb5032e60 WARN ducati gstducatih264dec.c:402:gst_ducati_h264dec_set_sink_caps:<ducatih264dec0> num-reorder-frames not found on caps, calculation from stream parameters gives 16
0:00:00.625272609 499 0xb5032e60 WARN ducati gstducatih264dec.c:410:gst_ducati_h264dec_set_sink_caps:<ducatih264dec0> Using 16 frames for reordering
0:00:00.632206112 499 0xb5032e60 WARN ducati gstducatividdec.c:552:codec_process:<ducatih264dec0> err=-1, extendedError=00208001
0:00:00.632252960 499 0xb5032e60 ERROR ducati gstducati.c:63:gst_ducati_log_extended_error_info: Bit 0 (00000001): no error-free slice
0:00:00.632283379 499 0xb5032e60 ERROR ducati gstducati.c:63:gst_ducati_log_extended_error_info: Bit 15 (00008000): fatal
0:00:00.632310219 499 0xb5032e60 ERROR ducati gstducati.c:63:gst_ducati_log_extended_error_info: Bit 21 (00200000): unsupported resolution
0:00:00.632718838 499 0xb5032e60 WARN ducati gstducatividdec.c:572:codec_process:<ducatih264dec0> changing max-ref-frames in caps to 18
---------------------------------- end -----------------------------------------
In case the pipeline is launched successfully (rarely) then a video stream is correctly displayed. Behaviour is the same while changing fakesink instead of xvimagesink. Decoder and postprocessing (VPE) seems to work properly in case the pipeline is launched successfully - CPU usage is about 4% while using the pipeline with fakesink and almost static scene for a camera. Do you know where the problem might be?
Thank you.
Hi Bartosz,
Meanwhile can you ask customer to include queue elements in the pipeline. Some queues could be full or src element start dropping frames causing the deadlock.
As well include the gst-debug. Also can they generate the stream with base profile or other profiles with no B frames and test it?
Common variable for a shorter form> GST_COMMON='gst-launch-1.0 -v --gst-debug=2 rtspsrc location="rtsp://192.168.35.212/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=1024x768&fps=25" ! queue ! rtph264depay ! queue ! h264parse ! queue '
********************************
$GST_COMMON ! queue ! ducatih264dec ! queue ! waylandsink
Manisha,
customer adjusted the pipeline upon your request and tested it again both with Main profile and with Baseline profile. Unfortunately, the result is the same. You can find log files with output messages below:
You have captured the recommended solution well -
There was a recent issue identified with waylandsink (not relevant to reported frozen screen situation). Check below e2e post to learn about the issue and download the patch.
Linux/DRA746: waylandsink: output has wrong width and height - Processors forum - Processors - TI E2E...
Hi Bartosz,
The rotation of particular individual "surface" or "layer" can be achieved using the IVI shell.
You can use the following section as a starting point:
After you are able to launch some applications, you can then try:
LayerManagerControl set surface <id> orientation <0/1/2/3> for steps of rotation in 90 degree angles.
Hi Manisha,
thanks for your great support and for the hint with IVI shell. We have a couple of questions regarding your information:
1) Does the Qt5 from the PLSDK rootfs have ivi-application protocol support?
2) Is it possible to render gstreamer video (waylandsink) into Qt widget?
We found it might be possible with similar snippet:
QPlatformNativeInterface *native = QGuiApplication::platformNativeInterface();
struct wl_surface *surface = static_cast<struct wl_surface *>(native->nativeResourceForWindow("surface", this->windowHandle()));
gst_video_overlay_set_window_handle(
GST_VIDEO_OVERLAY (GST_MESSAGE_SRC (msg)),
(guintptr) surface
);
gst_video_overlay_set_render_rectangle(
GST_VIDEO_OVERLAY (GST_MESSAGE_SRC (msg)),
rect.x(), rect.y(),
rect.width(), rect.height()
);
Do you have any experience with this functionality?
Thank you in advance.
Regards,
Bartosz
Bartosz Marcinkowski said:1) Does the Qt5 from the PLSDK rootfs have ivi-application protocol support?
QT application can also run on ivi shell. We have not tested it though using PLSDK. Please try at your end and let us know how it goes.
Bartosz Marcinkowski said:2) Is it possible to render gstreamer video (waylandsink) into Qt widget?
We don't have any experience with this functionality.
Bartosz Marcinkowski said:1) Does the Qt5 from the PLSDK rootfs have ivi-application protocol support?
yes, it can. Please refer to
For individual windows rotation, scaling, positioning, please refer to below link -
"I noticed there was released a new SDK (PROCESSOR-SDK-LINUX-AM57X 04_01_00_06). I downloaded a new refecence root filesystem. It should include patch solving Waylandsink issue which you mentioned. Unfortunatelly, bad behaviour has remained.
Bad behaviour is also observed while using pipeline with VPE without capsfilter:
gst-launch-1.0 -v rtspsrc latency=1000 location="rtsp://192.168.35.212/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=1024x768&fps=25" ! rtph264depay ! h264parse ! ducatih264dec ! vpe ! waylandsink
You can observe that Waylandsink sink has wrong width and height again:
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1152\,\ height\=\(int\)864\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)0/1"
Are we able to reproduce this problem with the IP camera on our end?
Regards,
Bartosz
Bartosz Marcinkowski said:I downloaded a new refecence root filesystem. It should include patch solving Waylandsink issue which you mentioned. Unfortunatelly, bad behaviour has remained.
Processor SDK Linux picks kernel release for integration that has happen 6 weeks before. So any kernel patch that gets applied in six week duration before that quarter end misses integrating into PLSDK release. Waylandsink issue fix patch missed the integration cycle. Please apply it offline and continue with the testing.
Bartosz Marcinkowski said:Bad behaviour is also observed while using pipeline with VPE without capsfilter:
gst-launch-1.0 -v rtspsrc latency=1000 location="rtsp://192.168.35.212/axis-media/media.amp?videocodec=h264&h264profile=main&resolution=1024x768&fps=25" ! rtph264depay ! h264parse ! ducatih264dec ! vpe ! waylandsink
With capsfilter it works well??
Bartosz Marcinkowski said:Are we able to reproduce this problem with the IP camera on our end?
We don't have setup to test with IP camera.
Hi Manisha, Team,
we are still stuck with AM57XX, because positioning of video windows is crucial feature in this application and our customer is not able to do that in any way.
Please see below feedback and provide us any guidelines to move forward.
Thanks.
"In Weston with default desktop shell backend, it is not possible to position windows from script at all. Despite the fact that with IVI shell it is possible to achieve this feature, launching Gstreamer with Waylandsink ends with Segmentation fault. Do you have any progress with this issue on your end?
In case we would not be able to position windows with HW accelerated video, we cannot use AM57XX. If you have any suggestion, please send it. "
On top, regarding previous questions:
"Our setup consists of TI HW (EVM) + TI SW (SDK) within this issue (Gstreamer Waylandsink + IVI shell). So could you reproduce it?
"Waylandsink issue fix patch missed the integration cycle. Please apply it offline and continue with the testing."
As far as I know Waylandsink patch you provided is just "update" of userspace gstreamer plugin and has nothing to do with Linux kernel. Is that correct?
Software manifest for the latest SDK release reveals, that gstreamer1.0-plugins-bad-waylandsink is installed in version 1.8.3-r5.0 where 1.8.3 is version of particular SW and r5.0 is version of TI recipe which apply additional patches and so on. According to Arago GIT repository the patch should be already applied in the gstreamer1.0-plugins-bad-waylandsink which is installed in the latest version of SDK. But maybe I am wrong?