• State TI Thinks Resolved
  • Locked Locked
  • Replies 8 replies
  • Answers 1 answer
  • Subscribers 100 subscribers
  • Views 2100 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5708: QT5 and accelerated video playback using eglfs backend

stef boerrigter
Expert 1050 points
Part Number: AM5708

Hi Ti Team,

We use the H264 video decoding plugin using the Ducati plugin for decoding video and want to play this accelerated in our QT application. Running, decoding and displaying the video using Gstreamer pipeline seems to work just fine:

gst-launch-1.0 filesrc location=/big_buck_bunny_720x480_H264_AAC_25fps.mkv ! matroskademux ! h264parse ! ducatih264dec ! kmssink

When using Qt, we notice that QT tries to set the sink to qmlsink or any other. When using the Qt examples that you use in the SDK we see that we get a screen with green edges (same as refered to in https://e2e.ti.com/support/processors/f/791/t/550005?Artifacts-on-H-264-videos-playing-on-Qt5 )

It seems that Matroska Demux in the pipeline detects that the input image is the correct width and height:

0:00:02.431555642 274 0x107a60 INFO matroskademux matroska-demux.c:1500:gst_matroska_demux_add_stream:<matroskademux0> Adding pad 'video_0' with caps video/x-h264, level=(string)3, profile=(string)main, codec_data=(buffer)014d401effe10018674d401eda02d0f6c044000003000400000300c83c58ba8001000468ef3c80, stream-format=(string)avc, alignment=(string)au, width=(int)720, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1

Ducati plugin also seems to detect this pretty good:

video/x-raw, framerate=(fraction)25/1, width=(int)896, height=(int)576, pixel-aspect-ratio=(fraction)1/1, drm_mem=(boolean)true, max-ref-frames=(int)18, format=(string)NV12, interlace-mode=(string)progressive
0:00:26.401782107 292 0xb1434fb0 INFO ducati gstducatividdec.c:692:codec_process:<ducatih264dec0> active frame region 32, 24, 752, 504, crop 720x480

But when using QT; it seems that this cropping is not done:

video/x-raw, framerate=(fraction)25/1, width=(int)896, height=(int)576, pixel-aspect-ratio=(fraction)1/1, drm_mem=(boolean)true, max-ref-frames=(int)18, format=(string)NV12, interlace-mode=(string)progressive

Which is identical in both cases. so the difference seems to be in the videosink (kmssink vs qt (some) sink)??

Performance between QT and Native GStreamer

Also performance seems to be an issue, when using the gst-launch or playbin i see an CPU load of roughly 3 / 5% as i would expect on the AM5708 with Hardware video decoding, but when displaying via Qt i see about 70% CPU load and also memory load very high.

Could you help us out here?

  • 0
    TI__Mastermind 44735 points

    Hello,

    While running the Qt application, I am wondering if you have Weston enabled or disabled. Also, if your application does not require any Windows manager, I would recommend running the application with the following flag "<app_exec> -platform linuxfb"

    Regards,
    Krunal

  • 0 in reply to Krunal Bhargav34
    Expert 1050 points

    Hello Krunal,

    We are not using any window manager, and therefore not Weston. 

    currently we have eglfs implemented 

    ~# cat /etc/profile.d/qtenv.sh 
#/bin/sh

export QT_QPA_PLATFORM=eglfs
export QT_QPA_EGLFS_INTEGRATION=none
export QT_QPA_EGLFS_PHYSICAL_WIDTH=216
export QT_QPA_EGLFS_PHYSICAL_HEIGHT=135

    We have tried several options for displaying the video in QT:

    1. display using Video, VideoOutput and MediaPlayer QML objects in QT

            - we get an video (high load and green edges) in QT when using no -platform options (defaults to eglfs), 
            - when using -platform linuxfb we see no video output but do hear the audio. Load seems OK though.

    2. using coded gstreamer pipeline in C++. 

            - when using platform eglfs (default) we get an error regarding DRM device already locked / busy. 
            - when using -platform linuxfb i see the video displayed full screen over the QT application, load is OK. (This looks promising, but how to control the output / where it is displayed)

    Could you help us with choosing the options above and how to continue?

    Also, perhaps explain what is happening between platform eglfs / linuxfb?

  • 0 in reply to stef boerrigter
    TI__Mastermind 44735 points

    Hello,

    Please refer to the following page for more information about eglfs/linufb: https://doc.qt.io/qt-5/embedded-linux.html.

    Regards,
    Krunal

  • 0 in reply to Krunal Bhargav34
    Expert 1050 points

    Hi,

    Thanks for sharing, i have seen this page before but i can see it has some (usefull) updates since QT 5.11 and later. 
    What i'm curious about is the following:

    Currently we use platform eglfs which performs fine and works nice for our application and we rather keep that implementation if working so:
    i would be curious if there is some sort of solution to have the Video Decoding pipeline (with gstreamer or ....? ) output video in some sort of overlay? or in a QT QML Object? 

    The section https://doc.qt.io/qt-5/embedded-linux.html#advanced-eglfs-kms-features seems to be interesting, perhaps you have some examples on this topic specific for video displaying?

    Best Regards.

  • 0 in reply to stef boerrigter
    TI__Mastermind 44735 points

    Hello,

    Based on my internal discussion, we do not have any examples that integrate Qt and Gstreamer into one application. We do have a couple of examples related to display: example1 and example2.

    Regards,
    Krunal 

  • 0 in reply to stef boerrigter
    Prodigy 130 points

    Hi stef,

    We have encountered the same problem as you, can you tell me how you finally solved it? Thank you very much.

    Regards,
    xiaoma

  • 0 in reply to xiao ma
    Expert 1050 points

    Hi Xiaoma,

    We have not (yet) solved this issue, the examples have not yet been tested / reviewed also due to the fact that these implementations are a bit off compared to our product / usage of the platform.

    We do see 2 topics to dig in further based on the examples: eglfs_kms; which we didn't get working in a day yet, so it might be worthfile to investigate in this further. Also nulldrm using linuxfb to get full drm access seems interresting, but first we want to dig into eglfs_kms plugin. 

    Still looking for a solution, if we find one we surely will share it here (if the topic is still open then), 
    Currently we have other project priorities to be solved first, 

    Best Regards,

    Stef.