This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TMDSIDK574: Video acceleration with Qt multimedia

Part Number: TMDSIDK574

Hi,

my name is Thomas and im currently working with the evaluation board TI TMDSIDK574 and the according SDK with linux. The sdk version is 5.00.00.15. The Qt Version in this sdk is Qt 5.9.6

I'm currently trying to create a Qt application, showing a viedo. Qt is configured to use gstreamer.

First thing I've done was to try to show the video with gstreamer alone, which worked fine with the following pipeline:


gst-launch-1.0 playbin uri="file:///home/root/VideoForschung/am5728-gst-tests/video-samples/TearOfSteel-Short-720x420.mpeg2" video-sink=waylandsink

...  Using top, I could verify, that the CPU is utilized in this case for approximately 4%, which lets me assume, that hardware acceleration is properly used.

Now I built a small QML based application, which does the same in QT. Looks like the following:

import QtQuick 2.9
import QtQuick.Window 2.2
import QtMultimedia 5.9
import QtQuick.Layouts 1.3
Window {
    visible: true

    property string videoFile : "file:///home/root/VideoForschung/am5728-gst-tests/video-samples/TearOfSteel-Short-720x420.mpeg2"
    width: 1280
    height: 800
    title: qsTr("Hello World")
    RowLayout {
        anchors.fill: parent
        Rectangle {
            anchors.fill: parent
            Layout.fillWidth: true
            Layout.minimumWidth: 720
            Layout.minimumHeight: 420

            color: "black"

            MediaPlayer {
                id: player
                source : videoFile
                autoPlay: true
            }

            VideoOutput {
                id: videoOutput
                width: 720
                height: 420
                source: player
                orientation : 1
            }
        }
    }
}

This application works too, but 'top' shows me a CPU load of 95%, although the resolution is not yet the resolution I intend to display. Using GST_DEBUG settings showed me, that gstreamer is used in this case too and even the ducatih264 decoder is used.

My question is now: Is this the normal performance in connection with QT, or have I missed something?

Thank you in advance

Thomas

  • Hello,

    Gstreamer point of view, when you are using this pipeline:

    gst-launch-1.0 playbin uri="file:///home/root/VideoForschung/am5728-gst-tests/video-samples/TearOfSteel-Short-720x420.mpeg2" video-sink=waylandsink

    VPE is also linked between decoder and waylandsink.

    I am not very aware with QML but since I know in this case playbin is used.

    Thomas Hofmann1 said:
    This application works too, but 'top' shows me a CPU load of 95%, although the resolution is not yet the resolution I intend to display. Using GST_DEBUG settings showed me, that gstreamer is used in this case too and even the ducatih264 decoder is used.

    I have a suspicions that after the decoder element, videoconvert element is linked when you are using this in qt app. Videoconvert is a software element. If there is conversion from nv12 to something else via this element the CPU load will be high.

    Best Regards,

    Margarita

     

  • Hello Margarita
    Thank you for your answer. Are you aware of some debugging feature, which might enable me to see the pipline, which is used by QT? And what may I do, if We can validate your assumtion?
    Best Regards
    Thomas
  • Hello,

    When you are using gstreamer pipeline if you add -v (--verbose) will display you the linked elements, capsfilters :
    gst-launch-1.0 playbin uri="file:///home/root/VideoForschung/am5728-gst-tests/video-samples/TearOfSteel-Short-720x420.mpeg2" video-sink=waylandsink -v

    "Using GST_DEBUG settings showed me, that gstreamer is used in this case too and even the ducatih264 decoder is used."
    Try the same way but search for vpe or videoconvert. If it is videoconvert check the format that you have on output(src), on input(sink) you will have nv12.

    Please notice this suggestion is from gstreamer point of view.

    Best Regards,
    Margarita
  • Hello,
    sorry for the late answer. I'm affraid, that you are right with your assumtion that the Qt Video Widget adds an additional video conversion in software, which slows don the video performance. Unfortunately I have no real possibility to influence the QT realization for this problem.
    So I tried to mix gstreamer output from an optimized pipline and a Qt widget via the video overlay interface of the waylandsink.
    It already works partially and I'm currently stucked on resizing a running video. The currently applied pipline is similar to the one in the first post with a vpe and a capsfilter just before the waylandsink. My intention was to dynamically manipulate the capsfilter for the new resolution, but this doesn't work. I think VPE doesn't take care of the caps filter, once the filter is adjusted and anything is running.
    What do I have to do, if I want to resize the output of a currently running pipeline which uses VPE?
    (perhaps, this might be a new thread, but this question arised, while working on the realization of a video widget with hardware acceleration)

    Best Regards
    Thomas
  • Hello,

    Please open a new thread regarding gstreamer and how to dynamically manipulate the capsfilter for the new resolutions.
    Thank you!

    Best Regards,
    Margarita