Linux/AM5728: GStreamer H.264 decode error

phil class

Part Number: AM5728

Tool/software: Linux

Hi - I am using gstreamer from the latest TISDK on my custom AM5728 Board:

root@am57xx:~# gst-inspect-1.0 --version
gst-inspect-1.0 version 1.12.2
GStreamer 1.12.2
Unknown package origin
root@am57xx:~# uname -r
4.4.41

I can successfully use gstreamer on the board to encode data into an H264 File with this command:

GST_DEBUG=3 gst-launch-1.0 filesrc location=/am5728-gst-tests/video-samples/airshow_p352x288.yuv ! videoparse width=352 height=288 format=nv12 ! ducatih264enc intra-interval=4 ! queue ! filesink location=/home/root/test.h264

I can then also successfully play back the H264 encoded file on my Ubuntu 16.04 machine with this command ( the resulting video looks great ):

root@build:/home/me/am57xx/build/exported-nfs/home/root# ffplay test.h264

Now, I want to use Qt5.11's QMultimedia framework ( which runs on top of GStreamer ) to play the H264 file. The entire simple test code is here:

#include <QApplication>
#include <QVideoWidget>
#include <QMediaPlayer>
#include <QGraphicsView>
#include <QtMultimedia>
#include <QGraphicsVideoItem>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    QGraphicsView * graphicsView = new QGraphicsView;
    QMediaPlayer * player = new QMediaPlayer;

    QGraphicsScene * scene = new QGraphicsScene;
    graphicsView->setScene(scene);

    QGraphicsVideoItem *item = new QGraphicsVideoItem;
    player->setVideoOutput(item);
    graphicsView->scene()->addItem(item);
    graphicsView->show();

    player->setMedia(QUrl::fromLocalFile("/home/root/test.h264"));
    player->play();

    return a.exec();
}

I run the Qt App with GST_DEBUG and QT_DEBUG_PLUGINS. It finds the gstreamer plugins, and the debugging prints show that it attempts to use the gstducatih264dec functionality. The problem is that GStreamer throws and error "Cant query buffers". Here is the error:

Got keys from plugin meta data ("gstreamermediaplayer")
QFactoryLoader::QFactoryLoader() checking directory path "/usr/bin/mediaservice" ...
loaded library "/usr/lib/qt5/plugins/mediaservice/libgstmediaplayer.so"
QFactoryLoader::QFactoryLoader() checking directory path "/usr/lib/qt5/plugins/resourcepolicy" ...
QFactoryLoader::QFactoryLoader() checking directory path "/usr/bin/resourcepolicy" ...
QFactoryLoader::QFactoryLoader() checking directory path "/usr/lib/qt5/plugins/accessiblebridge" ...
QFactoryLoader::QFactoryLoader() checking directory path "/usr/bin/accessiblebridge" ...
QFactoryLoader::QFactoryLoader() checking directory path "/usr/lib/qt5/plugins/video/gstvideorenderer" ...
QFactoryLoader::QFactoryLoader() checking directory path "/usr/bin/video/gstvideorenderer" ...
QFactoryLoader::QFactoryLoader() checking directory path "/usr/lib/qt5/plugins/accessible" ...
QFactoryLoader::QFactoryLoader() checking directory path "/usr/bin/accessible" ...
0:00:00.311770349  1256    0x31e00 WARN                     oss gstosssink.c:399:gst_oss_sink_open:<audiosink-actual-sink-oss> error: Could not open audio device for playback.
0:00:00.312187101  1256    0x31e00 WARN                     oss gstosssink.c:399:gst_oss_sink_open:<audiosink-actual-sink-oss> error: system error: No such file or directory
0:00:00.321556701  1256    0x31e00 WARN                 basesrc gstbasesrc.c:3480:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.328267514  1256    0x31e00 WARN                 basesrc gstbasesrc.c:3480:gst_base_src_start_complete:<source> pad not activated yet
Warning: "Could not open audio device for playback."
0:00:00.634284017  1256 0xb281ae30 WARN                  ducati gstducatih264dec.c:415:gst_ducati_h264dec_set_sink_caps:<decoder> num-reorder-frames not found on caps, calculation from stream parameters gives 16
0:00:00.634583161  1256 0xb281ae30 WARN                  ducati gstducatih264dec.c:423:gst_ducati_h264dec_set_sink_caps:<decoder> Using 0 frames for reordering
0:00:00.697457081  1256 0xb281ae30 WARN                  ducati gstducatividdec.c:590:codec_process:<decoder> changing max-ref-frames in caps to 19
0:00:00.713963193  1256 0xb281ae30 WARN                     vpe gstvpebufferpool.c:471:gst_vpe_buffer_pool_import: Allocating a new input buffer index: 0/128, 0
0:00:00.714733094  1256 0xb281ae30 ERROR                    vpe gstvpebufferpool.c:603:gst_vpe_buffer_pool_set_streaming: Cant query buffers
0:00:00.715642401  1256 0xb281ae30 ERROR                    vpe gstvpebufferpool.c:603:gst_vpe_buffer_pool_set_streaming: Cant query buffers

How can I get the H264 decoding to work ? Thank you.

over 7 years ago

0 Margarita Gashova over 7 years ago

TI__Guru* 80190 points

Hello,

What is the PSDK version that you are using?
Are you able to play the file by using this pipeline?
gst-launch-1.0 playbin uri=file:////home/root/test.h264 video-sink=waylandsink

BR
Margarita

0 phil class over 7 years ago in reply to Margarita Gashova

Intellectual 640 points

Hi, Thanks for the reply.

I am using a Phytec SOM containing the AM5728, and their software release ( BSP-Yocto-TISDK-AM57xx-PD18.1.0 ). That release contains by default GStreamer 1.6.3 - I upgrade it to 1.12 in Yocto myself.

Kernel: $YOCTO_DIR/build/arago-tmp-external-linaro-toolchain/work/<MACHINE>-linux-gnueabi/linux-phytec-ti/4.4.91+git_v4.4.91-phy1-r7a/git/
- The device tree file to modify within the linux kernel source is: am572x-phycore-rdk.dts and its dependencies.
u-boot: $YOCTO_DIR/build/arago-tmp-external-linaro-toolchain/work/<MACHINE>-linux-gnueabi/u-boot-phytec/2017.01+git_v2017.01-phy1-r0/git/

It looks like playbin does not work either with the same error:

root@am57xx:~# GST_DEBUG=3 gst-launch-1.0 playbin uri=file:/home/root/xx.h264 video-sink=fakesink

Setting pipeline to PAUSED ...

0:00:00.195568440  1823    0xde620 WARN                 basesrc gstbasesrc.c:3480:gst_base_src_start_complete:<source> pad not activated yet

0:00:00.199679352  1823    0xde620 WARN                 basesrc gstbasesrc.c:3480:gst_base_src_start_complete:<source> pad not activated yet

Pipeline is PREROLLING ...

0:00:00.345973292  1823 0xb561b230 WARN                  ducati gstducatih264dec.c:415:gst_ducati_h264dec_set_sink_caps:<decoder> num-reorder-frames not found on caps, calculation from stream parameters gives 16

0:00:00.346095130  1823 0xb561b230 WARN                  ducati gstducatih264dec.c:423:gst_ducati_h264dec_set_sink_caps:<decoder> Using 0 frames for reordering

0:00:00.515939463  1823 0xb561b230 WARN                  ducati gstducatividdec.c:590:codec_process:<decoder> changing max-ref-frames in caps to 19

0:00:00.606052078  1823 0xb561b230 WARN                     vpe gstvpebufferpool.c:471:gst_vpe_buffer_pool_import: Allocating a new input buffer index: 0/128, 0

0:00:00.606463300  1823 0xb561b230 ERROR                    vpe gstvpebufferpool.c:603:gst_vpe_buffer_pool_set_streaming: Cant query buffers

0:00:00.607254673  1823 0xb561b230 ERROR                    vpe gstvpebufferpool.c:603:gst_vpe_buffer_pool_set_streaming: Cant query buffers

^Chandling interrupt.

Interrupt: Stopping pipeline ...

ERROR: pipeline doesn't want to preroll.

Setting pipeline to NULL ...

0 Margarita Gashova over 7 years ago in reply to phil class

TI__Guru* 80190 points

Hello,

I am sorry I was OoO.
We are supporting TI PSDK releases here.
Please try this pipeline for encoding:
gst-launch-1.0 filesrc location=/usr/share/ti/video/airshow_p352x288.yuv ! videoparse width=352 height=288 format=nv12 ! video/x-raw, width=352, height=288 ! ducatih264enc ! h264parse ! queue ! filesink location=/home/root/test.h264
This is for display
/etc/init.d/weston stop
gst-launch-1.0 playbin uri=file:///home/root/test.h264

BR
Margarita

0 Margarita Gashova over 7 years ago in reply to Margarita Gashova

TI__Guru* 80190 points

Hello,

Please if this answers your question click the "This resolved my issue" button.
If not, post a reply below or open a new thread if the thread has locked due to time-out.
Thank You!

BR
Margarita

0 phil class over 7 years ago in reply to Margarita Gashova

Intellectual 640 points

Hi Margarita,

Thank you, your example helped a lot - but I am now getting odd results on playback. When I run playbin, the video plays with the proper framerate and colors, but the edges of the frames are green and the left and bottom areas of the frames are clamped to the pixel value at the corresponding left and bottom regions of the good looking part of the frame. It is difficult to describe so I have included an image of the problem below:

I believe this has to do with print statements I am seeing from GStreamer indicating that it is cropping the frames:

0:00:12.396840784   957 0xb5d316c0 INFO                  ducati gstducatividdec.c:692:codec_process:<decoder> active frame region 32, 24, 384, 312, crop 352x288
0:00:12.653791490   957 0xb5d316c0 INFO                  ducati gstducatividdec.c:692:codec_process:<decoder> active frame region 32, 24, 384, 312, crop 352x288
0:00:12.657835871   957 0xb5d316c0 INFO                  ducati gstducatividdec.c:692:codec_process:<decoder> active frame region 32, 24, 384, 312, crop 352x288
0:00:12.663093583   957 0xb5d316c0 INFO                  ducati gstducatividdec.c:692:codec_process:<decoder> active frame region 32, 24, 384, 312, crop 352x288

I am using the following commands.

Encode:

gst-launch-1.0 filesrc location=/home/root/am5728-gst-tests/video-samples/airshow_p352x288.yuv ! videoparse width=352 height=288 format=nv12 ! vpe ! ducatih264enc ! h264parse ! queue ! filesink location=/home/root/test.h264 sync=false

Decode:

GST_DEBUG=4 gst-launch-1.0 playbin uri=file:///home/root/test.h264 fbdevsink device=/dev/fb1
 sync=false

How can I disable cropping and get the appropriate frame size from the test video? Thanks.

0 Margarita Gashova over 7 years ago in reply to phil class

TI__Guru* 80190 points

Hello,

Please try this:
/etc/init.d/weston stop
gst-launch-1.0 playbin uri=file:///home/root/test.h264

or

/etc/init.d/weston start
gst-launch-1.0 playbin uri=file:///home/root/test.h264 video-sink=waylandsink

BR
Margarita

0 Margarita Gashova over 7 years ago in reply to Margarita Gashova

TI__Guru* 80190 points

Hello,

I have tried on my side, I do not see the green lines.
Here are the pipelines that I have tested:

/etc/init.d/weston stop
gst-launch-1.0 playbin uri=file:///home/root/test.h264

gst-launch-1.0 filesrc location=test1.h264 ! queue ! h264parse ! 'video/x-h264' ! ducatih264decvpe ! 'video/x-raw,format=(string)NV12,width=352,height=288' ! kmssink

gst-launch-1.0 filesrc location=test1.h264 ! queue ! h264parse ! 'video/x-h264' ! ducatih264decvpe ! 'video/x-raw,format=(string)NV12,width=1280,height=720' ! kmssink

BR
Margarita

0 phil class over 7 years ago in reply to Margarita Gashova

Intellectual 640 points

It turns out that GStreamer version 1.6.3 has the green lines bug and the newer gstreamer versions ( 1.12 ) seem to work fine.

Thanks for the help on this.

Phil

0 Margarita Gashova over 7 years ago in reply to phil class

TI__Guru* 80190 points

Hello,

I am glad that the issue is solved.
Please if this answers your question click the "This resolved my issue" button.
If you have new questions/issues you could open a new topic.
Thank you!

BR
Margarita

Processors

Processors forum

Linux/AM5728: GStreamer H.264 decode error