PROCESSOR-SDK-AM57X: Matrix browser H.264 playback issue

Hello,

Please correct me if I am wrong but are you trying to play videos through the matrix-gui? If yes, please refer to the following project on how to launch gstreamer pipeline from the matrix-gui,

Regards,
Krunal 

Unfortunately not.

I am using ruby on rails, and locally run a puma server that renders the HTML5 code. On the other end I run matrix_browser to view the page.

Everything looks like working fine and embedded in HTML video player plays the files. The only problem is that it looks like GPU acceleration does not kick in, and the heavy load is being done by the CPU, and the cpu usage can go up to 100%.

I hope that makes more sense.

All best,

Kamil

Kamil,

It sounds to me like the underlying gstreamer pipeline isn't using the correct H.264 decoder, instead I suspect it's using the vanilla decoder that runs on the Arm and that's why you're seeing high CPU loading. On the AM57xx we don't use the GPU to accelerate video decode, although I suspect somebody could make that work, instead, we use a dedicated video decode acceleration block in the SoC called IVA-HD (sometimes referred to as ducati). TI doesn't directly expose the IVA-HD accelerator as a general purpose IP block that you can go in and program yourself. Instead we give you a gstreamer plug-in called ducatih264dec. This will offload the video decoding from the CPU almost completely.

I found this nice reference that has a bunch of example gstreamer pipelines several which use the ducati decoder.

Hi Brad,

Using gstreamer with kmssink is less ideal solution, but we do consider it, if there is nothing else we can do. I was actually able to get the desired size and position of the video using "kmssink render-rectangle" option. The problem is that we would have to render it on top of the GUI which at this moment is wayland/weston. When I try to run gst-launch-1.0 while Weston is running, I get: drmModeSetPlane failed: Permission denied (-13), error, probably because weston is holding up the graphics renderer(?)  which is something I do not fully understand. Another question regarding kmssink is, how would I tell gstreamer to pause the video if I need to? I do believe that maybe matrix_browser could be compiled(or somehow appointed) to use different gstreamer pipeline (in this case ducati) to render video. This would be ideal.

Best Regards,

Kamil Piekutowski

I'm pretty sure there's no connection between ducatih264dec and the kmssink. The key thing is to make sure your pipeline uses ducatih264dec rather than the generic one. You should be able to use any video sink you want.

I'm not an expert on gstreamer so not sure if there's a way to pause it.

Matrix should already be using the ducati version on the decoder, this is on an AM572x EVM correct?

If you look in /usr/bin you'll see the scripts Matrix invokes. Take a look at /usr/bin/runIvahdH264Dec.sh. If you run this on the command line, do you see low CPU usage?

Yes I have seen these scripts. Matrix_GUI is using command line to start gstreamer to play video in fullscreen mode and it uses waylandsink. Problem with waylandsink is that if the video is played in windowed mode, it appears at random position. 

Please note: Matrix_GUI and matrix_browser are related they are NOT the same thing.

I managed to use kmssink, but it requires Weston to be turned off, and it looks like videos cannot be paused. Unfortunately this is not an option.

Ideal solution is to use a regular web browser to play videos. I test the gui on chromium and on matrix_browser. 

It appears that none of the browsers take advantage of codec acceleration although both can play certain formats of videos.

Please note that demos supplied with the EVM only show that videos can indeed be played via gstreamer, but have very limited usage. 

Thank you,

-Kamil P.

Hello Kamil,

Based on the following thread, it does not seem like waylandsink supports window positioning. Also, I do not see an out of box example where matrix_browser/chromium uses IVA to perform video encode/decode. I will check also check internally and get back to you. I am out of office for the next few days and I will get back to you by mid next week.

Regards,
Krunal

Hi Krunal,

Yes you are correct, wayland does not support video positioning. I have mentioned it earlier. There is also no out of the box example to support playing video

directly in the browser using the codecs. Videos play, but use cpu. I have investigated that the answer will probably be in the qtwebkit. source code. I think qtwebkit that I have uses vanilla plugin

pipeline instead of the one with ducati  matrix_browser itself is just a wrapper around the qtwebkit. I suspect that there is a way to specify which gstreamer plugins to use in the bitbake recipe, but my

experience with it is very little, and I would not know which gst-plugins are the ones that support ducati plugins.

I hope this is any of help and I will keep digging,

-Kamil P.

Hi Kamil,

Please refer to the section "TI Provided Gstreamer Plugins" in the following document. It contains a list of plugins provide by TI for ducati decoding/encoding.

Regards,
Krunal

Krunal,

I already did. This does not resolve the issue. Once again, the examples there use gstreamer that either uses kmssink or waylandsink. None of these options are valid, because:

1. kmssink requires Weston to be turned off (meaning no GUI at all)

2. Weston/Wayland does not allow positioning.

Qtwebkit at the other hand allows very easy video sizing and positioning since it is handled by its html, css and java engine built into it. Only problem is that it does not accelerate video.

So I will ask a direct question, is there a way to build qtwebkit with gstreamer plugins that TI provides (especialym ducati one). If there is

does anyone know how. 

Best regards,

-Kamil P..

Hello Kamil,

I was out of office for the past few days and I apologize for the delayed response. I am still looking into the offloading the CPU usage and I will get back to you within 48 hours.

Regards,
Krunal

Hi Kamil,

Currently, I am exploring the ivi-shell and I am trying to figure out if there is a way to position/resize the window on the latest SDK. I will keep you posted on my results.

Regards,
Krunal 

Hi Krunal,

I have a question that may prevent waste of time. Is this going to eventually allow matrix_browser to decode video using hardware  or this is only workaround

to play video on top of gui. Also would playing, pausing, video and volume change be possible?

If NOT, then I have I better suggestion.

By looking into qtwebkit code I saw that there is implementation of media player, that uses gstreamer plugins.

It looks to me that if we could tell the compiler somehow to use ducatio gstreamer plugins instead of

whatever is using now, will solve the issue.

Everything else like, screen size and position is implemented in that code. 

FYI. Resizing the video is not a problem, it works. Positioning it in wayeland looks like a challenge, and not really supported.

Best regards,

Kamil