Part Number: DRA744
Other Parts Discussed in Thread: TVP5158
We have an analog TV decoder (for cameras) that produces raw video in YUYV format. The application uses QT, which in turn depends on gstreamer. To put video on the display, the stream has to be converted to a format acceptable to the sink element. We're experimenting with the 'qmlglsink' element (as an alternative to QtMultimedia), so that we can control the gstreamer pipeline used by the application. qmlglsink seems to use glimagesink internally (or at least it behaves in a very similar way). glimagesink accepts only RGBA format, so colorspace conversion is needed. Here is a gstreamer pipeline that closely simulates the behavior in our QT app:
gst-launch-1.0 -e videotestsrc ! 'video/x-raw, format=YUY2' ! glupload ! glimagesink sync=false
That basically works, but the problem is that it causes a very high level of CPU usage (about 130%, out of 200%, with 2 core processors). But this, on the other hand, doesn't cause the same problem:
gst-launch-1.0 -e videotestsrc ! glupload ! glimagesink sync=false
It takes only about 20% CPU.
So presumably most of the extra CPU usage is due to glcolorconvert (which is used implicitly for the YUY2 --> RGBA colorspace conversion, as shown by the gstreamer logs). But the CPU usage makes no sense, because the gl elements are supposed to offload this processing from the CPU to the GPU hardware, aren't they?
What are we doing wrong?
This is on TI Arago Yocto 3.1/dunfell, with Linux kernel 5.10. (By the way, qmlglsink / glimagesink do not work by default on this distribution. We needed several gstreamer patches before these pipelines would even start up at all. I can share them on request.)
