This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM3354: OpenGL/QT font rendering problem - corrupt glyphs

Part Number: AM3354

Greetings,

I am using AM3354 processor. I just did a major Yocto upgrade from "daisy" to "thud" branch (I am using meta-ti layer). The Qt version is 5.11.3.

Recently I noticed that sometimes a few characters/glyphs happen to be slightly corrupt. The problem occurs completely randomly, but when the same character is rendered multiple times on the screen, all of the characters are corrupted the same way. 

It looks like this:

I tried using various QT environment variable, namely:

  • QML_USE_GLYPHCACHE_WORKAROUND=1
  • QML_DISABLE_DISTANCEFIELD=1
  • QT_DISTANCEFIELD_DEFAULT_BASEFONTSIZE=54 (and other values)
  • QT_DISTANCEFIELD_DEFAULT_TILESIZE=64 (and other values)
  • QT_DISTANCEFIELD_DEFAULT_SCALE=16 (and other values)
  • QT_DISTANCEFIELD_DEFAULT_RADIUS=80 (and other values)
  • QT_DISTANCEFIELD_HIGHGLYPHCOUNT 2000 (and other values)
  • QSG_ANTIALIASING_METHOD=vertex (tried also msaa)

, but none of them worked. The only thing that worked was to enable software rendering by setting QT_QUICK_BACKEND=software. However, this obviously led to a very significant slowdown, so it's not an option.

We're not using X11 or wayland. We're not modifying the graphics drivers or QT layer in any way. When launching, we're setting the following environment variables:

  • QT_QPA_EGLFS_INTEGRATION=none
  • QT_QUICK_CONTROLS_1_STYLE=Flat
  • QT_QPA_EGLFS_NO_LIBINPUT=1

Qt configure summary is attached.

qt_do_config.txt
Configure summary:

Building on: linux-oe-g++ (arm, CPU features: neon) Building for: linux-oe-g++ (arm, CPU features: neon)
Configuration: cross_compile compile_examples enable_new_dtags largefile neon silent shared release c++11 c++14 c++1z concurrent dbus reduce_exports stl no-widgets Build options:
  Mode ................................... release
  Optimize release build for size ........ no
  Building shared libraries .............. yes
  Using C++ standard ..................... C++1z
  Using ccache ........................... no
  Using gold linker ...................... no
  Using new DTAGS ........................ yes
  Using precompiled headers .............. no
  Using LTCG ............................. no
  Target compiler supports:
    NEON ................................. yes
  Build parts ............................ libs tools Qt modules and options:
  Qt Concurrent .......................... yes
  Qt D-Bus ............................... yes
  Qt D-Bus directly linked to libdbus .... yes
  Qt Gui ................................. yes
  Qt Network ............................. yes
  Qt Sql ................................. yes
  Qt Testlib ............................. yes
  Qt Widgets ............................. no
  Qt Xml ................................. yes Support enabled for:
  Using pkg-config ....................... yes
  udev ................................... yes
  Using system zlib ...................... yes Qt Core:
  DoubleConversion ....................... yes
    Using system DoubleConversion ........ no
  GLib ................................... no
  iconv .................................. no
  ICU .................................... yes
  Tracing backend ........................ <none>
  Logging backends:
    journald ............................. no
    syslog ............................... no
    slog2 ................................ no
  Using system PCRE2 ..................... no Qt Network:
  getifaddrs() ........................... yes
  IPv6 ifname ............................ yes
  libproxy ............................... no
  Linux AF_NETLINK ....................... yes
  OpenSSL ................................ yes
    Qt directly linked to OpenSSL ........ no
  OpenSSL 1.1 ............................ yes
  SCTP ................................... no
  Use system proxies ..................... yes Qt Gui:
  Accessibility .......................... yes
  FreeType ............................... yes
    Using system FreeType ................ yes
  HarfBuzz ............................... yes
    Using system HarfBuzz ................ no
  Fontconfig ............................. no
  Image formats:
    GIF .................................. yes
    ICO .................................. no
    JPEG ................................. yes
      Using system libjpeg ............... yes
    PNG .................................. yes
      Using system libpng ................ yes
  EGL .................................... yes
  OpenVG ................................. no
  OpenGL:
    Desktop OpenGL ....................... no
    OpenGL ES 2.0 ........................ yes
    OpenGL ES 3.0 ........................ no
    OpenGL ES 3.1 ........................ no
    OpenGL ES 3.2 ........................ no
  Vulkan ................................. no
  Session Management ..................... no Features used by QPA backends:
  evdev .................................. yes
  libinput ............................... yes
  INTEGRITY HID .......................... no
  mtdev .................................. no
  tslib .................................. no
  xkbcommon-evdev ........................ no QPA backends:
  DirectFB ............................... no
  EGLFS .................................. yes
  EGLFS details:
    EGLFS OpenWFD ........................ no
    EGLFS i.Mx6 .......................... no
    EGLFS i.Mx6 Wayland .................. no
    EGLFS RCAR ........................... no
    EGLFS EGLDevice ...................... no
    EGLFS GBM ............................ yes
    EGLFS VSP2 ........................... no
    EGLFS Mali ........................... no
    EGLFS Raspberry Pi ................... no
    EGL on X11 ........................... no
  LinuxFB ................................ no
  VNC .................................... yes
  Mir client ............................. no Qt Widgets:
  GTK+ ................................... no
  Styles ................................. Fusion Windows Qt PrintSupport:
  CUPS ................................... no Qt Sql:
  DB2 (IBM) .............................. no
  InterBase .............................. no
  MySql .................................. no
  OCI (Oracle) ........................... no
  ODBC ................................... no
  PostgreSQL ............................. no
  SQLite2 ................................ no
  SQLite ................................. no
    Using system provided SQLite ......... no
  TDS (Sybase) ........................... no Qt Testlib:
  Tester for item models ................. yes

Note: -hostbindir is not a subdirectory of -hostprefix.
Note: -hostlibdir is not a subdirectory of -hostprefix.
Note: -hostdatadir is not a subdirectory of -hostprefix.

WARNING: Feature xinput2 is insignificant in this configuration, ignoring related command line option(s).

WARNING: Feature xkb is insignificant in this configuration, ignoring related command line option(s).

WARNING: Feature xkbcommon-system is insignificant in this configuration, ignoring related command line option(s).

Qt is now configured for building. Just run 'make'.
Once everything is built, you must run 'make install'.
Qt will be installed into '/home/nws/wsl/arago/ggl-yocto-tools/build/arago-tmp-external-arm-toolchain/work/armv7at2hf-neon-linux-gnueabi/qtbase/5.11.3+gitAUTOINC+08de243eaa-r0.arago16/recipe-sysroot/usr'.

I have spent quite enough time trying to find the solution. I found quite a few Qt bugs that were similar to my issue (e.g. https://bugreports.qt.io/browse/QTBUG-65496), but the problem seems to be more OpenGL related than Qt related.

I can provide whatever information is needed.

Has anyone experienced similar problem? Is there any configuration detail I might be missing?

Regards,

Ondrej

  • Hello Ondrej,

    Are you experiencing the same problem using OpenGL ES? Here is an example guide I found online: https://learnopengl.com/In-Practice/Text-Rendering. Also, as mentioned in the following document, Qt is part of the open source software that gets integrated in our SDK but TI does not own, maintain and support all the FOSS(free and open source software). With regards to Qt bugs, please reach out to the Qt community.  

    Regards,
    Krunal

  • Hi Krunal,

    I am pretty sure this is not a Qt issue. Even if it was Qt issue, I'm not modifying Qt or graphics drivers in any way. I fully understand that TI does not maintain Qt.  I am using the default configuration together with patches supplied by meta-arago layer, which is - as you mentioned - integrated in your SDK.

    Besides that, I tried various versions of Qt and even the long-term support "5.12.6" from "warrior" branch doesn't do the trick.

    Any suggestions what can I look into?

    Thank you,

    Ondrej

  • Hello,

    I am wondering if you have tried running the out of example examples provided by Qt (/usr/share/qt5/examples/quick/text). I do not observe any text issues on my setup. Also, have you tried running your test on a TI EVM (BBB/AM335x EVM)?

    Regards,
    Krunal

  • Hi Krunal,

    Yes, I have a minimal reproducible example. I created a small application that just prints a lot of text on the screen. The labels have various font sizes and colors and the text changes every few seconds. Every time I run the application, there is at least a single character that is corrupt. It seems that the smaller the font is, the higher chance is it will have a corrupt character - usually it's the first line that gets artifacts, but in some runs also the bigger labels are affected.

    The source code is here: https://github.com/pohlondrej/OpenGlTextCorruptionTest

    I don't understand your "TI EVM" question. The board I'm using is derived from AM335x EVM design, but even if it didn't, it uses the exact same chip with the exact same GPU, with the exact same GPU driver. How would the custom board design affect the GPU?

    Can you please try the application out with your setup?

    Regards,

    Ondrej

  • Hi Ondrej,

    I just ran your test on a Beagle Bone Black using PSDK6.0. On my setup, I ran the test with Weston and EGLFS but I was not able to replicate you issues.

    Regards,
    Krunal

  • Hi Krunal,

    Thank you very much for trying. May I ask you what does your environment look like (what environment variables do you have and what are the values), what branch of TI SDK are you using (thud, zeus) and what git revision are you on? If you could give me this information also for u-boot and kernel, that would be very helpful. Also, do you have an output from Qt configure? That would also help me a lot.

    Kind regards,

    Ondrej

  • Hello Ondrej,

    Please refer to the following software manifest for details. Here are the environment variables:

    root@am335x-evm:~# cat /etc/qt5/eglfs_kms_cfg.json
    {
      "device": "/dev/dri/card0",
      "hwcursor": false,
      "pbuffers": true,
      "outputs": [
        {
          "name": "VGA1",
          "mode": "off"
        },
        {
          "name": "HDMI1",
          "mode": "1024x768"
        }
      ]
    }
    
    root@am335x-evm:~# cat /etc/profile.d/qt_env.sh 
    #!/bin/sh
    
    ### QT Environment Variables ###
    export QT_QPA_EVDEV_TOUCHSCREEN_PARAMETERS="rotate=180"
    export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
    export QT_QPA_EGLFS_KMS_CONFIG=/etc/qt5/eglfs_kms_cfg.json
    export QT_QPA_EGLFS_INTEGRATION=none
    export QT_WAYLAND_SHELL_INTEGRATION=wl-shell
    root@am335x-evm:~# 

    Regards,
    Krunal

  • Hi Krunal,

    We stopped this conversation because the answers we received from you were not very helpful. See, the answer "it works for us" is really easy for you, but it's not going to help your customers.

    We already tried the configuration above, even before you sent it to us. It had no effect, the issue is still there. In my last response I asked whether you can provide me the following things:

    • environment variables and their values (the only answer we actually received)
    • branch and commit of TI SDK that you are using (rocko, thud, zeus)
    • branch and commit of bootloader (u-boot) and kernel
    • output of Qt configure

    But now, we were able to reproduce the issue using official TI AM335x ARM Microprocessors Evaluation Module. Without any modifications, I downloaded the official SDK from http://software-dl.ti.com/processor-sdk-linux/esd/AM335X/latest/index_FDS.html and followed instructions here: http://software-dl.ti.com/processor-sdk-linux/esd/docs/06_01_00_08/linux/Overview_Getting_Started_Guide.html#linux-sd-card-creation-guide to create a bootable SD card. Then I inserted the SD card and started the device. It looked fine at the first look, but after launching a few examples I managed to get this screen:

    https://ibb.co/DRqmZGq

    As you can see, the “C” characters are all completely garbled. This means that not only there is a problem in the official TI SDK, but you did a very poor job at testing our example code. It does not even matter, because the official examples are also broken. It took about 20 restarts of the example application to get this result - literally 2 minutes. This is the exact same problem we are observing on our custom solution.

    I am not saying the problem is not in Qt or other third party, but since these are used and integrated in the SDK and we were told multiple times that everything is working properly and the problem is on our side. No, the problem is now on your side, because even the official TI EVM kit is not working with the new software!

    We spent over 5 weeks trying to find a problem in our configuration, mainly because we were reassured several times by you that everything works on your side. Whether this is a lie or a plain oversight, this investigation was already pretty expensive on our side. I hope you understand that and that you will fix the issue promptly.

    Sorry for the dramatic tone, but we are very disappointed about the situation and about the lack of proper support.

    Kind regards,
    Ondrej

  • Hello Ondrej,

    I apologize for the miscommunication but the software manifest contains all the details about our SDK and the following page contains a brief summary. I am wondering if you could please attach your corrupted picture again because I am not able to view the picture. Also, here is how I initiated your test:

    root@am335x-evm:~/OpenGlTextCorruptionTest# ./FontRenderingTest -platform eglfs
    Unable to query physical screen size, defaulting to 100 dpi.
    To override, set QT_QPA_EGLFS_PHYSICAL_WIDTH and QT_QPA_EGLFS_PHYSICAL_HEIGHT (in millimeters).
    Loading font family from "/home/root/OpenGlTextCorruptionTest/bin/release/fonts/Roboto.rcc"
    4 loaded.
    Loading font ":font/Roboto/Roboto-BoldItalic.ttf"
    Loading font ":font/Roboto/Roboto-Italic.ttf"
    Loading font ":font/Roboto/Roboto-Regular.ttf"
    Loading font ":font/Roboto/Roboto-Bold.ttf"
    BANANA1
    BANANA2

    I have also attached a video of my test after the 20th run and please let me know if I am overlooking anything from my side.

    I have also ran the "/usr/share/qt5/examples/quick/text" on the AM335x EVM using PSDK6.1 and I do not see any issues after the running twenty times. 

    Regards,
    Krunal

  • Hi Krunal,

    The link to the image is here and it is working for me: https://ibb.co/DRqmZGq ... anyways, attaching the image here just for completeness:

    Also, I found the bug right in your video. The "%" sign is not displaying correctly. For the lack of better options I circled it in MS paint:

    I hope this makes things more clear.

    Kind regards,
    Ondrej

  • Hello Ondrej,

    I am wondering if you have a sample application that shows the bug consistently. I have been running your application multiple times but it is very difficult to find the glyph corruption.

    Regards,
    Krunal

  • Hi Krunal,

    Yes, we also were not able to find the bug consistently. Yes, the testing is very time consuming. It occurs in about 30% of cases and it is usually mild case, so you have to observe the characters carefully. I generally do these steps:

    1. Run the FontRenderingTest application
    2. Carefully observe the first row - it has the highest number of characters, so there is a higher chance spotting a corruption
    3. If I don't see any corruption after 5 screen refreshes (after 50 seconds), I restart the application.
    4. Do steps 1-3 until I see a text corruption

    However, I was able to reproduce the bug with 100% success every single run with the "stocqt" example application. Steps to reproduce:

    1. Run /usr/share/qt5/examples/quick/demos/stocqt/stocqt
    2. Touch/swipe the screen to trigger transition to another screen
    3. One or more characters become corrupted

    I am attaching the image of the screen running the StocQt application once again, just for completeness:

    Please let me know if you need further explanation. Fixing this issue is our #1 priority, because it is blocking our software release.

    Thank you,
    Ondrej

  • Hi Ondrej,

    After further testing, we are consistently able to replicate the corruption on our side. We have identified an issue with the GPU driver and we are still investigating it. I will keep you posted on any updates.

    Regards,
    Krunal  

  • Hi Krunal,

    This is good news! Thank you for the update. Looking forward hearing from you soon!

    Kind regards,
    Ondrej

  • Hello Ondrej,

    I just wanted to update you that we are still debugging the text corruption and it is being treated with a high priority. Eddie has mentioned me about your deadlines and we will keep you posted on further updates.

    Regards,
    Krunal 

  • Hi Krunal,

    Thank you for keeping us updated! I am very glad this is a priority task.

    Kind regards,
    Ondrej

  • Hi Krunal,

    Are there any updates on this issue?

    Thank you,
    Ondrej

  • Hi Ondrej,

    Currently, we are engaged with our GPU vendors and still working towards to a solution. However, due to COVID-19, please except some delays.

    Regards,
    Krunal 

  • Hi Ondrej,

    The debug is still on-going and the GPU vendor needs specific access to hardware to debug. Due to the global situation, this access cannot be attained at this time and will not change until government restrictions change. As this timeline is undefined and outside of our control, we cannot provide a specific schedule.

    Regards,
    Krunal

  • Hi Krunal,

    Do you mean Imagination needs access to AM335x-EVM for PowerVR GPU? I thought they would be having the particular GPU with them because you said they were able to reproduce the issue.

    But thanks for keeping us posted.

    regards

    Ankur

  • Hi Ankur,

    At TI, we have been able to reproduce the issue (thanks to Ondrej's application) and we have been collecting data and sharing with Imagination.  Due to the current conditions, their engineers are limited by the test setup and I will keep you guys posted on any updates.  

    Regards,
    Krunal

  • Hi Krunal,

    Are there any news regarding this issue? It has been more than 6 weeks since you were also able to identify and reproduce it. What happened during these 6 weeks? We understand there is a global pandemic right now, but from our point of view it looks like no progress has been made at all, even though it is a strictly software problem. If there are software repositories and proper design documents, a lot can be done even without access to specialized hardware.

    Thank you,
    Ondrej

  • Hello Ondrej,

    We have been actively debugging the issue with the IP vendor since the last the few weeks. Unfortunately, there is no solution available yet and I will keep you posted as soon as I get an update. A similar issue has been reported by another customer and the issue is being treated with a high priority. 

    Thank you for your patience.

    Regards,
    Krunal

  • Hello Ondrej,

    The status has not changed and we are still continuing the debug discussions. 

    Regards,
    Krunal 

  • Hello Ondrej,

    Imagination (GPU vendor) has given us a work around for the corruption issue and I have not been able to replicate the issue. I have attached the updated binaries and I am wondering if you could please test them on your side. They have also confirmed that the work around does not have any impact on the performance and please let us know if you see any issues.

    Thank you for your patience.

    Regards,
    Krunal

    am335x-test-ddk.tar.gz

  • Hi Krunal,

    Thank you for the update. Do you have any instructions on how to install the binaries? By simply copying them on the target device to corresponding folders I get the following error:

    PVR:(Error): LoadCompilerModule: Couldn't load library libglslcompiler.so [0, ]
    QOpenGLShader::compile(Vertex): failed
    *** Problematic Vertex shader source code ***
    #line 1
    attribute highp vec4 vertexCoord;
    attribute highp vec4 vertexColor;
    
    uniform highp mat4 matrix;
    uniform highp float opacity;
    
    varying lowp vec4 color;
    
    attribute highp float _qt_order;
    uniform highp float _qt_zRange;
    void main()
    {
        gl_Position = matrix * vertexCoord;
        color = vertexColor * opacity;
        gl_Position.z = (gl_Position.z * _qt_zRange + _qt_order) * gl_Position.w;
    }
    ***

    Thank you,
    Ondrej

  • Hello Ondrej,

    Could you please confirm which PSDK/Kernel version you are using for your development? Based on the release, I will work with our driver team to provide the updated libraries.

    Regards,
    Krunal

  • Hi Krunal,

    Our kernel version is 4.19.73. In our build we are using meta-ti linux kernel from "thud" branch. In short:

    Thank you,
    Ondrej

  • Hi Ondrej,

    Based on my internal discussion, the attached binaries are based on the "dunfell" release. If you would like the binaries for the "thud" branch, please give us 48-72 hours. 

    Regards,
    Krunal

    2577.am335x-test-ddk.tar.gz

  • Hi Krunal,

    yes, please! We need the binaries for "thud", as this is the version we're currently working on.

    Thank you,
    Ondrej

  • Hi Ondrej,

    I have attached the binaries and please copy the two folders to your existing rootfs.  I have tested the binaries on Beagle Bone Black running PSDK6.1 (close to your setup). I have also tested your application+Qt stock app and I have not seen the corruption. Please confirm and let us know if you run into any problems.

    Regards,
    Krunal


    am335x_gpu_images.tar.gz