This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

[FAQ] AM625: Is Flutter the right GUI toolkit for me: an engineers perspective.

Part Number: AM625
Other Parts Discussed in Thread: SK-AM62

In today’s world, people tend to spend more time interacting with machines through a display screen rather than controlling them mechanically. For example, in automotive vehicles or home appliances, there is a graphical user interface (GUI) running on the main screen. Through the GUI, one can control devices and activate features such as music, radio, phone, temperature control and etc. Similarly, at the airports or restaurants, the public can communicate with the respective companies through a self-serving kiosk. The kiosk runs a GUI on a display device and through touchscreen, machine can fulfil the customer’s request. In summary, more trends are being observed in the industry wherein the end consumer can interact with GUIs in a convenient and efficient manner to accomplish tasks.

From a developer’s perceptive, creating an elegant GUI is vital because it is one of the key components of communication between the customer and the products. When it comes to running GUIs on an embedded device, there are many GUI frameworks to choose from. Each framework is unique in its own ways and needs to support some of the following features (the list below is by no means a comprehensive list):

  1. Touch: Most of the products will connect a display that will have resistive or capacitive touch support.
  2. Aesthetics: GUIs with a more modern appearance or user friendly features will attract more customers.
  3. Hardware (HW) acceleration: Several embedded devices have a graphics processing unit (GPU) and it is capable of performing various tasks like alpha blending, color conversion, scaling and rotation. It is crucial that the GUI utilizes the GPU for rendering complex scenes and offloads the central processing unit (CPU) for other tasks.
  4. Memory: Developers are working with a limited memory footprint so they need to ensure the GUI and their software stack fits within the required footprint.
  5. Scalability: Depending on the project, developers prefer to have a GUI that is portable across multiple platforms. Having a framework that scales across multiple devices and operating system(OS) can be beneficial.
  6. License: Software licensing is a key aspect to consider. 

How to Enable Flutter on the AM6254 ARM Processor Family

Though choosing a GUI framework is challenging, enabling and optimizing the framework on an embedded platform are more demanding tasks; the objective of the current article is to assist developers in enabling and optimizing the Flutter framework on TI platforms. With TI’s next generation device (AM625), various modern GUIs have already been enabled and for more information, please refer to the following page:https://dev.ti.com/tirex/content/am62_design_gallery_01_00_00_00_eng__all/landing_page.html. TI devices and the software development kit (SDK) offer flexibility to the developers to choose from any framework.

Flutter, created by Google, has recently gained traction and has support for touch, includes contemporary GUI, contains smaller memory stack and has a less stringent licensing clause. If a developer is interested in Flutter and wants to develop on TI’s embedded platform, the process will be described in the steps below:

Steps: 

1)	Pre-Req: https://software-dl.ti.com/processor-sdk-linux/esd/AM62X/08_05_00_21/exports/docs/linux/Overview_Building_the_SDK.html#prerequisites-one-time-setup
2)	git clone https://git.ti.com/git/arago-project/oe-layersetup.git tisdk
3)	cd tisdk
4)	cp configs/processor-sdk/processor-sdk-08.05.00.21-config.txt configs/flutter-config.txt
5)	echo "meta-flutter,https://github.com/ardera/meta-flutter.git,dunfell,HEAD" >> configs/flutter-config.txt
6)	./oe-layertool-setup.sh -f configs/flutter-config.txt
7)	cd build/
8)	export TOOLCHAIN_PATH_ARMV7=$HOME/gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf
9)	export TOOLCHAIN_PATH_ARMV8=$HOME/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu
10)	echo 'IMAGE_INSTALL_append = " flutter-pi-runtimedebug flutter-gallery-runtimedebug "' >> conf/local.conf
11)	echo 'TOOLCHAIN_HOST_TASK:append = " nativesdk-flutter-sdk"' >> conf/local.conf
12)	echo 'FLUTTER_SDK_TAG = "stable"' >> conf/local.conf
13)	echo 'SRCREV_pn-flutter-gallery-runtimedebug = "9776b9fd916635e10a32bd426fcd7a20c3841faf"' >> conf/local.conf
14)	. conf/setenv
15)	MACHINE=am62xx-evm bitbake-layers add-layer ../sources/meta-flutter
16)	MACHINE=am62xx-evm bitbake tisdk-default-image
17)	Flash SD card with "arago-tmp-external-arm-glibc/deploy/images/am62xx-evm/tisdk-base-image-am62xx-evm.wic.xz" image
18)	Run “flutter-pi /usr/share/flutter/gallery/” on EVM

How to optimize Flutter for higher rendering

Investigation

When investigating the performance of flutter on the SK-AM62 board, initially the performance was relatively poor. It was at about 10-15FPS in the example KDAB cruise demo app. When profiling with the very helpful PowerVR tools, the GPU was actually not that busy, so the problem had to come from elsewhere.

In CPU profiling, Skia (which flutter uses for rendering) spent a lot of time in some software rendering functions. Further debugging revealed Skia enabled a lot of workarounds for OpenGL driver bugs when it discovers it’s running on a PowerVR GPU, which mainly seem to have been in place for older drivers/GPUs and are obsolete for this board. That’s, at least, what testing without the workarounds was showing.

More precisely, skia’s context setup looked like this:

// We found that on Wembley devices (PowerVR GE8320) that using tessellation path renderer would
// cause lots of rendering errors where it seemed like vertices were in the wrong place. This
// led to lots of GMs drawing nothing (e.g. dashing4) or lots of garbage. The Wembley devices
// were running Android 12 with a driver version of 1.13. We previously had TecnoSpark devices
// with the same GPU running on Android P (driver 1.10) which did not have this issue. We don't
// know when the bug appeared in the driver so for now we disable tessellation path renderer for
// all matching gpus regardless of driver version.
if (ctxInfo.renderer() == GrGLRenderer::kPowerVRRogue) {
   fDisableTessellationPathRenderer = true;
}


// The Wembley device draws the mesh_update GM incorrectly when using transfer buffers. Buffer
// to buffer transfers affect draws earlier in the GL command sequence.
// Android API: 31
// GL_VERSION : OpenGL ES 3.2 build 1.13@5720833
// GL_RENDERER: PowerVR Rogue GE8300
// GL_VENDOR  : Imagination Technologies
if (ctxInfo.renderer() == GrGLRenderer::kPowerVRRogue) {
   fTransferFromBufferToBufferSupport = false;
}


// The Wembley device fails shader compilations with no error message when there is a const
// parameter. Given that we've already passed through SkSL compilation and enforced that the
// parameter is never written, it is harmless to strip the const off when writing GLSL.
// Android API: 31
// GL_VERSION : OpenGL ES 3.2 build 1.13@5720833
// GL_RENDERER: PowerVR Rogue GE8300
// GL_VENDOR  : Imagination Technologies
if (ctxInfo.renderer() == GrGLRenderer::kPowerVRRogue) {
   fShaderCaps->fRemoveConstFromFunctionParameters = true;
}

The workarounds which has the largest impact on performance here is the first one. Apparently, the tessellation path renderer is the one that usually renders text; which meant that in this case, all text was drawn using software rendering.

 Initially, developed a custom patch in the yocto recipe that disables those workarounds, but the issue was reported to the flutter team and it was addressed very quickly, so the most performance-critical workaround is now also disabled upstream as of 3.7.0.

 App-specific performance improvements

When optimizing flutter apps for the AM62xx platform, one has to worry about the GPU. While the raw compute performance of the GPU is okay, the memory bandwidth is fairly limited; which is mostly where the power efficiency of the GPU comes from. So when developing flutter apps for the AM62xx, avoid memory-bandwidth expensive rendering.

Generally, Imagination Technologies provide pretty good guidelines regarding the performance optimizations on PowerVR hardware. Some of them might not apply, because flutter hides most of the actual low-level details, but some are good to know when developing flutter apps too:

Avoid alpha blending, if possible.

  • prefer alpha-blending any images before-hand (if the background is known).
  • If alpha-blending needs to be done, reduce it to a minimum.
  • For example, clip textures to their visible pixels.
  • There’s a tool that’ll soon be published as part of https://github.com/KDAB/cabin-demo that’ll help in this case.
  • All flutter images are drawn with alpha-blending enabled, by default. If you have lots of completely opaque images, it could be beneficial to draw them without alpha-blending, by specifying src instead of the default BlendMode.srcOver. That, unfortunately, can not be achieved with the default Image widget, so one would have to implement a custom image widget for that or use a CustomPainter.

Avoid textures / images, if possible.

  • sometimes a texture can instead be replaced by some Vertices, or a Path and be drawn using a CustomPaint instead
  • solid-color triangles save bandwidth because the GPU doesn’t have to fetch the texture from memory.
  • memory bandwidth requirements can also be worse for different texture interpolation techniques. Nearest-neighbor is the most efficient interpolation technique in that regard.

Conclusions

GUIs are an essential aspect of interaction with machines. Selecting an appropriate GUI can be problematic but regardless of the GUIs chosen by the developer, TI’s next generation device, AM62x, is capable of running any modern GUIs.

Additional resources

  • TI collaborated with KDAB to enable Flutter. KDAB will also be showcasing a Flutter demo at Embedded World 2023 at the KDAB booth Hall 4-302. For more information, visit: https://kdab.group/
  • Find an AM62 processor for your design.