The Challenge of Kivy on Linux Mobile

On a previous blog I discussed my logic in selecting a crossplatform mobile development framework.

Even though I chose Kivy as I was fairly confident I was going to be able to get it to work for Linux mobile, I knew there would be issues due to me working in uncharted waters.

I am going to briefly go over the issues I tackled in hopes that others will not be caught flatfooted when they encounter one of these alligators.

This article is written about Kivy versions 2.2.0-2.3.0. These issues may be addressed in future versions of Kivy, Phosh, or Wayland.

Unexpected Duplicated Touch Events

One of the first issues I encountered was the different way touch events were registered on Linux touch screens versus Android touch screens.

Every Linux touch screen I have encountered (4 in total) have all registered duplicate touch events. An internet search revealed other people encountering this issue on Linux but I never saw a solution.

This issue can be programmed around in several ways. However if your app is sufficiently complex some of these solutions may interfere with your ability for the correct widget(s) to process your touch event as events have a one to many relationship in Kivy.

Softkeyboard Incompatiblities

A pre-beta version of Muscle Buddy used keyboard_mode: 'auto' on TextInput widgets. This is the setting you use to allow the OS to automatically map soft and hard keyboard events to your app input fields. That worked pretty well for Android but it was a disaster on Linux mobile.

OS or third party software keyboard support for Linux in Kivy was just not well implemented. The main software keyboard I was working with at the time was Squeekboard, the software keyboard used on systems using the Phosh desktop. Squeekboard screwed up the screen rendering and touch screen calibration when ever it was activated.

I resorted to disabling Squeekboard when Muscle Buddy was running to prevent users from encountering compatibility problems.

But Muscle Buddy needed a software keyboard. This feature was a necessity.

I ended up using Kivy’s VKeyboard widget and setting all TextInput widgets to keyboard_mode: 'managed' and wrote custom keyboard layouts and a custom input system for the app. I like to call it the Mighty Morphin Muscle Keyboard (MMMK for short) as the systems changes layouts based on your input context.

I am sure some people would prefer to use their OS’s native keyboard. I am not opposed to adding the code necessary to support another keyboard system. However it was not necessary to get Muscle Buddy to a beta release so I left it as a “nice to have” back burner project.

Rumbler Support was Absent

I like knowing when I have pressed a button on a mobile device, so if the device has a rumbler/vibrator I want to be able to use it to provide haptic feedback.

Kivy uses Plyer to control the vibrator on Android and iOS, but it does not support Linux.

So to work around this deficiency one can locate the gpio_vibrator device file in the /dev/input directory and interact with it. This can be achieved using the evdev Python module.

The end result was Android and Linux mobile provide seemingly identical haptic responses from the perspective of the app user.

Image Scaling Problems

I still do not have a good feel for Kivy’s render engine. Kivy has a render engine sort of like the way your favorite web browser has a HTML engine. One builds GUIs by layering widgets on top or inside of each other with relative or exact scaling parameters to generate a UI. GUI design and development has always been a challenge for me so creating the modest GUI Muscle Buddy possesses took a significant effort.

In the end I scaled many of the widgets from the mobile device’s screen height and width. This allowed me to give an interface that should work well in portrait mode on most devices. But display width and height on Linux mobile does not behave the same way it does on Android. Width and height on Linux mobile changes while the app is starting. Upon start on Linux the width x height always reports as 800x600 regardless the real screen dimensions and later corrects itself. The incorrect dimensions interferes with my efforts to properly scale widgets when starting the app.

My solution for this is a bit of a hack. The device goes through a timed screen calibration stage and then stores the device’s resolution once it has stabilized and uses it in future application starts. This means that the app has to work harder on its first start than it does on subsequent starts.

That works for the devices I have access to, but it can break if run on a device slow enough so that the calibration occurs before the resolution has stablized. I expect the PinePhone to be the slowest device that Muscle Buddy will likely ever be run on and the calibration code has enough time to execute on it.

I’m open to better solutions but this approach works for most devices so for now I am going with it.

Touch Screen Calibration Issues

From my experience, Kivy apps typically work best on Linux mobile running in fullscreen mode. If it is sharing screen real-estate with Phosh, touch events will end up registering off target.

On Linux, Kivy triangulates the position of the touch device events and maps them to there relative location on the screen. But when the app is not in fullscreen mode, the touch events may not correctly map to the location of the graphic widget being visibly touched.

The below image illustrates this problem. Phosh has taken the top and bottom screen real-estate with Muscle Buddy running. Notice the red X. When Phosh is taking up screen real-estate pressing the 0 key can register as pressing the 5 key. One would have to reach lower and higher on the screen to touch their chosen widget when this is occurring.

I currently do not have a good solution for this problem. This occurs on a Phosh device when the user swaps to another app and then swaps back. When Muscle Buddy was in pre-beta this problem could be fixed by calling Kivy’s Window.maximize() method, but that strategy stopped working after some system update.

The best solution for a Phosh user is to not swap between apps when running Muscle Buddy. That works for my personal use, but I hope to find a programming solution for this problem some day.

Summary

So what is the score?

problem	solved	notes
duplicate touch events
softkeyboards incompatible
rumbler support lacking
image scaling problems		Keeping my eye out for a more elegant solution
touch screen calibration issues		The user can avoid the problem with some discipline

My solutions can be found by reading through Muscle Buddy’s source code. In the future I may write detailed blogs detailing how to write code to solve these problems for the sake of simplifying these issues for the enterprising Linux mobile programmer.

Do I think Kivy is the best framework for writing apps to run on Linux mobile? Not at all. There are other GUI frameworks that would work better. But if you want your app to work on Android, iOS, Linux mobile, and Windows tablets; I still know of no better choice.

Now go forward and make something awesome!

Enjoy!