Random Thoughts

This is part two of a series of blog posts that I’m sure is going to go on forever: Wherein I Bitch About Wayland And Modern Life In General.

So, last time, I finally bit the bullet and moved my travelling Yoga laptop to Wayland. This time around, I’m trying to get one more piece of the puzzle to work: The onscreen keyboard.

But first of all, I should perhaps explain my use case, because otherwise this will all seem incomprehensible. I mean, even more.

When I’m travelling, I bring a Lenovo Yoga laptop with me, because it’s so handy to watch movies on (in hotels and on planes etc). I guess most people solve this by bringing a tablet of some kind, but I’ve never found anything as convenient as just using one of these laptops. (Your mileage will vary.) Since the physical keyboard isn’t available in the “tent” position like up in the picture there, I’ve implemented a simple touch interface for Emacs (based on libevdev). Which means that I can just touch virtual squares on the screen to do various stuff:

Here’s the same view, but with labels on the squares to remind me where the various functions are.

Similarly, then actually playing the movie (with mpv), I have a whole bunch of convenient functions available by touching various squares…

… here with the grid shown with labels.

And all this works fine in Wayland, too, since it doesn’t really talk to Wayland stuff at all — it just pokes around in the Linux evdev hierarchy to get the events.

So what’s the problem? The problem is that sometimes I really, really need a keyboard, like when searching for things.

Now, all desktop environments have onscreen keyboards (or as some systems call them, “virtual keyboards” or the like). And… that’s the problem!

That is, there is no One True Onscreen Keyboard — instead they’re implemented for each specific desktop system, and the result is that they’re all lacking in features.

In the Good/Bad Olden Days, if people implemented a thing, they’d make sure that it worked for Linux, Macos, Windows, FreeBSD, whatever, and they’d slowly accrue a lot of features and become pretty useful. These days, a thing like this is implemented as a plugin — we’re here seeing the Onboard keyboard for Gnome Shell. But, as is always the case with these things:

Since it’s tied to Gnome Shell and uses internalish Gnome Shell features, it needs to be updated constantly as Gnome Shell changes. (And they usually aren’t.)

But if all these desktops have onscreen keyboards, why does it matter so much which one I’m using here? Because most keyboards don’t have the feature I need, which is to be able to pop up programmatically, and Onboard has:

qdbus org.onboard.Onboard /org/onboard/Onboard/Keyboard \
org.onboard.Onboard.Keyboard.Show

Beautiful, eh? All the other keyboards I’ve explored work “automatically” — that is, you enable the concept of “onscreen keyboards”, and they pop up when you focus on an input field or the like, like the amusingly named Improved OSK keyboard.

(The name is a typical result of these plugin application ecologies:

It’s a fork of something that already has a bad name, “Improved Onscreen Keyboard” — instead of making one app that works well, we get a large number of vague variations.)

So they’re all useless for my use case, which requires that I’m able to pop up a keyboard when I hit the right square on the screen.

And besides, it “pushes” all the windows up to occupy its own region, which is annoying.

Even if an ideal onscreen keyboard does exist, that fits my requirements like Onboard did — I have no faith in that it’ll still exist and be usable next month.

So what to do? Well, I guess I’ll just… implement yet another crappy onscreen keyboard, won’t I? Won’t I just.

My first thought was to use xdotool, but that doesn’t work under Wayland, of course. People are talking about ydotool as a replacement — but it’s not. It basically just lets you send rawish events to the currently focused window, which is … pretty weird. But I mean, you can switch windows by sending the correct key sequences, but that’s incredibly brittle, and I wonder what the use case for ydotool is at all. And:

Uhm… ok…

So my solution is very specific to my problem: Since I’m using Emacs to get the events, I’m also using Emacs to dispatch the events. That is, if the user hits the q square and we’re in “emacs” mode, we let Emacs handle the q event. If we’re in “mpv” mode, Emacs parses the ~/.config/mpv/input.conf file and sees which command the key maps to, and sends that over to mpv. For instance, for m we have:

(movie-send-mpv-command '((command . ["cycle" "mute"])))

Emacs is already communicating with mpv, of course, so this is pretty easy to add.

But I’m beginning to understand why all these onscreen keyboards are so tightly bound to specific desktop systems: Because the Wayland people haven’t been interested in providing any good protocols for automation and tinkering. (Extensions to the Wayland protocol have been proposed, and finally implemented by some compositors, or course, but not Gnome. But perhaps this will work in a general way in just another decade on Wayland.)

Anyway, let’s see what this looks like now, after tinkering with this for a couple hours…

So here’s the laptop normally…

And with my onscreen keyboard. Hey, that’s not so awful…

So what about when mpv is running? Yeah, that’s fine. I mean, it’s not as if I’m going to type a lot on this — it’s just as an escape hatch when there’s something that’s not available in the normal touch grid.

And I have to find a way to make mpv ignore the touches here, because it pops up some annoying stuff on the screen (title and some controls), but that should hopefully be simple enough.

Oops — Gnome Shell died? Again? That’s the third time that has happened when mpv is closing: Something mpv is doing makes Gnome Shell die. Oh, it’s a known issue, but nobody seems to know why Gnome crashes?

Tra, la, la. Being on Wayland is like being back on X in 1993 — random things you do can take down the entire laptop.

This is surely the Year of Linux on the Laptop!

To remap the PrtSc key to the Hyper modifier key under Wayland with Gnome Shell on Ubuntu/Debian on the 28th of April 2024 (around noon) with a us keyboard mapping (I’m hedging my bets here), put the following in the ~/.config/xkb/symbol/us file:

partial alphanumeric_keys modifier_keys
xkb_symbols "hyper" {
    name[Group1]= "Hyper (US)";
    include "us(basic)"
    key <PRSC> { [ Hyper_L ] };
    modifier_map Mod3 { <PRSC> };
};

And then log out and back in again.

That’s it. That’s definitely all I had to say. Or perhaps…

Yes, it’s bitching and moaning time again, but I feel like I’m entitled — because I’ve been struggling with this for (and I’m not exaggerating) years. That is, I’ve taken stabs at making this work going back at least a decade. In total, I’ve probably spent more than a dozen hours pulling my hair and swearing at the computer. So:

In the Olden Days Of Yore, the first thing that happened when you used a computer for the first time was getting really annoyed with the keyboard because nothing was ever exactly like you wanted. (Especially with the national varieties of different keyboards from different companies — the horror!) I remember back at the university in the 90s, people had all kinds of fancy and amusing layouts, making it impossible for anybody to use anybody elses keyboards — keyboard encryption as a security feature, eh? And these shenanigans were possible because it was trivial to modify keyboard definitions. If you wanted to have a Hyper key on PrtSc (if that key did indeed exist back then; probably not on most keyboards), you put this in ~/.Xmodmap and went on with your life:

keycode 107 = Hyper_L
add Mod3 = Hyper_L

This still works to this day, three decades later — sort of. Any changes to the HID device hierarchy these days and the OS is likely to blank the setting and then you have to xmodmap it “manually”. And this only works if you’re still running X, of course, which is getting to be more and more difficult.

I got a new Lenovo Yoga X1 Gen 8, which is a nice laptop — I like a Yoga while travelling, because you can flip it around and watch movies on it without having the keyboard in front of you.

See? It’s nice.

And it has a built-in WWAN/5G modem, so I can use the laptop when there’s no WIFI and without tethering and stuff.

But the modem didn’t work in Debian/stable, so I installed Ubuntu. The modem works fine there, but:

I opened the Settings app, and there were all these horizontal stripes and glitches and stuff? I mean… what? This is a totally normal laptop with built-in normal Intel graphics:

00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-P [Iris Xe Graphics] (rev 04)

And still, just opening the Settings apps displays oodles of glitches in the default configuration? So they’ve obviously stopped even testing in Xorg — everything is moving to Wayland anyway, right?

And then I tried watching a video with mpv! My god!

Googling showed a lot of people having problems in this area, and the solutions were along the lines of:

i915.enable_dc=0
intel_idle.max_cstate=2

And other boot parameters, none of which had any effect for me. So after trying that for a few hours, the writing’s on the wall, even if I’ve resisted for years: I have to switch to Wayland before I get so old that there’s no chance of me ever figuring out any computer stuff ever again: I have to finally make The Configuration For The Future.

So how do you do something as trivial as assigning Hyper_L to the PrtSc key? As the keyboard(5) man page says:

The specification of the keyboard layout in the keyboard file is based on the XKB options XkbModel, XkbLayout, XkbVariant and XkbOptions. Un- fortunately, there is little documentation how to use them.

The keyboard definition in an Ubuntu/Debian machine is in /etc/default/keyboard:

cat /etc/default/keyboard
# KEYBOARD CONFIGURATION FILE

# Consult the keyboard(5) manual page.

XKBMODEL="pc105"
XKBLAYOUT="us"
XKBVARIANT=""
XKBOPTIONS=""

BACKSPACE="guess"

OK. And Wayland is using the xkb stuff? Confusing, but OK. So I google how to redefine things in this system, and… there’s a million people as confused as I am. Because surely things can’t be as bad as this, fifteen years on since Wayland was created?

The story told on a thousand web sites is that to alter the keyboard layout, you “just” alter the layout definition file, which is in /usr/share/X11/xkb/symbols. (In my case, it’s /usr/share/X11/xkb/symbols/us, since I’m using a US layout.) But this is absurd! That’s a file that’s updated when the OS updates, so either your changes will be overwritten, or updating won’t touch the file at all, so you won’t get updates from the OS. (Which can be important — keyboards rarely change, but they do evolve slowly over time, and you want a solution you can use for decades, right?)

There’s official-looking pages like this that tries to tell you how to add new layout files:

OK, ignoring the empty “writing a symbol file” entry, they suggest creating a new file /usr/share/X11/xkb/symbols/test, which is nice, because it won’t be overwritten by the OS… but then you have to register that file in two other files, /usr/share/X11/xkb/rules/xorg.lst and /usr/share/X11/xkb/rules/xorg.xml, which will be overwritten by the OS! This shit is infuriating!

But I did take a stab at it — created the new file and changed /etc/default/keyboard to point to it… and it had absolutely no effect in Gnome Shell, because it doesn’t really rely on it? Only slightly? (Did I mention that none of this is really documented well?)

xkb has specific options for swapping around certain modifiers:

setxkbmap -option caps:hyper

So you can presumably put that into XKBOPTIONS in /etc/default/keyboard if you wanted to map Caps Lock to Hyper… but these options only exist for certain combinations of keys! How insane is that? I mean, it’s nice that the XKB developers thought it would be handy to swap certain keys around, but it’s absurd that they thought they could enumerate them all? Instead of making a handy general utility that allowed people to do whatever they wanted, they hard-coded a few possibilities and called it a day.

But some suggest you do something like this:

cd /usr/share/X11/xkb/symbols
mv us us-real
cp hyper us

And then you have

/usr/share/X11/xkb/symbols

be just:

xkb_symbols "basic" {
    name[Group1]= "Hyper (US)";
    include "us-real"
    key <PRSC> { [ Hyper_L ] };
    modifier_map Mod3 { <PRSC> };
};

And this works, and it has the advantage of you being able to actually make sense of whatever changes you’ve made to the layout. But still, it has the problem of not being updated when the OS updates (or conversely, being overwritten). (And in any case, keeping personal configuration under /usr is a horrible long term solution, anyway — you want your config to follow you around as you upgrade laptops, and that ideally means just copying your home directory.)

So many people are trying to make sense of this system — like this guy, who has been through my entire journey, and amusingly enough, he’s also trying to rebind PrtSc to Hyper!

This sounds excellent! Just what I need! This should be the official documentation for when you google “how do I rebind a key”, right? Except that it doesn’t work — at least not under Gnome Shell (the default in Ubuntu), which just seems to ignore the XKB_DEFAULT_LAYOUT variable. I’m guessing the “before starting my compositor” means that he runs Sway or something…

*sigh*

And one more deeply infuriating thing about this whole mess: Whenever you’re testing something, you can never know whether logging out and then in again, or a full reboot is necessary, because it’s never clear whether the changes are applied on login, or by the OS (probably in systemd somewhere nasty) when booting up. What used to be a priority in Unix — being able to try things out iteratively and interactively (with informative error messages when you screw something up) has been replaced by mysterious systems that nobody can tell exactly how interact. So while Wayland uses the XKB files, you can’t use setxkbmap to try out the changes you’re making.

(Instead, if you make a syntax error and log out/in again, the OS will just display the White Screen of Death saying that something’s wrong (but not what, of course.). ssh to the rescue.)

And people are desperately trying to make things work:

I didn’t find any well-working solutions in 2024, so I’ve made one. Search for the “Shyriiwook” extension in the GNOME Shell Extensions store.

I applaud their efforts, but surely this shouldn’t have to be necessary!

But there are other ways to remap key in Linux, surely? OK, let’s take a look at a guide. It lists four different applications that work under Wayland: keyd, kbct, kmonad and interception. These keyboard remapper things all rely on daemons that talk to the evdev layer in the Linux kernel, and should work no matter whether you use Wayland or X.

keyd looks nice, and is available on Ubuntu by default. But:

It doesn’t support a Hyper key.

kbct, kmonad and interception aren’t in Ubuntu, and I didn’t test them, because most guides on the Googles seem to suggest that the generically named input-remapper (which was previously called key-remapper) is the bee’s knees. And it’s in Ubuntu! But when I started it, it didn’t look anything like what people were talking about, and doesn’t match the documentation. Were there several of these?

No, it just turns out that there’s a new version out, and it looks totally different, and has a different configuration hierarchy:

Thanks, thanks. So I downloaded that from Github, managed to get it installed and the daemon running, and

# input-remapper-control --symbol-names | grep -i hyper
#

It doesn’t support remapping a key to Hyper.

*sigh*

This excellent page is what finally pointed me in the right direction. The recipe given there doesn’t work for me, and as he says:

Now, I fully agree that this is cumbersome, clunky and feels outdated.

But after testing a number of variations on this theme, I ended up with the solution I posted above. Which works. Today.

Who knows what these Gnome/Wayland people will come up with tomorrow. It would be nice if it were something that, you know, works, but I’m not really holding my breath.

So now that the first hurdle is out of the way, I just have to fix all the other things that don’t quite work under Wayland. I mean, it’s only fifteen years old; one can’t expect everything to work yet…