Random Thoughts

This is the third part of a probably never-ending series of blog posts that started here and continued here. But! I think I’m getting pretty close to having everything working, so perhaps this is the final post?

So I’ve got this Lenovo Yoga thing, and the point of it is that you can flip it around to this “tent” position and then use it to watch movies. (And I’ve sown a handle for it so you can carry it around conveniently.) This means that you have to change the rotation of the screen when you flip it around. This normally happens automatically, but it’s so chaotic and unpredictable in Gnome (at least) that it’s unusable — instead of going 180° it may decide to go 270° if you’re not doing it exactly… right… and then you have to flip it back over and back again and AARGH.

So you have to disable the automatic flipperoo action and do the rotation “manually”. I.e., I’ve implemented listening to evdev and if I get a SWITCH_TOGGLE event, I rotate the screen. This is trivial under X, of course: You just use the standard xrandr function.

Gnome under Wayland, on the other hand, is a ball of spaghetti where everything is supposed to just happen magically, so there’s no built-in way of triggering something like this externally. Fortunately, it’s been implemented, and you’d think that Debian would pack up something as obviously useful, but nope. In any case, it’s easy enough to download and build yourself.

Since it’s a touch interface, it’s nice to be able to show a help grid…

… and the previous blog post was mostly about implementing an onscreen keyboard for this use case. And I used qiv to display the grid, since it’s got a convenient, simple interface, and allows having a transparent background. (But not alpha values.)

In addition to not having alpha support, it’s also just dog slow when doing transparencies. Look at how it’s painting the screen here — glitchy and ugly.

But! There’s something called pqiv which supports real alpha channels (meaning that I can have the rest of the screen “dim” a bit to make the overlay more readable).

See? All nice and stuff. However, there’s an issue with pqiv, too:

If I touch anywhere on the screen, it triggers some mouse handling logic designed to allow panning over an image. Fortunately, pqiv is written in an easy to hack way, so adding support for ignoring events is trivial.

I was also plagued by mpv killing Gnome when mpv exited, so I tried building the development versions of mpv and ffmpeg, and that seems to have fixed the problem? At least I haven’t seen any crashes today. (The mpv build system has changed once again, so I’ve updated the build instructions.)

Let’s see… is everything working now? Well, my setting to do nothing when I close the lid has disappeared from Gnome, and it’s now apparently a setting in systemd instead. *sigh* Because a user setting belongs in /etc/systemd/logind.conf instead of under /home, apparently?

And the Yoga has an OLED screen, so I really for it to switch the screen off when it’s not in use. And fast. (I’ve ruined another OLED screen by not taking precautions.) Gnome, of course, has mechanism for doing that — but it ties screen locking to screen blanking, and after googling for half an hour, I can’t find any way to disable that. Since I want the screen to blank fast when not in use, it’d be annoying to have to type in a password every time to unblank, right?

So I reached for the trusty xscreensaver, which indeed offers this. And it wakes up when I move the mouse, but… not when I hit a key!? What new hell is this!? And again, googling doesn’t help, but I find a lot of grumbling about the state of screensavers under Wayland, so perhaps the upshot is that this just doesn’t work because of… er… keyboard grabbing issues or something?

Fortunately, my Emacs is already listening to all events, so it’s trivial to just add some more code to wake up xscreensaver if somebody hits a key. This also means that this doesn’t work unless Emacs is running, but hey — let’s not be absurd! Emacs is always running!

(If you’re interested in looking at this work-in-progress code, you can find it on Microsoft Github. It’s probably not very usable for people in general, though — it’s geared towards my personal movie watching flow.)

But now everything is perfect, surely!

This is part two of a series of blog posts that I’m sure is going to go on forever: Wherein I Bitch About Wayland And Modern Life In General.

So, last time, I finally bit the bullet and moved my travelling Yoga laptop to Wayland. This time around, I’m trying to get one more piece of the puzzle to work: The onscreen keyboard.

But first of all, I should perhaps explain my use case, because otherwise this will all seem incomprehensible. I mean, even more.

When I’m travelling, I bring a Lenovo Yoga laptop with me, because it’s so handy to watch movies on (in hotels and on planes etc). I guess most people solve this by bringing a tablet of some kind, but I’ve never found anything as convenient as just using one of these laptops. (Your mileage will vary.) Since the physical keyboard isn’t available in the “tent” position like up in the picture there, I’ve implemented a simple touch interface for Emacs (based on libevdev). Which means that I can just touch virtual squares on the screen to do various stuff:

Here’s the same view, but with labels on the squares to remind me where the various functions are.

Similarly, then actually playing the movie (with mpv), I have a whole bunch of convenient functions available by touching various squares…

… here with the grid shown with labels.

And all this works fine in Wayland, too, since it doesn’t really talk to Wayland stuff at all — it just pokes around in the Linux evdev hierarchy to get the events.

So what’s the problem? The problem is that sometimes I really, really need a keyboard, like when searching for things.

Now, all desktop environments have onscreen keyboards (or as some systems call them, “virtual keyboards” or the like). And… that’s the problem!

That is, there is no One True Onscreen Keyboard — instead they’re implemented for each specific desktop system, and the result is that they’re all lacking in features.

In the Good/Bad Olden Days, if people implemented a thing, they’d make sure that it worked for Linux, Macos, Windows, FreeBSD, whatever, and they’d slowly accrue a lot of features and become pretty useful. These days, a thing like this is implemented as a plugin — we’re here seeing the Onboard keyboard for Gnome Shell. But, as is always the case with these things:

Since it’s tied to Gnome Shell and uses internalish Gnome Shell features, it needs to be updated constantly as Gnome Shell changes. (And they usually aren’t.)

But if all these desktops have onscreen keyboards, why does it matter so much which one I’m using here? Because most keyboards don’t have the feature I need, which is to be able to pop up programmatically, and Onboard has:

qdbus org.onboard.Onboard /org/onboard/Onboard/Keyboard \
org.onboard.Onboard.Keyboard.Show

Beautiful, eh? All the other keyboards I’ve explored work “automatically” — that is, you enable the concept of “onscreen keyboards”, and they pop up when you focus on an input field or the like, like the amusingly named Improved OSK keyboard.

(The name is a typical result of these plugin application ecologies:

It’s a fork of something that already has a bad name, “Improved Onscreen Keyboard” — instead of making one app that works well, we get a large number of vague variations.)

So they’re all useless for my use case, which requires that I’m able to pop up a keyboard when I hit the right square on the screen.

And besides, it “pushes” all the windows up to occupy its own region, which is annoying.

Even if an ideal onscreen keyboard does exist, that fits my requirements like Onboard did — I have no faith in that it’ll still exist and be usable next month.

So what to do? Well, I guess I’ll just… implement yet another crappy onscreen keyboard, won’t I? Won’t I just.

My first thought was to use xdotool, but that doesn’t work under Wayland, of course. People are talking about ydotool as a replacement — but it’s not. It basically just lets you send rawish events to the currently focused window, which is … pretty weird. But I mean, you can switch windows by sending the correct key sequences, but that’s incredibly brittle, and I wonder what the use case for ydotool is at all. And:

Uhm… ok…

So my solution is very specific to my problem: Since I’m using Emacs to get the events, I’m also using Emacs to dispatch the events. That is, if the user hits the q square and we’re in “emacs” mode, we let Emacs handle the q event. If we’re in “mpv” mode, Emacs parses the ~/.config/mpv/input.conf file and sees which command the key maps to, and sends that over to mpv. For instance, for m we have:

(movie-send-mpv-command '((command . ["cycle" "mute"])))

Emacs is already communicating with mpv, of course, so this is pretty easy to add.

But I’m beginning to understand why all these onscreen keyboards are so tightly bound to specific desktop systems: Because the Wayland people haven’t been interested in providing any good protocols for automation and tinkering. (Extensions to the Wayland protocol have been proposed, and finally implemented by some compositors, or course, but not Gnome. But perhaps this will work in a general way in just another decade on Wayland.)

Anyway, let’s see what this looks like now, after tinkering with this for a couple hours…

So here’s the laptop normally…

And with my onscreen keyboard. Hey, that’s not so awful…

So what about when mpv is running? Yeah, that’s fine. I mean, it’s not as if I’m going to type a lot on this — it’s just as an escape hatch when there’s something that’s not available in the normal touch grid.

And I have to find a way to make mpv ignore the touches here, because it pops up some annoying stuff on the screen (title and some controls), but that should hopefully be simple enough.

Oops — Gnome Shell died? Again? That’s the third time that has happened when mpv is closing: Something mpv is doing makes Gnome Shell die. Oh, it’s a known issue, but nobody seems to know why Gnome crashes?

Tra, la, la. Being on Wayland is like being back on X in 1993 — random things you do can take down the entire laptop.

This is surely the Year of Linux on the Laptop!