Useful Consumer Review

I needed a new scanner, and I wanted one that was significantly faster than the one I’ve been using until now. After some Googling, I landed on the Epson DS-50000, which is an A3+ scanner with a promise of being able to scan an A3 300DPI page in four seconds.

The web site I bought it from said that it’s a 4PPM scanner.

THOSE ARE NOT THE SAME THING!

So I was excited when I finally got it, two months after I ordered it. Was it going to be 4PPM or 4SPP?

The latter! From hitting “enter” on my USB-connected laptop to having the scanned page up in my Emacs, it takes 3.5 seconds. Very impressive, I think. It’s only USB2, and I thought that might be a bottleneck, perhaps, but apparently not. It also feels very sturdy and it doesn’t have a fan, so it’s quite silent. It’s very nice. And it makes good-quality scans, too.

But.

Yes, I connected it to a Linux laptop and you all knows what comes next: A tale of woe.

$ scanimage -d epsonds:libusb:001:019 --resolution 300dpi > /tmp/file

Worked perfectly the first time I tried it, so I thought I was in luck for once. But then I tried it again:

$ scanimage -d epsonds:libusb:001:019 --resolution 300dpi > /tmp/file
scanimage: open of device epsonds:libusb:001:019 failed: Error during device I/O

Gah! Basically, nothing helped. And it’s not just when scanning: Any command that talks to the scanner works the first time:

$ scanimage -L
device `epsonds:libusb:001:019' is a Epson DS-50000 ESC/I-2

But the second time:

$ scanimage -L

No scanners were identified. If you were expecting something different,
check that the scanner is plugged in, turned on and detected by the
sane-find-scanner tool (if appropriate). Please read the documentation
which came with this software (README, FAQ, manpages).

If I unplug it and replug it, then it works again.

After googling a bit, it seems that somebody had the same issue some years back with a different scanner, and the way to fix it was to build scanimage and friends from the current source tree, but that didn’t help.

So after some head scratching, I thought of a different approach: Obviously, something scanimage is doing is leaving the scanner in a bad state. What if I just reset the USB interface? Is that something that’s even possible?

Yes!

I added a Makefile and put it on Microsoft Github for your convenience. All the code does is basically:

ioctl(fd, USBDEVFS_RESET, 0);

And this resets just the single USB port, so nothing else wonky happens to the USB sub-subsystem. And the command is extremely fast, so it adds no delay to the scanning process.

You do get a lot of these messages in your kern.log, though:

Oct 28 21:03:06 corrigan kernel: [972003.683893] usb 1-2: reset high-speed USB device number 19 using xhci_hcd

But who cares.

It’s truly the year of Linux on the Laptop.

Innovations in Emacs Touch Interfacing

I’ve long attempted to hack some touch interfaces for laptops in non-keyboard configurations.

The sad thing is that there aren’t really any good solutions in GNU/Linux. If you want to be able to respond to more complex events like “two finger drag”, you have to hack GTK and use Touchégg, and then it turns out that doesn’t really work on Wayland, and then most of the events disappeared from the X driver, and then…

In short, the situation is still a mess. And then my lug-around-the-apt-while-washing-TV-laptop died (ish), so I had to get a new one (a Lenovo X1 Yoga (2nd gen (which I had to buy from Australia, because nobody would sell it with the specs I wanted (i.e., LTE modem if I wanted to also take it travelling (the 3rd gen has an LTE modem that’s not supported by Linux))))):

And now, with Ubuntu 18.04, everything is even worse, and I’m not able to get any multi finger events at all! All the touch events are just translated into mouse events! Aaaargh!

After despairing for an eternity (OK, half a day), I remembered another touch interface that I quite like: The Perfect Reader.

It’s a bit hard to tell here, but the idea is that you divide the screen into areas, and then you just tap one of the areas to have the associated action happen.

Surely even Linux can’t have fucked up something so basic: It must be possible to get that kind of access.

And it’s possible! Behold!

Er… What’s going on on the other side of the backyard?

Eeek! Kitten! Go back inside!

That’s not a safe place to play! … *phew* It sat down, and turned around and went back inside. *heart attack averted*

ANYWAY!

The idea is that there’s one action grid overlay when Emacs is in the forefront, and another when the mpv video player is.  All the events go via Emacs, though, which controls mpv via the mpv socket interface.  (And, by the way, I have to say that I’m really impressed with mpv.  It has all the commands you want it to have.  The documentation is somewhat lacking, though.)

Here’s a demo:

Basically, I’m just reading the output from libinput-debug-events (which outputs everything that goes through /dev/input/event* (so you have to add your user to the input group to access them)), and then execute things based on that. libinput is allegedly the new hotness, and replaces libev and the synaptics X driver, and is supposed to be supported on both Wayland and Xorg, so hopefully this attempt at an interface will last a bit longer than the previous ones.

I wrote the controlling thing in Emacs, of course, and you can find it on Github. I’ve been using an Emacs-based movie/TV viewer since 2004, and I’m not giving up now! So there.

mpv Is Nice

I’ve been watching movies and TV using an Emacs interface since 2004, according to the movie.el file.  Under the hood, though, I’ve hacked away at mplayer to make it do what I want, which includes a convenient way to switch audio outputs and output progress info so that I can store that to restart the video in the same place I left off.

And various bits and bobs.

But with the advent of 4K Bluray, it seems like mplayer just doesn’t hack it anymore.  So I started watching 4K material with mpv and the rest with mplayer.

And then I got a new TV machine (because the old one wasn’t fast enough to play Crouching Tiger, Hidden Dragon in 4K) and, as usual, mplayer didn’t really want to compile on the new Debian version, so I thought it was time to switch completely to mpv.

I thought I was going to have to do the same hacking on mpv as mplayer, but it turns out that mpv is really nice: It has almost all the features I need, and in a very pleasant-to-work-with interface: Just talk to mpv by sending JSON via a Unix-domain socket.

The only thing it didn’t have was a way to set the screensaving prefix on-the-fly (which I need for making GIFs), so I hacked that in.  The mpv people have really cleaned up the mplayer source code, I have to say.  Everything is nice and logical and clean, apparently, and I adding the feature took like 15 minutes.  Kudos.  I’ll try upstreaming the patch… if I can figure out how the mpv people like receiving patches.

Just one note on building mplayer and FFmpeg: Some things in Debian break if you have the new FFmpeg libraries in /usr/local, so put it somewhere else.  In case somebody needs to build mpv from git in Debian, this is what works for me, and is non-breakey:

/usr/src/FFmpeg $ ./configure --prefix=/usr/plocal --enable-gnutls 
/usr/src/FFmpeg $ make
/usr/src/FFmpeg $ sudo make install
/usr/src/FFmpeg $ cd /usr/src/mpv
/usr/src/mpv $ ./bootstrap.py
/usr/src/mpv $ PKG_CONFIG_PATH=/usr/plocal/lib/pkgconfig ./waf configure
/usr/src/mpv $ ./waf build

And then the mpv executable is in /usr/src/mpv/build/mpv, and can just live there without being installed.

So Much 4K

I got a 4K TV a while back, but I haven’t had any 4K media.

Until yesterday!

A couple of months ago, the encryption on 4K Bluray media was partially broken. It’s not just rip’n’go as it is with DVDs and 2K Bluerays, but, basically, you can anyway. You just download this file that has some hashes in it, and then you can use the wonderful makemkv program to rip the DVDs.

If your disc isn’t covered by the hash file, makemkv will create a dump file which you can then mail to the makemkv guy, who will then do some magic and a couple of days later the hash file will be updated with the necessary stuff to rip it. People are speculating about just what’s going on: Perhaps there’s a hardware thing that’s been “compromised” and can output the required data based on the dump file? Or perhaps the makemkv guy doesn’t feel comfortable releasing the full software because of reasons?

In any case, this means that I can finally watch 4K media on my Linux setup! Yay!

You need a “friendly” drive, though. I’ve got a drive that identifies itself as “BD-RE ASUS BW-16D1HT 3.01”, and it’s a SATA drive that I use a USB3-to-SATA interface to connect to the computer.

See? Totes gorge.

What drives work changes all the time, and the first one I bought (a USB3 LG drive) refused to play along. And things may change at any point, so if you find one that works, never upgrade the firmware.

I tested by ripping Thor: Ragnarok, and the resulting file was 50GB big. (The normal 2K Bluray is 25GB. Logical.)

But then the problem is: How do you play this stuff? It’s way too large to decode and render in software, so you need hardware support, and that support is only in the latest version of some of the players.

I chose mpv, because they seem to be most up-to-date. The versions that are distributed in Debian are way too old. So you have to build from source.

You need mpv itself, ffmpeg and libav, apparently, and all quite recent. Clone them all, build and install ffmpeg and libav, and then build mpv using that weirdo waf build system. (The instructions are on the mpv page.)

And now I can watch Thor! On Linux! Whoho!

I have an Nvidia card, so my command line is “mpv –vo=opengl –hwdec=cuda-copy”. And is 4K better than 2K?

Just look at Chris Whatsisname’s eyebrows in 4K:

Compare to 2K:

Ewww! You can’t even count the hairs in 2K! So horrible!

And now… I’m going to watch some more Bergman sourced from an 80s VHS tape upconverted to DVD.

Fast Music, or: USB Is Weird

I have my music on an USB3 RAID5 consisting of three external disks connected to one of these, which isn’t a bad little computer: It’s has a 1.7GHz i7-3517UE (Ivy Bridge) CPU, so it’s small, but not horribly slow.

But then one of the disks went AWOL and I thought that perhaps it was time to upgrade the disk array. It’s about seven years old, after all, and surely things have gotten better in the meantime.

My main use case, is, well, it’s a file server that I play music off of:

That’s not a very strenuous task: It basically has to feed out FLAC files over NFS faster than my stereo machine (pictured above) can play them, and basically no machine made after 1987 is too slow for that task.

But my Emacs-based music player doesn’t do any caching of metadata, so if I ask it “show me all the 8K records I have in chronological order”, it has to read eight thousand files, and that takes a while if the disk is slow. This is a problem that has grown year by year, of course, so it’s another reason to explore faster disks.

(I mean, I could add a caching system to my music system, but to quote what Leonard Nimoy said in The Empire Strikes Back: “Meh.”)

So I got a couple of USB3 SSDs. Splurge! I connected them up to the Intense PC and started copying things over. I wondered how slow the original RAID was, and it turned out to be 50MB/s, which is very slow, indeed. With the new disks, I should get like, er, more! MORE!

Copying finished, I did indeed get higher speed. 100MB/s. Which is pitiful. The native speed of the SSD should be 500MB/s, but given USB3, it should be slower, but not 20% of the speed.

So after much head-scratching, I noticed that the CPU was pegged to 100% whenever I read intensively from the disks. Is it possible that USB is such a crappy system that a 1.7GHz Xeon CPU from some years ago would be the bottleneck here?

So I extended a USB3 cable to the other server I had in the same closet, which I had bought a month earlier to do the RAID for my film collection:

It has a i5-7260U CPU @ 2.20GHz, so not much difference in Hurtzes, but it’s a 7th gen Intel CPU, and the other machine has a 3rd gen.

And… Wow! 320MB/s reading speed! 3x faster than the older machine, with the same SSDs, USB3 hub and everything.

I quickly rejuggled my setup and made that machine do the /music array, too, and sighed a breath of relief.

Now I can play music six times faster than before! Whoho!

But then!

The RAID went AWOL, always with the same messages about “tag#0 FAILED”, “USB disconnect” and “I/O error” on various /dev/sdx-es.

I first suspected the USB3 hub, so I got a new one… A couple of days later, the same thing. Tried a different USB3 cable (it’s always the cable!); same thing.

Of course, after each time this happens I have to rebuild the RAID, things get inconsistent and stuff.

Finally, I move the USB from the port on the left there to the right…

And two weeks later, still haven’t had a single disk brown-out.

So: The takeaway here is: 1) USB is a janky thing. It’s not quite like SCSI in olden days (no goat sacrifices needed), but it’s janky. 2) If your USB is slow, get a faster CPU.

The good thing about USB setups like this is that, in my experience, once you get them going satisfactorily, they’re pretty stable. Unless you do something crazy like insert a new USB device. Then all bets are off.

Of course, having a machine with room for plenty of SATA disks internally would be better, but I’ve never seen one that’s a) small and IV) allows easy access to disks that have failed and have to be replaced.

But look!

I can now display all the albums from 1975 by the snap of your fingers! If your fingers snap really slowly. But still!

And since the /dvd disks spin down automatically, my computer setup is now 100% without anything mechanical moving around normally, and I can walk past that closet without hearing any humming sounds.

Well, beyond my tinnitus, that is.

The Ever-Shifting Sands of udev.rules

I use Telldus Tellstick to do home automation *cough* I mean control the lights:

It’s an unassuming USB stick that implements a serial interface so that you can talk to it by just sending some strings to it and read the response. Ideal for Linux! Yes!

But.

You do want the device to show up at a default space so that you can find it, and if you have more than one serial USB interface thing, which one of /dev/ttyUSB0, /dev/ttyUSB1 og /dev/ttyUSB2 is it?

udev to the rescue!

And here your woes begin, because every fucking time you upgrade Linux, the fucking udev people change the fucking syntax vaguely and how you fucking refer to fucking devices. Fucking.

In the really olden days you said

KERNEL=="ttyUSB*", SYSFS{idVendor}=="1781", SYSFS{idProduct}=="0c31", 
  MODE="0666", NAME="tellstick"

But then they changed SYSFS to ATTR in the slightly less olden days:

KERNEL=="ttyUSB*", ATTR{idVendor}=="1781", ATTR{idProduct}=="0c31", 
  MODE="0666", NAME="tellstick"

Then they disallowed renaming devices, and instead you add symbolic links to the device:

KERNEL=="ttyUSB*", ATTR{idVendor}=="1781", ATTR{idProduct}=="0c31", 
  MODE="0666", SYMLINK+="tellstick"

Then they changed ATTR to ATTRS:

ATTRS{idVendor}=="1781", ATTRS{idProduct}=="0c31", MODE="0666",
  SYMLINK+="tellstick"

Then they changed what the attributes refer to, so instead of getting this:

[larsi@stories ~]$ ls -l /dev/tellstick 
blrwxrwxrwx 1 root root 7 Feb 25 16:12 /dev/tellstick -> ttyUSB1

you get this:

root@stories:/home/larsi# ls -l /dev/tellstick 
lrwxrwxrwx 1 root root 15 Feb 25 16:27 /dev/tellstick -> bus/usb/001/014

And that doesn’t work, because that’s not a serial USB interface, but a raw USB interface of some kind which can’t be opened like a serial interface.

And, remember, you can’t refer to /dev/ttyUSB*, because that’s the problem you’re trying to solve.

The solution to these problems is this following command:

 # udevadm info -a -n /dev/ttyUSB1

You’ll get output like

 looking at device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0/ttyUSB1/tty/ttyUSB1':
    KERNEL=="ttyUSB1"
    SUBSYSTEM=="tty"
    DRIVER==""

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0/ttyUSB1':
    KERNELS=="ttyUSB1"
    SUBSYSTEMS=="usb-serial"
    DRIVERS=="ftdi_sio"
    ATTRS{latency_timer}=="16"
    ATTRS{port_number}=="0"

Well, nothing there to distinguish the Tellstick from anything else, to continue down the output…

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0':
    KERNELS=="1-10:1.0"
    SUBSYSTEMS=="usb"
    DRIVERS=="ftdi_sio"
    ATTRS{authorized}=="1"
    ATTRS{bAlternateSetting}==" 0"
    ATTRS{bInterfaceClass}=="ff"
    ATTRS{bInterfaceNumber}=="00"
    ATTRS{bInterfaceProtocol}=="ff"
    ATTRS{bInterfaceSubClass}=="ff"
    ATTRS{bNumEndpoints}=="02"
    ATTRS{interface}=="TellStick"
    ATTRS{supports_autosuspend}=="1"

Yes! The ATTRS{interface}==”TellStick” thing looks like it’s something we can use to distinguish between Tellsticks and other devices, and it’s not so far down in the device hierarchy that we’re not talking about serial interfaces any more, and presto!

[larsi@stories ~]$ ls -l /dev/tellstick 
lrwxrwxrwx 1 root root 7 Feb 25 16:33 /dev/tellstick -> ttyUSB1

Here’s the magical setup file, for reference if anybody wants to write a udev rule for Tellstick devices that works on February 26th 2018, but probably not the week after:

$ cat /etc/udev/rules.d/10-tellstick.rules 
ATTRS{interface}=="TellStick", MODE="0666", SYMLINK+="tellstick"

You can also refer to the parent attributes by saying SUBSYSTEMS at a strategic point, so the following also works today:

KERNEL=="ttyUSB[0-9]*", SUBSYSTEM=="tty", 
  SUBSYSTEMS=="usb", ATTRS{idVendor}=="1781", 
  ATTRS{idProduct}=="0c30", 
  MODE="0666", SYMLINK+="tellstick"

Basically, when upgrading a Linux machine, most everything just works; all the peripherals and all the software. But I always end up twiddling the udev setup for half an hour.

Fonts, Swashes, Linux, Problems

I recently bought a font called Jolie Romantique (for a future, er, project), and it worked without any problems: I plopped the .ttf file into ~/.fonts, created an SVG file with some text in it, ran it through ImageMagick “convert” and got this:

Of special interest here is that end-of-word swash.  (That’s apparently the technical term for those things: Swashes.  I know.)

Here’s the SVG file:

<svg width="600" height="200" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> <rect width="600" height="200" x="0" y="0" fill="white"></rect> <text y="100" x="65" text-anchor="left" font-family="JRS" fill="black" stroke="black" font-weight="regular" font-size="120">Like#02</text> </svg>

Did you catch the fun bit?  Yes, it’s that “#02” in there.  The swashes are apparently implemented as ligatures or something, and they’re encoded in the font as #01, #02 and #03.  Here’s the same file with the #03 swash:

Fun!

Like I said, this worked perfectly…  on my Debian jessie machine.  Then I upgraded it to Debian stretch, and the swashes stopped working.  Now the results look like this:

This is sad.

And I have no idea where to report this bug.  It’s not a bug in imagemagick or the SVG libraries, because X programs (mis)behave the same way.  So it’s in the TTF rendering libraries?  What’s responsible for that?

I’ve tried googling for this, but the pickings are rather slim.  Apparently not too many people are that into swashing.

If I can’t get this fixed, I’ll have to set up an Emacs-as-a-Service thing on a jessie machine to create the images for my er project.  So, no biggie.  But! This should just, like work.  And it doesn’t.

I just tried this on Ubuntu 17.10, and it doesn’t work there, either.  So it’s a cross-platform bug, I guess?