The Ever-Shifting Sands of udev.rules

I use Telldus Tellstick to do home automation *cough* I mean control the lights:

It’s an unassuming USB stick that implements a serial interface so that you can talk to it by just sending some strings to it and read the response. Ideal for Linux! Yes!

But.

You do want the device to show up at a default space so that you can find it, and if you have more than one serial USB interface thing, which one of /dev/ttyUSB0, /dev/ttyUSB1 og /dev/ttyUSB2 is it?

udev to the rescue!

And here your woes begin, because every fucking time you upgrade Linux, the fucking udev people change the fucking syntax vaguely and how you fucking refer to fucking devices. Fucking.

In the really olden days you said

KERNEL=="ttyUSB*", SYSFS{idVendor}=="1781", SYSFS{idProduct}=="0c31", 
  MODE="0666", NAME="tellstick"

But then they changed SYSFS to ATTR in the slightly less olden days:

KERNEL=="ttyUSB*", ATTR{idVendor}=="1781", ATTR{idProduct}=="0c31", 
  MODE="0666", NAME="tellstick"

Then they disallowed renaming devices, and instead you add symbolic links to the device:

KERNEL=="ttyUSB*", ATTR{idVendor}=="1781", ATTR{idProduct}=="0c31", 
  MODE="0666", SYMLINK+="tellstick"

Then they changed ATTR to ATTRS:

ATTRS{idVendor}=="1781", ATTRS{idProduct}=="0c31", MODE="0666",
  SYMLINK+="tellstick"

Then they changed what the attributes refer to, so instead of getting this:

[larsi@stories ~]$ ls -l /dev/tellstick 
blrwxrwxrwx 1 root root 7 Feb 25 16:12 /dev/tellstick -> ttyUSB1

you get this:

root@stories:/home/larsi# ls -l /dev/tellstick 
lrwxrwxrwx 1 root root 15 Feb 25 16:27 /dev/tellstick -> bus/usb/001/014

And that doesn’t work, because that’s not a serial USB interface, but a raw USB interface of some kind which can’t be opened like a serial interface.

And, remember, you can’t refer to /dev/ttyUSB*, because that’s the problem you’re trying to solve.

The solution to these problems is this following command:

 # udevadm info -a -n /dev/ttyUSB1

You’ll get output like

 looking at device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0/ttyUSB1/tty/ttyUSB1':
    KERNEL=="ttyUSB1"
    SUBSYSTEM=="tty"
    DRIVER==""

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0/ttyUSB1':
    KERNELS=="ttyUSB1"
    SUBSYSTEMS=="usb-serial"
    DRIVERS=="ftdi_sio"
    ATTRS{latency_timer}=="16"
    ATTRS{port_number}=="0"

Well, nothing there to distinguish the Tellstick from anything else, to continue down the output…

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0':
    KERNELS=="1-10:1.0"
    SUBSYSTEMS=="usb"
    DRIVERS=="ftdi_sio"
    ATTRS{authorized}=="1"
    ATTRS{bAlternateSetting}==" 0"
    ATTRS{bInterfaceClass}=="ff"
    ATTRS{bInterfaceNumber}=="00"
    ATTRS{bInterfaceProtocol}=="ff"
    ATTRS{bInterfaceSubClass}=="ff"
    ATTRS{bNumEndpoints}=="02"
    ATTRS{interface}=="TellStick"
    ATTRS{supports_autosuspend}=="1"

Yes! The ATTRS{interface}==”TellStick” thing looks like it’s something we can use to distinguish between Tellsticks and other devices, and it’s not so far down in the device hierarchy that we’re not talking about serial interfaces any more, and presto!

[larsi@stories ~]$ ls -l /dev/tellstick 
lrwxrwxrwx 1 root root 7 Feb 25 16:33 /dev/tellstick -> ttyUSB1

Here’s the magical setup file, for reference if anybody wants to write a udev rule for Tellstick devices that works on February 26th 2018, but probably not the week after:

$ cat /etc/udev/rules.d/10-tellstick.rules 
ATTRS{interface}=="TellStick", MODE="0666", SYMLINK+="tellstick"

You can also refer to the parent attributes by saying SUBSYSTEMS at a strategic point, so the following also works today:

KERNEL=="ttyUSB[0-9]*", SUBSYSTEM=="tty", 
  SUBSYSTEMS=="usb", ATTRS{idVendor}=="1781", 
  ATTRS{idProduct}=="0c30", 
  MODE="0666", SYMLINK+="tellstick"

Basically, when upgrading a Linux machine, most everything just works; all the peripherals and all the software. But I always end up twiddling the udev setup for half an hour.

Fonts, Swashes, Linux, Problems

I recently bought a font called Jolie Romantique (for a future, er, project), and it worked without any problems: I plopped the .ttf file into ~/.fonts, created an SVG file with some text in it, ran it through ImageMagick “convert” and got this:

Of special interest here is that end-of-word swash.  (That’s apparently the technical term for those things: Swashes.  I know.)

Here’s the SVG file:

<svg width="600" height="200" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> <rect width="600" height="200" x="0" y="0" fill="white"></rect> <text y="100" x="65" text-anchor="left" font-family="JRS" fill="black" stroke="black" font-weight="regular" font-size="120">Like#02</text> </svg>

Did you catch the fun bit?  Yes, it’s that “#02” in there.  The swashes are apparently implemented as ligatures or something, and they’re encoded in the font as #01, #02 and #03.  Here’s the same file with the #03 swash:

Fun!

Like I said, this worked perfectly…  on my Debian jessie machine.  Then I upgraded it to Debian stretch, and the swashes stopped working.  Now the results look like this:

This is sad.

And I have no idea where to report this bug.  It’s not a bug in imagemagick or the SVG libraries, because X programs (mis)behave the same way.  So it’s in the TTF rendering libraries?  What’s responsible for that?

I’ve tried googling for this, but the pickings are rather slim.  Apparently not too many people are that into swashing.

If I can’t get this fixed, I’ll have to set up an Emacs-as-a-Service thing on a jessie machine to create the images for my er project.  So, no biggie.  But! This should just, like work.  And it doesn’t.

I just tried this on Ubuntu 17.10, and it doesn’t work there, either.  So it’s a cross-platform bug, I guess?

VHS, Linux, Problems

I’ve been trying to tidy up the storage locker in the loft this autumn, getting rid of old junk (so that I can put more, slightly newer junk up there). I happened unto this box:

A nice stack of VHS tapes. If I remember correctly, the reason I kept these was that during the 80s and 90s I recorded quite a lot of music (videos and live stuff) from the TV, and I had always thought about some day going through them, picking out the good bits and uploading them to Youtube or something.

Perhaps this year is finally the year?

I have an old, old DV conversion thingie somewhere, but it’s so old that it’s a Firewire dongle. And I have no Firewire equipment, so I thought it might just be easier to buy something new and shiny. Surely?

I think you can already guess where this tale is going.

This is a Diamond VC500, and it has both composite in and S-Video in, and it seems to be a popular choice.

My old VHS player (a Panasonic DMS-ES35V) has S-Video out, so I hoped that it would, like, just kinda work, and it does. With composite, but not S-Video. That is, after I discovered how to switch the input, something worked:

$ v4l2-ctl -d /dev/video0 -i 1 -s 5

(That’s S-Video input and the PAL standard.)

But the video was in black-and-white, and after googling for a few days, it turns out that this is a known problem: S-Video on the VC500 doesn’t work, at least not in Linux. And S-Video is a much better signal than composite, so that’s a bummer.

So I got another one of these dongles at random, and this is from StarCom and presents itself as

 Bus 001 Device 012: ID eb1a:5051 eMPIA Technology, Inc.

And S-Video works! However, sound doesn’t, and googling for another two days show that this is a common problem with the “StarTech.com S-Video / Composite to USB Video Capture Cable Adapter”, at least under Linux.

But audio is easy: I could just hook it up to the internal sound card, and after fiddling with alsacontrol for a few hours, I managed to get some sound out of it.

All set! Recording time, here we go!

Unfortunately, I had lost the remote control to my VHS player and adjusting the tracking without it is impossible.

Ebay to the rescue!

It’s amazing that things like VHS head cleaners are still easily obtainable, so I got one of those, too.

Now, surely, I’m ready to record.

Except finding some acceptable settings for ffmpeg or memcoder to process the data and output in a format that’s good for editing turned out to require several more days worth of googling. Here’s what I ended up with:

$ ffmpeg -f v4l2 -t 04:00:00 -thread_queue_size 1024 -i /dev/video0 -f alsa -i hw:0 -aspect 4:3 -c:v prores_ks -profile:v 3 -q:v 4 -vf yadif=0:-1:0,crop=iw:ih-6:0:0 -c:a pcm_s16le -af aresample=async=1000 /video/vhs/v10.mov

So the codec here is ProRes. I tried a dozen different codecs, but either they were too slow (leading to drops from ffmpeg) or they were in formats that crashed Lightworks (the video editor I use). huffyuv, raw, h.264, whatever…

But this one works for me, at least.

Oh, and the “crop” thing is because the 6 final lines output from the StarTech dongle are bright green.  And I’m deinterlacing since VHS is interlaced.

An action shot of the entire machinery. So techy.

Now the only problem is that I had to upgrade the kernel to Linux 4.12 and the machine to the latest Debian, since the support for the StarTech USB thingie first appeared in that kernel. And after upgrading LightWorks didn’t want to start. So I upgraded to Lightworks 14, which starts, but doesn’t accept the old license, so I had to buy a new license, and now the license server is … wrong (“There are no unactivated licenses available”).

Anyway, I can use the free version for this, since 720p should be enough.  I created a new account on Youtube for this stuff, since it’s, er, best to keep it separated from the more important free jazz vids.

Hm.  Or perhaps I should just use HTML <video> things and host the .mp4 files privately?  That would avoid any Youtube hassles…  Hm…  But the discoverability on Youtube is nice.  Perhaps both?

I’ve done the first four hour tape now, and there was about 20 minutes of things that are possibly interesting.  For instance, this Kristin Hersh piece where she does four songs live in the 120 Minutes studio:

Oh, it’s apparently not possible to do simple <video> embeds on a WordPress.com site?

Oh, it is possible now?

Yes!  All those pages that said that it wasn’t allowed were outdated.  Apparently WordPress.com started allowing this earlier this year?

Anyway.

Linux Can 4K @ 60 Haz

I tried getting 4K @ 60Hz using Intel built-in graphics, and I failed miserably.

Rather than spend more days on that project (yes, this is the year of Linux on the TV, I Mean Desktop), I bought a low-profile Nvidia card, since there are several people on the interwebs that claim to have gotten that to work.

It’s brand new, cheap and fanless, too: A Geforce GT 1030, which is Nvidia’s new budget card, launched last month, I think.

It’s finnier than Albert and takes up several PCI Express slots.

However, that’s not really a problem in this media computer: Lots and lots of space for it to spread itself out over. Just one PCI Express slot, though.

But it’s on the long side: If I had any hard disks installed in this machine, I would have had to get creative. Instead I just removed that HD tray thing.

But! There’s two … capacitors down there where the PCI Express “key” thing. Like just quarter millimetre too much to the right…

I bent them ever so gently over and I managed to get the card in. How utterly weird.

SO MUCH DRAMA!

Anyway: Mission complete. This card has a DVI plug in addition to the HDMI, but I’m not going to use that, so I just left it with the protective rubber.

See? Lots of space. Of course, it would have been better to remove the cooler completely and hook it up via heat pipes to the chassis, but… that’s like work.

But did this solve my problems? After installing Nvidia’s proprietary drivers (apparently Nouveau doesn’t support the GT 1030 yet, since it’s a Kepler card)…

Yes! 3840×2160 @ 59.95 Hz, which is pretty close to 60Hz. Yay!

Of course, I have no 4K material on my computer, so the only thing that’s actually in 4K now is my Emacs-based movie interface. Here’s whatsername from Bewitched in 2K:

Eww! How awful! Horrible!

See! 4K! How beautiful!

(Let’s pretend that the entire difference isn’t in the different moire patterns!)

*phew*

And the Futura looks much better in 4K too, right?

Right?

This was all worth it.

Isn’t 4K@60Hz HDMI possible in Linux?

I bought a spiffy new 4K TV the other week, and I wanted to set up a complete 4K pipeline. Not that there’s that much 4K stuff available: For instance, Netflix has a pitiful 101 list of shows, most of them made by Netflix themselves.

Anyhoo! I was thinking a bit about how to set this all up so that I’d have a 4K pipeline from my computer, while there’s a 2K pipeline from the Ipod (which I use to watch Amazon stuff) and a 4K pipeline from the Chomecast (which I planned on using to watch Netflix stuff). All the while being able to record shows so that I can screenshot them for the “The World” blog series. It’ll be simpler if I just draw a diagram:

See? Easy peasy. I have to insert an “HDMI splitter” (which is a euphemism for “HDCP stripper”) in between the Ipod and the HDMI recorder because DRM. But! Then I realised that I could just watch non-4K Netflix on the Ipod, so that simplified things hugely:

So simple! So I have the Ipod as a DRM device, and then all the rest is FREEDOM!

This is what it looks like in reality:

So neat and orderly.

So everything is A-OK, then? No. The thing is that I’m not able to get my PC to do 4K at 60Hz. This is what xrandr says:

See the 3840×2160 entry? That’s the one I want, and it has a max refresh rate of 30Hz. TV is 50Hz or 60Hz, so having only 30Hz sucks.

As you’ve probably also noticed, xrandr says that DP1 is connected, not HDMI. That’s because I plugged in an external DisplayPort to HDMI adapter, which is supposed to support 4K @ 60Hz. But that apparently doesn’t work.

I also tried the native “HDMI” plug on this motherboard (which is an ASRock Fatal1ty Z270 Gaming-ITX/ac LGA1151 HDMI 2.0 ITX motherboard *phew*), but that turns out to also be DisplayPort, really, but with a built-in DP->HDMI adapter (to allow 4K@60Hz).

This stuff is apparently called LSPCON, and support for it landed in Linux somewhat recently.

I’ve tried very up-to-date kernels (I’m now running on 4.12.0-994-generic), and my Debian is stretch, so it’s also quite recent. I’ve tried plugging the computer directly into the TV via an HDMI 2.0 cable, bypassing all switches and everything, and the result is the same.

My TV is a Sony A1E, which claims to support 4K@60Hz, but I’m unable to make either the Chromecast or the TV say what frame rate it has when it’s doing 4K.

So… is this supposed to work or not? Has anybody gotten 4K @ 60Hz HDMI to work in Linux?

[Edit a day later]:

Googling around shows a lot of people with similar problems that they apparently are able to resolve by adding the proper ModeLines.  In 2017.

So I tried this:

xrandr --newmode "3840x2160_60.00" 712.34 3840 4152 4576 5312 2160 2161 2164 2235 -HSync +Vsync
xrandr --addmode DP1 "3840x2160_60.00"
sleep 15
xrandr --verbose --output DP1 --mode "3840x2160_60.00"

But the Xorg log says:

[  2222.946] (II) intel(0): switch to mode 3840x2160@60.0 on DP1 using pipe 0, position (0, 0), rotation normal, reflection none
[  2222.967] (EE) intel(0): failed to set mode: Invalid argument [22]
[  2223.014] (--) intel(0): HDMI max TMDS frequency 300000KHz
[  2223.031] (II) intel(0): resizing framebuffer to 1920x1080
[  2223.031] (II) intel(0): switch to mode 1920x1080@60.0 on DP1 using pipe 0, position (0, 0), rotation normal, reflection none
[  2223.084] (--) intel(0): HDMI max TMDS frequency 300000KHz

So that’s a no go..

My New Fashion Designer Blog x Useful Consumer Review

I’ve been trying to use an Android device as my “lug around the apt. while doing stuff” device, but it’s just not good enough. None of the apps for sshfs file browsing or video watching are beyond the “well, it kinda works” level. The ssh times out and doesn’t come back again until you do *stuff* and all the video apps have audio/video sync issues.

(Don’t all Android users play TV via sshfs?)

So! Back to Linux!

I got this laptop which has a “tent” configuration where the keyboard is in the back. Since it’s Linux, sshfs and mplayer work perfectly. The only issue is just the portability.  Physically. See, it’s all tent-ey and stuff, but when it’s in that configuration, picking it up is very awkward. There’s nothing to grab hold on.

So! I got my sewing kit out (a gift from my mother like ten years ago and seldom used since) and bought a ribbon thingie.

Oh, yeah, the computer is a… is that a d and a q? So it’s a … Dairy Queen laptop?

Let’s go with that.

I had planned on wrapping the ribbon around a wine cork, but the wine tonight turned out to be from New Zealand and had a screw cap. So instead I just rolled the ribbon up…

… and then expertly stitched it up. (Don’t show that seam to anybody with sewing skills unless you want them to have a heart attack.)

I did one at either end and now I have a handle!

See! It works!!! (He says while gingerly walking around with it.)

(Oh, the thing being shown is an episode of … Killjoys. Yeah, that’s what it’s called. It’s about people pointing guns at each other… IN SPACE!)

This is what the Dairy Queen looks like from the other angle. Not quite as … TV-ey. That’s a word. (The keyboard is switched off when it’s in this position.)

I had to adapt my Emacs video viewer for touch action, and I used Touchéegg along the lines for the music player. Seems to work OK. I added touch actions for mplayer, too, so that I can pause and skip and stuff.

THIS ALL MAKES SENSE!

Ipad, Screenshots and Linux

It’s become increasingly clear over the past few months that many recent, fun-sounding films from countries with smaller film industries will never get a physical DVD release. The only way to see these films is via Amazon Prime, and since Amazon Prime isn’t conveniently available on Linux machines, I’m having to use an Ipad to watch these films.

Which is fine.

But I have to take screenshots. (I mean. I have to!) That leaves a lot of images on the Ipad, and I need a convenient way to get those from the Ipad to my Linux laptop.

Now, there’s Dropbox, and… stuff, but all those things require some manual work at the Ipad end or the Linux end. I hate manual work.

So wouldn’t it be nice if there was some way to just “do something” and then all the screenshots from the Ipad would magically appear in my Emacs on the Linux laptop?

There is!

It’s a bit fiddly, though, so let’s just get started.  (“A bit fiddly” is code for “seven pages of text is to follow”.  This is the year of Linux on the Desktop.)

My Laptop runs Ubuntu Linux, and it comes with an ifuse distribution that’s built with GnuTLS and not OpenSSL. This doesn’t work with IOS 10, so you need to build it yourself.

I’ve streamlined the build instructions a bit. The utilities and libraries needed to talk to Idevices is spread over four repositories, but the following should pull them all down, build them, and install them under /usr/local.

#!/bin/bash 
sudo apt-get build-dep ifuse 
for elem in libusbmuxd libimobiledevice usbmuxd ifuse; do 
  git clone https://github.com/libimobiledevice/${elem}.git 
  (cd $elem; sh ./autogen.sh; make; sudo make install) 
done

Check that your path is picking up the correct version by saying

$ type ifuse 
ifuse is /usr/local/bin/ifuse

Then plug in the Ipad via USB, and test that you can talk to the device. You may have to press “Trust this device” on the Ipad while connecting.

$ sudo mkdir /media/ipad 
$ sudo chown larsi.users /media/ipad 
$ idevicepair pair SUCCESS: Paired with device 37b633350ab83dc815a6a97dcd6d327b12c41968 
$ ifuse /media/ipad

You should now have the Ipad mounted, and the screenshots are under /media/ipad/DCIM.

Now for the fun part: Make the laptop copy over the files automatically whenever you plug in the Ipad.  I’m using the general setup from the usb-automount setup I did a few months ago.

The main difference is this udev.rules file:

ATTR{idVendor}="05ac", ATTR{idProduct}="12ab", PROGRAM="/bin/systemd-escape -p --template=usb-automount@.service $env{DEVNAME}", ENV{SYSTEMD_WANTS}+="%c"

And then I use the following script to actually copy over the contents to the current “viewing directory”.

#!/bin/bash

command="$1"

if [ "$command" = "remove" ]; then
    umount /media/ipad
    exit
fi

# Mount the Ipad.
ifuse /media/ipad

if [ -d "/media/ipad/DCIM/100APPLE" ]; then
    cd /media/ipad/DCIM/100APPLE
    for pic in IMG*; do
        to="/home/larsi/pics/ipad"
        if [ ! -f "$to/$pic" ]; then
            cp -av "$pic" "$to/$pic"
            chown larsi.users "$to/$pic"
            if echo "$pic" | grep PNG > /dev/null; then
                shot=`echo "$pic" | sed 's/IMG_/shot/' | sed 's/PNG/png/'`
                if [ ! -f "/home/larsi/.movie-current/$shot" ]; then
                    ln "$to/$pic" "/home/larsi/.movie-current/$shot"
                fi
            fi
        fi
    done
fi

cd /
umount /media/ipad

Or something like that.  You obviously have to adjust the script to your needs if you want to do something like this, but the general idea should be sound, I think…

Look!  The images appeared in Emacs!  As if by magic!

It seems to work reliably, also after rebooting the laptop.  Apparently the “idevicepair pair” think only has to be done once?  Or something?

The only minor annoyance is that Ubuntu pops up an icon in the right-hand menu every time I plug in the Ipad.  Is there any way to say to Ubuntu “ignore this device in the UI”?  There is a way to make Ubuntu ignore all auto-mounted devices, but that’s not what I want, and there is a way to make Linux ignore a specific USB device completely (echo 0 > /sys/bus/usb/devices/2-1/authorized), but that’s not what I want, either.  I just want the UI to ignore this specific device…

Oh, never mind.

[Edit: I’ve now put the scripts on github.  That should make it easier to customize if anybody wants to use this setup…]