mpv Is Nice

I’ve been watching movies and TV using an Emacs interface since 2004, according to the movie.el file.  Under the hood, though, I’ve hacked away at mplayer to make it do what I want, which includes a convenient way to switch audio outputs and output progress info so that I can store that to restart the video in the same place I left off.

And various bits and bobs.

But with the advent of 4K Bluray, it seems like mplayer just doesn’t hack it anymore.  So I started watching 4K material with mpv and the rest with mplayer.

And then I got a new TV machine (because the old one wasn’t fast enough to play Crouching Tiger, Hidden Dragon in 4K) and, as usual, mplayer didn’t really want to compile on the new Debian version, so I thought it was time to switch completely to mpv.

I thought I was going to have to do the same hacking on mpv as mplayer, but it turns out that mpv is really nice: It has almost all the features I need, and in a very pleasant-to-work-with interface: Just talk to mpv by sending JSON via a Unix-domain socket.

The only thing it didn’t have was a way to set the screensaving prefix on-the-fly (which I need for making GIFs), so I hacked that in.  The mpv people have really cleaned up the mplayer source code, I have to say.  Everything is nice and logical and clean, apparently, and I adding the feature took like 15 minutes.  Kudos.  I’ll try upstreaming the patch… if I can figure out how the mpv people like receiving patches.

Just one note on building mplayer and FFmpeg: Some things in Debian break if you have the new FFmpeg libraries in /usr/local, so put it somewhere else.  In case somebody needs to build mpv from git in Debian, this is what works for me, and is non-breakey:

/usr/src/FFmpeg $ ./configure --prefix=/usr/plocal --with-gnutls 
/usr/src/FFmpeg $ make
/usr/src/FFmpeg $ sudo make install
/usr/src/FFmpeg $ cd /usr/src/mpv
/usr/src/mpv $ ./bootstrap.py
/usr/src/mpv $ PKG_CONFIG_PATH=/usr/plocal/lib/pkgconfig ./waf configure
/usr/src/mpv $ ./waf build

And then the mpv executable is in /usr/src/mpv/build/mpv, and can just live there without being installed.

So Much 4K

I got a 4K TV a while back, but I haven’t had any 4K media.

Until yesterday!

A couple of months ago, the encryption on 4K Bluray media was partially broken. It’s not just rip’n’go as it is with DVDs and 2K Bluerays, but, basically, you can anyway. You just download this file that has some hashes in it, and then you can use the wonderful makemkv program to rip the DVDs.

If your disc isn’t covered by the hash file, makemkv will create a dump file which you can then mail to the makemkv guy, who will then do some magic and a couple of days later the hash file will be updated with the necessary stuff to rip it. People are speculating about just what’s going on: Perhaps there’s a hardware thing that’s been “compromised” and can output the required data based on the dump file? Or perhaps the makemkv guy doesn’t feel comfortable releasing the full software because of reasons?

In any case, this means that I can finally watch 4K media on my Linux setup! Yay!

You need a “friendly” drive, though. I’ve got a drive that identifies itself as “BD-RE ASUS BW-16D1HT 3.01”, and it’s a SATA drive that I use a USB3-to-SATA interface to connect to the computer.

See? Totes gorge.

What drives work changes all the time, and the first one I bought (a USB3 LG drive) refused to play along. And things may change at any point, so if you find one that works, never upgrade the firmware.

I tested by ripping Thor: Ragnarok, and the resulting file was 50GB big. (The normal 2K Bluray is 25GB. Logical.)

But then the problem is: How do you play this stuff? It’s way too large to decode and render in software, so you need hardware support, and that support is only in the latest version of some of the players.

I chose mpv, because they seem to be most up-to-date. The versions that are distributed in Debian are way too old. So you have to build from source.

You need mpv itself, ffmpeg and libav, apparently, and all quite recent. Clone them all, build and install ffmpeg and libav, and then build mpv using that weirdo waf build system. (The instructions are on the mpv page.)

And now I can watch Thor! On Linux! Whoho!

I have an Nvidia card, so my command line is “mpv –vo=opengl –hwdec=cuda-copy”. And is 4K better than 2K?

Just look at Chris Whatsisname’s eyebrows in 4K:

Compare to 2K:

Ewww! You can’t even count the hairs in 2K! So horrible!

And now… I’m going to watch some more Bergman sourced from an 80s VHS tape upconverted to DVD.

Fast Music, or: USB Is Weird

I have my music on an USB3 RAID5 consisting of three external disks connected to one of these, which isn’t a bad little computer: It’s has a 1.7GHz i7-3517UE (Ivy Bridge) CPU, so it’s small, but not horribly slow.

But then one of the disks went AWOL and I thought that perhaps it was time to upgrade the disk array. It’s about seven years old, after all, and surely things have gotten better in the meantime.

My main use case, is, well, it’s a file server that I play music off of:

That’s not a very strenuous task: It basically has to feed out FLAC files over NFS faster than my stereo machine (pictured above) can play them, and basically no machine made after 1987 is too slow for that task.

But my Emacs-based music player doesn’t do any caching of metadata, so if I ask it “show me all the 8K records I have in chronological order”, it has to read eight thousand files, and that takes a while if the disk is slow. This is a problem that has grown year by year, of course, so it’s another reason to explore faster disks.

(I mean, I could add a caching system to my music system, but to quote what Leonard Nimoy said in The Empire Strikes Back: “Meh.”)

So I got a couple of USB3 SSDs. Splurge! I connected them up to the Intense PC and started copying things over. I wondered how slow the original RAID was, and it turned out to be 50MB/s, which is very slow, indeed. With the new disks, I should get like, er, more! MORE!

Copying finished, I did indeed get higher speed. 100MB/s. Which is pitiful. The native speed of the SSD should be 500MB/s, but given USB3, it should be slower, but not 20% of the speed.

So after much head-scratching, I noticed that the CPU was pegged to 100% whenever I read intensively from the disks. Is it possible that USB is such a crappy system that a 1.7GHz Xeon CPU from some years ago would be the bottleneck here?

So I extended a USB3 cable to the other server I had in the same closet, which I had bought a month earlier to do the RAID for my film collection:

It has a i5-7260U CPU @ 2.20GHz, so not much difference in Hurtzes, but it’s a 7th gen Intel CPU, and the other machine has a 3rd gen.

And… Wow! 320MB/s reading speed! 3x faster than the older machine, with the same SSDs, USB3 hub and everything.

I quickly rejuggled my setup and made that machine do the /music array, too, and sighed a breath of relief.

Now I can play music six times faster than before! Whoho!

But then!

The RAID went AWOL, always with the same messages about “tag#0 FAILED”, “USB disconnect” and “I/O error” on various /dev/sdx-es.

I first suspected the USB3 hub, so I got a new one… A couple of days later, the same thing. Tried a different USB3 cable (it’s always the cable!); same thing.

Of course, after each time this happens I have to rebuild the RAID, things get inconsistent and stuff.

Finally, I move the USB from the port on the left there to the right…

And two weeks later, still haven’t had a single disk brown-out.

So: The takeaway here is: 1) USB is a janky thing. It’s not quite like SCSI in olden days (no goat sacrifices needed), but it’s janky. 2) If your USB is slow, get a faster CPU.

The good thing about USB setups like this is that, in my experience, once you get them going satisfactorily, they’re pretty stable. Unless you do something crazy like insert a new USB device. Then all bets are off.

Of course, having a machine with room for plenty of SATA disks internally would be better, but I’ve never seen one that’s a) small and IV) allows easy access to disks that have failed and have to be replaced.

But look!

I can now display all the albums from 1975 by the snap of your fingers! If your fingers snap really slowly. But still!

And since the /dvd disks spin down automatically, my computer setup is now 100% without anything mechanical moving around normally, and I can walk past that closet without hearing any humming sounds.

Well, beyond my tinnitus, that is.

The Ever-Shifting Sands of udev.rules

I use Telldus Tellstick to do home automation *cough* I mean control the lights:

It’s an unassuming USB stick that implements a serial interface so that you can talk to it by just sending some strings to it and read the response. Ideal for Linux! Yes!

But.

You do want the device to show up at a default space so that you can find it, and if you have more than one serial USB interface thing, which one of /dev/ttyUSB0, /dev/ttyUSB1 og /dev/ttyUSB2 is it?

udev to the rescue!

And here your woes begin, because every fucking time you upgrade Linux, the fucking udev people change the fucking syntax vaguely and how you fucking refer to fucking devices. Fucking.

In the really olden days you said

KERNEL=="ttyUSB*", SYSFS{idVendor}=="1781", SYSFS{idProduct}=="0c31", 
  MODE="0666", NAME="tellstick"

But then they changed SYSFS to ATTR in the slightly less olden days:

KERNEL=="ttyUSB*", ATTR{idVendor}=="1781", ATTR{idProduct}=="0c31", 
  MODE="0666", NAME="tellstick"

Then they disallowed renaming devices, and instead you add symbolic links to the device:

KERNEL=="ttyUSB*", ATTR{idVendor}=="1781", ATTR{idProduct}=="0c31", 
  MODE="0666", SYMLINK+="tellstick"

Then they changed ATTR to ATTRS:

ATTRS{idVendor}=="1781", ATTRS{idProduct}=="0c31", MODE="0666",
  SYMLINK+="tellstick"

Then they changed what the attributes refer to, so instead of getting this:

[larsi@stories ~]$ ls -l /dev/tellstick 
blrwxrwxrwx 1 root root 7 Feb 25 16:12 /dev/tellstick -> ttyUSB1

you get this:

root@stories:/home/larsi# ls -l /dev/tellstick 
lrwxrwxrwx 1 root root 15 Feb 25 16:27 /dev/tellstick -> bus/usb/001/014

And that doesn’t work, because that’s not a serial USB interface, but a raw USB interface of some kind which can’t be opened like a serial interface.

And, remember, you can’t refer to /dev/ttyUSB*, because that’s the problem you’re trying to solve.

The solution to these problems is this following command:

 # udevadm info -a -n /dev/ttyUSB1

You’ll get output like

 looking at device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0/ttyUSB1/tty/ttyUSB1':
    KERNEL=="ttyUSB1"
    SUBSYSTEM=="tty"
    DRIVER==""

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0/ttyUSB1':
    KERNELS=="ttyUSB1"
    SUBSYSTEMS=="usb-serial"
    DRIVERS=="ftdi_sio"
    ATTRS{latency_timer}=="16"
    ATTRS{port_number}=="0"

Well, nothing there to distinguish the Tellstick from anything else, to continue down the output…

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0':
    KERNELS=="1-10:1.0"
    SUBSYSTEMS=="usb"
    DRIVERS=="ftdi_sio"
    ATTRS{authorized}=="1"
    ATTRS{bAlternateSetting}==" 0"
    ATTRS{bInterfaceClass}=="ff"
    ATTRS{bInterfaceNumber}=="00"
    ATTRS{bInterfaceProtocol}=="ff"
    ATTRS{bInterfaceSubClass}=="ff"
    ATTRS{bNumEndpoints}=="02"
    ATTRS{interface}=="TellStick"
    ATTRS{supports_autosuspend}=="1"

Yes! The ATTRS{interface}==”TellStick” thing looks like it’s something we can use to distinguish between Tellsticks and other devices, and it’s not so far down in the device hierarchy that we’re not talking about serial interfaces any more, and presto!

[larsi@stories ~]$ ls -l /dev/tellstick 
lrwxrwxrwx 1 root root 7 Feb 25 16:33 /dev/tellstick -> ttyUSB1

Here’s the magical setup file, for reference if anybody wants to write a udev rule for Tellstick devices that works on February 26th 2018, but probably not the week after:

$ cat /etc/udev/rules.d/10-tellstick.rules 
ATTRS{interface}=="TellStick", MODE="0666", SYMLINK+="tellstick"

You can also refer to the parent attributes by saying SUBSYSTEMS at a strategic point, so the following also works today:

KERNEL=="ttyUSB[0-9]*", SUBSYSTEM=="tty", 
  SUBSYSTEMS=="usb", ATTRS{idVendor}=="1781", 
  ATTRS{idProduct}=="0c30", 
  MODE="0666", SYMLINK+="tellstick"

Basically, when upgrading a Linux machine, most everything just works; all the peripherals and all the software. But I always end up twiddling the udev setup for half an hour.

Fonts, Swashes, Linux, Problems

I recently bought a font called Jolie Romantique (for a future, er, project), and it worked without any problems: I plopped the .ttf file into ~/.fonts, created an SVG file with some text in it, ran it through ImageMagick “convert” and got this:

Of special interest here is that end-of-word swash.  (That’s apparently the technical term for those things: Swashes.  I know.)

Here’s the SVG file:

<svg width="600" height="200" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> <rect width="600" height="200" x="0" y="0" fill="white"></rect> <text y="100" x="65" text-anchor="left" font-family="JRS" fill="black" stroke="black" font-weight="regular" font-size="120">Like#02</text> </svg>

Did you catch the fun bit?  Yes, it’s that “#02” in there.  The swashes are apparently implemented as ligatures or something, and they’re encoded in the font as #01, #02 and #03.  Here’s the same file with the #03 swash:

Fun!

Like I said, this worked perfectly…  on my Debian jessie machine.  Then I upgraded it to Debian stretch, and the swashes stopped working.  Now the results look like this:

This is sad.

And I have no idea where to report this bug.  It’s not a bug in imagemagick or the SVG libraries, because X programs (mis)behave the same way.  So it’s in the TTF rendering libraries?  What’s responsible for that?

I’ve tried googling for this, but the pickings are rather slim.  Apparently not too many people are that into swashing.

If I can’t get this fixed, I’ll have to set up an Emacs-as-a-Service thing on a jessie machine to create the images for my er project.  So, no biggie.  But! This should just, like work.  And it doesn’t.

I just tried this on Ubuntu 17.10, and it doesn’t work there, either.  So it’s a cross-platform bug, I guess?

VHS, Linux, Problems

I’ve been trying to tidy up the storage locker in the loft this autumn, getting rid of old junk (so that I can put more, slightly newer junk up there). I happened unto this box:

A nice stack of VHS tapes. If I remember correctly, the reason I kept these was that during the 80s and 90s I recorded quite a lot of music (videos and live stuff) from the TV, and I had always thought about some day going through them, picking out the good bits and uploading them to Youtube or something.

Perhaps this year is finally the year?

I have an old, old DV conversion thingie somewhere, but it’s so old that it’s a Firewire dongle. And I have no Firewire equipment, so I thought it might just be easier to buy something new and shiny. Surely?

I think you can already guess where this tale is going.

This is a Diamond VC500, and it has both composite in and S-Video in, and it seems to be a popular choice.

My old VHS player (a Panasonic DMS-ES35V) has S-Video out, so I hoped that it would, like, just kinda work, and it does. With composite, but not S-Video. That is, after I discovered how to switch the input, something worked:

$ v4l2-ctl -d /dev/video0 -i 1 -s 5

(That’s S-Video input and the PAL standard.)

But the video was in black-and-white, and after googling for a few days, it turns out that this is a known problem: S-Video on the VC500 doesn’t work, at least not in Linux. And S-Video is a much better signal than composite, so that’s a bummer.

So I got another one of these dongles at random, and this is from StarCom and presents itself as

 Bus 001 Device 012: ID eb1a:5051 eMPIA Technology, Inc.

And S-Video works! However, sound doesn’t, and googling for another two days show that this is a common problem with the “StarTech.com S-Video / Composite to USB Video Capture Cable Adapter”, at least under Linux.

But audio is easy: I could just hook it up to the internal sound card, and after fiddling with alsacontrol for a few hours, I managed to get some sound out of it.

All set! Recording time, here we go!

Unfortunately, I had lost the remote control to my VHS player and adjusting the tracking without it is impossible.

Ebay to the rescue!

It’s amazing that things like VHS head cleaners are still easily obtainable, so I got one of those, too.

Now, surely, I’m ready to record.

Except finding some acceptable settings for ffmpeg or memcoder to process the data and output in a format that’s good for editing turned out to require several more days worth of googling. Here’s what I ended up with:

$ ffmpeg -f v4l2 -t 04:00:00 -thread_queue_size 1024 -i /dev/video0 -f alsa -i hw:0 -aspect 4:3 -c:v prores_ks -profile:v 3 -q:v 4 -vf yadif=0:-1:0,crop=iw:ih-6:0:0 -c:a pcm_s16le -af aresample=async=1000 /video/vhs/v10.mov

So the codec here is ProRes. I tried a dozen different codecs, but either they were too slow (leading to drops from ffmpeg) or they were in formats that crashed Lightworks (the video editor I use). huffyuv, raw, h.264, whatever…

But this one works for me, at least.

Oh, and the “crop” thing is because the 6 final lines output from the StarTech dongle are bright green.  And I’m deinterlacing since VHS is interlaced.

An action shot of the entire machinery. So techy.

Now the only problem is that I had to upgrade the kernel to Linux 4.12 and the machine to the latest Debian, since the support for the StarTech USB thingie first appeared in that kernel. And after upgrading LightWorks didn’t want to start. So I upgraded to Lightworks 14, which starts, but doesn’t accept the old license, so I had to buy a new license, and now the license server is … wrong (“There are no unactivated licenses available”).

Anyway, I can use the free version for this, since 720p should be enough.  I created a new account on Youtube for this stuff, since it’s, er, best to keep it separated from the more important free jazz vids.

Hm.  Or perhaps I should just use HTML <video> things and host the .mp4 files privately?  That would avoid any Youtube hassles…  Hm…  But the discoverability on Youtube is nice.  Perhaps both?

I’ve done the first four hour tape now, and there was about 20 minutes of things that are possibly interesting.  For instance, this Kristin Hersh piece where she does four songs live in the 120 Minutes studio:

Oh, it’s apparently not possible to do simple <video> embeds on a WordPress.com site?

Oh, it is possible now?

Yes!  All those pages that said that it wasn’t allowed were outdated.  Apparently WordPress.com started allowing this earlier this year?

Anyway.

Linux Can 4K @ 60 Haz

I tried getting 4K @ 60Hz using Intel built-in graphics, and I failed miserably.

Rather than spend more days on that project (yes, this is the year of Linux on the TV, I Mean Desktop), I bought a low-profile Nvidia card, since there are several people on the interwebs that claim to have gotten that to work.

It’s brand new, cheap and fanless, too: A Geforce GT 1030, which is Nvidia’s new budget card, launched last month, I think.

It’s finnier than Albert and takes up several PCI Express slots.

However, that’s not really a problem in this media computer: Lots and lots of space for it to spread itself out over. Just one PCI Express slot, though.

But it’s on the long side: If I had any hard disks installed in this machine, I would have had to get creative. Instead I just removed that HD tray thing.

But! There’s two … capacitors down there where the PCI Express “key” thing. Like just quarter millimetre too much to the right…

I bent them ever so gently over and I managed to get the card in. How utterly weird.

SO MUCH DRAMA!

Anyway: Mission complete. This card has a DVI plug in addition to the HDMI, but I’m not going to use that, so I just left it with the protective rubber.

See? Lots of space. Of course, it would have been better to remove the cooler completely and hook it up via heat pipes to the chassis, but… that’s like work.

But did this solve my problems? After installing Nvidia’s proprietary drivers (apparently Nouveau doesn’t support the GT 1030 yet, since it’s a Kepler card)…

Yes! 3840×2160 @ 59.95 Hz, which is pretty close to 60Hz. Yay!

Of course, I have no 4K material on my computer, so the only thing that’s actually in 4K now is my Emacs-based movie interface. Here’s whatsername from Bewitched in 2K:

Eww! How awful! Horrible!

See! 4K! How beautiful!

(Let’s pretend that the entire difference isn’t in the different moire patterns!)

*phew*

And the Futura looks much better in 4K too, right?

Right?

This was all worth it.