Music I’ve bought this month.
After starting to sort the music I have by release date, I noticed that I have like absolutely no music from the 60s. So I’ve been trying to explore that slightly more (like with the absolutely wonderful Bobbie Gentry), but I’m not really sure what methodology to use.
I mean, I had a look at “best albums from 1967” and stuff, and the music there is like the worst.
) |  | %3A+Jack+Orion) |  |  |
 |  |  |  |  |
 |  |  |  |  |
 | ) |  |  |  |
 |  |  |  |  |
 |  |  |  |  |
) | ) |  |  |  |
 |  |  |  |  |
 |  |  | ) |  |
 |  |  |  |  |
 |  |  |  |  |
 |  | +(1)) | +(2)) |  |
 |  |
You know that old saying about gravitating to your favourite tool? Well, my go-to thing for compositing text on top of images (and stuff) was LaTeX, and I’ve been doing that for decades.
What I’m mostly using this stuff for these days is the extremely important task of displaying weather data on small, cheap USB-powered monitors that I have here and there around the apt.
For instance, in the hall I have something that plays a random-but-related youtube clip (with no audio, of course) of whatever’s playing on the stereo, and on top of that I have a LaTeX-generated overlay that says what temperature it is and the rain forecast over the next 24 hours (that’s the line; rain ahoy).
It’s composed by this Perl script that uses LaTeX to generate the clock and the temperature text, curl to download the weather data, gnuplot to make the nice line, ghostscript to make a .pnm file, and som pbmplus things to composite it all.
And what occurred to me the other night is that I wanted to have a tiny text at the bottom there that told me the name of what was playing to and I thought “that’s just so much work”, and then I started wondering whether I should just rewrite all this in Emacs Lisp and use my new favourite hammer: svg.el. 
Presto! Still rainy!
It’s now a pure-Emacs image compositing thing which I’ve put on Microsoft Github if you want to read it, but it’s not generally useful. But you could perhaps use a couple of the elements if you want to do something similar. I also finally got a reason to play with xelb/xcb, which I’ve always wondered about. Seems like a very nice library, and maps 1:1 the X concepts onto Emacs Lisp.
Compositing images via svg.el is fun!
One thing that I thing I’m going to expand upon and put into Emacs is the SVG path/bezier curve stuff, because I think that would be generally useful.
This bug report adds a new function, svg-path, but I think Eli’s objections are right. Instead of adding such a low-level function, instead we should have, like (svg-multi-line svg points :type ‘bezier :smooth 0.2) that would compute the boring Bezier stuff for you. SVG supports insanely complicated paths, but I don’t think that’s all that useful in general…
But I don’t have time to do implement that properly now, but perhaps sometime this winter…
Oh, and while implementing this, I miscomputed the font size for one of the displays (that just displays what’s playing), and I thought it was such a striking mistake that I kept it.
I walk a lot, and I used to listen to music while walking, using big, beefy, good wireless headphones or headphones with an mp3 player built in. That’s a solved problem; excellent products exist.
Last year I started listening to radio dramas instead. Mostly mysteries, and quite a lot of stuff from the 50s. It’s fun! But I don’t need a full set of headphones to listen to that stuff; it’s fine listening to in mono in one ear only.
And here my troubles began.
There’s quite a lot of wireless earbuds out there, but they all have the same problem: The range sucks.
The first one of these I tried was the original Earin, and it was pretty much unusable. But now there’s the Earin M-2, which promises to have solved all the problems.
Like the previous version, it comes with a round charging case…
… and both buds fit in there. The charging case is larger than the previous version, but it’s also a lot less fiddly. The buds slide in there like magic (probably magnets), and that’s nice.
And the buds themselves are also larger, and don’t look as cool as the M-1s.
The range is better than the M-1, but it’s not actually good. The frustrating thing when reading reviews about wireless headgear is that apparently nobody who reviews them uses them outdoors. Indoors, blutooth works well, because it can bounce the radio against the walls, but outdoors, there’s nothing, so the radio just dissipates unless it’s very powerful.
If I have my phone in my left front pocket and an earbud in my right ear, they’ll drop connection like crazy when outdoors. I have to put the phone on the same side of the body as the earbud is if it’s to have any chance of working semi-reliably.
So these are pretty much standard in the range dept… but the UX is horrible. Just horrible. They use a touch interface, and give no audible feedback. So the way to use them is to put one in an ear, wait for ten seconds and then hopefully it’ll have contacted the phone, and then tap it to unpause, and then nothing happens, and then tap again, and then again, and then you’ve tapped too many times and it unpaused and paused again.
And when you’re not using it, you can’t just put it in a pocket, because if you touch it, you turn it on and it’ll use all its battery while not doing anything.
So you have to carry the case around with you, which is a bother.
Compare the Earin M-2 disaster to this older, cheaper earbud that goes under many names, like Savfy and Rowkin Mini. They look slightly different depending on who the manufacturer is, but it’s the same product, I think.
See that thing there on the side? It’s a button. B-u-t-t-o-n. Button. You press it to switch the earbud on, an you press it again to pause/unpause the audio, and you long press it to switch it off. When the earbud connects to the phone, you hear a voice saying “connected”. When it switches off, it tells you that, too.
The UX is hard to improve on.
Of course, the range is horrible, just like the Earin M-2. Perhaps even worse, really: I frequently find myself putting my phone in my shirt pocket to have it stay connected if I do something crazy, like look around before crossing a street.
Perhaps somebody one day will make something better, but it hasn’t happened yet. I hope that someday somebody will realise that playing audio using an almost unbuffered wireless stream is absurd: The earbuds should instead pull down blocks of data from the device and then play it. That would also allow it to shut down its antenna, meaning power savings.
Of course, this won’t work for real-time audio (like talking on the phone) but does that even happen any more? 100% of my use case for these devices is listening to prerecorded data, so it’s not like it has to be ultra low latency or anything.
ANYWAY! I’m mostly listening to mystery radio dramas, and the challenge there is that the actors will alternatively shout “THERE”S THAT BAD GUY! STOP HIM! PEW PEW!” and whisper “ssssh here’s the solution to the mystery that I’m telling you while we’re in this hedge”, and I found myself having to turn the volume up and down a lot, which is totally annoying. Surely there’s a solution to this?
And of course, and it’s called “compression” or “normalisation” (even if those aren’t the same things). On Linux, sox is a fine program that can do many audio things, and it has a module called “compand” which is made to compress audio (i.e., make the loud parts less loud and the quiet parts louder). By Googling a lot and experimenting a bit, I found some settings that work well for radio theatre, and I wrote a little bash script that’ll take a tree of mp3s, normalise/compress the audio, and then write out new mp3s, and also copy over id3 tags and artwork and the like.
It’s a trivial script, but Googling all the moving parts took me a while, so it’s here all put together in one place so that you don’t have to do this work yourself if you find yourself in the same position and landed on this page through googling “audio compression no I’m not talking about file size compression but, like, compression of the audio range, dude”.
I’m guessing this will also work on podcasts and the like, although I would imagine that the audio range is much narrower there to begin with. I’ve been using this for a few months and it seems to work well, although sometimes some very quiet sounds do get amplified a bit much.
I’ve always been frustrated by the IMDB web pages. Usually when I’m looking at a director’s oeuvre, I’m not interested in all the shorts, videos and games the director has created, but just want a list of the movies.
When I’m looking at a specific movie, it’s often because I want to know who the character on the screen I’m watching is being played by, but the images are so tiny and low res that it’s impossible to guess who’s who.
And once I found myself wondering, “what’s the name of that actor that played in that film with her and her”, and I know who the latter two are, but not what film, so it would be nice to be able to do a cross reference kind of thing… right? RIGHT!?
OK, the last thing happened only once, but it was still something that I thought might perhaps have been vaguely useful. But, of course, I didn’t do anything about all this because well you know.
I did download the IMDB data set and do various things with it (but mostly based on grepping and ad-hoc searches), but that came to a complete stop when IMDB revamped their data exports. Instead of semi-human semi-readable files, it’s now basically a database dump. No longer can you just grep for “Isaach De Bankolé”, because in the new files, you first have to find his ID, and then you can grep for that, but that just gives you movie IDs, and…
Long story short, I made an Emacs mode for looking at the new-format data.
It’s based on this sqlite3 module for Emacs. My first attempt was based on just storing all the data in hash tables in Emacs, because I’ve got lots of RAM. I thought that since I routinely open multi-gigabyte buffers in Emacs, and that’s no problem, and at work, I routinely have multi-tens-of-gigabyte processes in Common Lisp servers, that should be no problem in Emacs either.
I had forgotten that Emacs’ garbage collector is, er, kinda primitive. It’s an old-style stop-and-copy collector. This is fine for huge buffers, because that’s basically just a handful of objects, no matter how large the buffer is. When you create a 200M element hash table, all those elements need to be mark-and-sweeped individually, and as I found halfway through implementing it, that makes Emacs pause for like ten minutes at a time.
So scratch that. I went for sqlite3 and I store the data on disk. But interacting directly with the sqlite3 database data is a drag, so I wrote a kind of ORM… well, it’s more of a PRM. Plist-Relational Mapping. Because strong typing is for dweebs.
I mean, professionals.
Anyway, the raw dataset from IMDB is about 3GB. The sqlite3 database is about 7GB, and takes an hour or two to create on my machine. (Before I found out that sqlite3 autocommits by default, I had like two transactions per second. By slapping a transaction around the import, I get a few thousands of inserts per second.)
The module has some bugs, but I’ve sent a pull request, so hopefully it gets fixed. Or pull down my fork instead, which is here, and you’ll need it to do regexp searches.
If you want to play with this, you need a newish Emacs built with module support, the sqlite3 module, and imdb-mode.el.
And then just eval (imdb-download-and-create) and wait for a few hours.
The PRM works pretty much as you’d expect.
(imdb-select 'movie :mid "tt0090798") => ((:_type movie :mid "tt0090798" :type "movie" :primary-title "Caravaggio" :original-title "Caravaggio" :adultp "N" :start-year 1986 :end-year nil :length 93))
See?
(pp (imdb-select 'crew :mid "tt0090798") (current-buffer)) ((:_type crew :mid "tt0090798" :pid "nm0147599" :category "writer") (:_type crew :mid "tt0090798" :pid "nm0413897" :category "writer") (:_type crew :mid "tt0090798" :pid "nm0418746" :category "director") (:_type crew :mid "tt0090798" :pid "nm0418746" :category "writer"))
Easy!
(imdb-select 'person :pid "nm0147599") => ((:_type person :pid "nm0147599" :primary-name "Suso Cecchi D'Amico" :birth-year 1914 :death-year 2010))
Anyway, that’s the low-level interface. Here’s what the user interface looks like:
And then “x” and the noise is gone!!!
Only the real movies! *gasp* It cannot be!
(I didn’t choose Fincher here because I like his films, but because he’s done so much junk that’s it’s impossible to use his IMDB page.)
But… when looking at some actors I knew pretty well, I soon noticed that not all the films that the actor appeared in were listed. Here’s Tilda Swinton most recent few years:
What’s going on?! No Doctor Strange? Could there be a bug in my code? That seems impossible? I mean, it’s my code!
But nope, the problem is with the IMDB dataset. The file that lists what films actors appear in, “title.principals.tsv”, isn’t a complete list of participants, but instead, as the name really sorta kinda implies, a list of the most important people in that film. That means that it lists directors, writers, the cinematographer, (some) producers and then a few actors. But never more than ten people per film.
This is really weird, because directors and writers are already in the “title.crew.tsv” file.
This made me sad until I realised that I could just resort to web scraping.
So now I use the data set as a base and then insert the missing things afterwards.
Doctor Strange!
And when I’ve resorted to web scraping for that, I can just scrape for actor images, too:
*sigh*
In short, IMDB doesn’t have a usable API, and IMDB no longer export enough data to do anything useful with that data. So I guess this Emacs mode will work until they tweak their HTML.
While implementing the scraping, IMDB suddenly went missing and I just got:
I then found out that I had er slightly miswritten the end of the recursion, so I was hitting imdb.com with dozens of invalid hits per second in several concurrent threads. I AM SORRY IMDB.
They unblocked me after an hour.
Here’s what it looks like in action:
(Click on the embiggen symbol to embiggen so that you can see what’s going on.)
I’m like, selecting a few things and then toggling what bits to see (acting/directing/all/shorts).
Exciting!
And now it’s feature complete, too, so I’ve definitely saved a lot of time by writing this.
Er… Oh, yeah. I was going to implement intersections (i.e., list movies that have several specific people involved. So let’s see which films Tilda Swinton did with Luca Guadagnino:
Easy peasy. Or the more complete, unfiltered version:
Heh. I think “Dias de cine” and stuff are just Italian long-running movie news shows that both of them have appeared on? Or in the same episode? Hm… probably the latter, because each TV episode has its own unique ID.
Radioman. Mary Kerr. 2012.
Whaaa! Another documentary! I’m developing a hatred of documentaries…
But mostly American ones. This one in British, by Mary Kerr, and it’s good. But you have to wonder about whether this film will do its subject any good or whether it’ll just… make him so public that he can’t continue to live his life.
But it’s certainly the most star-studded documentary ever in the history of ever, so I guess the film-makers had fun. But he’s still getting parts, so perhaps this film did him no harm.
I found this film on Amazon Prime.
OK. this is the last Tilda Swinton Project film in this catch-up batch. See you next in… a couple of years?
This post is part of The Tilda Swinton Project.