16×10%

Whaa… it’s been less than a month since the last of these posts (wherein I give a report from my gamified Emacs bug tracker spelunking).

I’ve been using various ways to select bug reports to handle since I started on this back in… 2019? Yes. I started, of course, with reports about things that I had experience with (Gnus, eww, etc), but I soon ran out of things to do that way.

So instead I started looking through all bug reports that hadn’t had any responses, and then all bug reports I had been the last person to respond (they have a tendency to end with “So I think we should do <foo>, anybody have any comments?” and then nobody does. So then I would do <foo>.

But then I ran out of those, too, so I started just sorting all the bug reports by the length of the discussion. It looks like this:

And this 10% stretch!

All the way to the bottom, Maggie! (Severed Heads Sample)

I made it all of the way to the bottom, Maggie!

The longest bug thread had 135 messages.

It turns out that, contrary to what I had imagined, many of those well-discussed bug reports had actionable conclusions: That is, after discussing something back and forth for a couple of years, the conclusion was that indeed we should do <foo>, but then nobody did.

So I did that now, which explains why this is the speediest ten-percenter in quite a while:

Started April 13th, done May 8th.

Of course, reading those threads too some time, but figuring out what to do usually takes more time.

Anyway, that means that there aren’t really anything big new feature to report this time over (except that Tree Sitter has landed on a feature branch), just a buttload of bug fixes and small new features. Here’s some of them:

Easier Scripting

Emacs now has an -x switch designed to make scripting easier. With the above, you can do:

Emacs almost had this capability before, but it was a bit messy and not very convenient.

Restarting Emacs

When trying stuff out in Emacs, you want to be able to restart Emacs conveniently so that you can see that things work as you want them to. A new M-x restart-emacs command now makes things easier.

*Help* improvements

My mission to make the help buffers prettier and easier to read continues, and menus which used to be explained this way…

… are now displayed this way:

You can now also edit variable values, and you can keep the *Help* window selected (without popping to other windows when clicking on buttons in that window).

C-h m has also been reformatted. It used to look like this:

It’s now:

I.e., that endless list of global minor modes has been moved to after the major mode.

Double-buffering on Windows

Emacs on Windows now has double-buffering (courtesy of Po Lu), so there should be less flickering when displaying animated images, and less flickering overall.

OK, I’m not going to go through the entire NEWS file; it’s just smaller items like this:

Lots and lots of teensy stuff. (As well as a buttload of bug fixes; about 30 commits per day.)

Well, onward and upward… literally. Because I’m now making my way back up again in the list, going through the reports I either skipped going downwards, or just missed (due to a buglet in debbugs-gnu: it didn’t sort merged bugs stably, so they appeared arbitrarily at the point of one of the bug reports in the list, so I missed them, at random, when making my way down the list).

We started this stretch at 2400 open bugs, and we’re now down to 2264. Which means that the next 10% is just 226. Mua ha ha.

Eclipse 1931: 東京の合唱

This is very, very unrestored. And silent. I mean, totally — there’s not even any music. So I listened to banging house music while watching this.

Ozu had made several dozens of movies before this (churning out half a dozen per year in the 20s), but this is apparently considered his first really good one.

Did I mention how unrestored this is? But it varies wildly from one moment to the next.

I’m impressed by how quickly the titles pass by — Japanese people can read fast.

And there’s not a lot of them — Ozu manages to tell a lot totally silently without cheating by resorting to titles.

Grody toilets, dude.

Did they shorten the English translation? Hm…

Ozu continued making silent movies until 1936, when he finally bowed to commercial pressures and went talkie.

And… this is such a fluid silent movie? Usually with silent movies, I feel like there’s something lacking, but this feels kinda complete the way it is. (Cultured people in the 30s and 40s bemoaned the vulgarity of talkies, and longed for the pure artistic expression of the silent movies, until Cahiers de cinema went *pfffffft* to all that jazz, and convinced everybody that missing sound was a technical limitation while black-and-white vs. colour was an artistic choice.)

It’s the slogan for the ages.

This is such a good movie. It’s billed as a comedy, and it’s pretty amusing now and then, but it’s really a melodrama? It reminds me, strangely enough, of Douglas Sirk’s movies a couple decades later.

It got a light touch, and a feel for character, that imbues the scenes with a sense of importance, of poignancy.

It’s not quite a masterpiece, but it’s kinda irresistable?

The banging house music didn’t hurt, either.

Oh, that’s just here…

Lovely movie. Best I’ve seen in ages.

Tokyo Chorus. Yasujirô Ozu. 1931.

This blog post is part of the Eclipse series.

Eclipse 1968: Wild 90

Heh heh.

Anyway, so this movie is three drunk, high guys pretending to be Italian mobsters and improvising, with D A Pennebaker filming.

I like Pennebaker’s camera work. It’s really cool. The three guys are kinda on the tedious side. They don’t say anything interesting — it’s like they have an idea that their prattle is fascinating — but they’re not saying anything, really. And they crack themselves up all the time without having really done anything funny.

But it’s like it almost works? If they’d planned things a bit better and had some interesting things to say? Instead it’s just some guys hamming it up.

That’s harsh:

The very urgency that Mailer has always tried to communicate makes it impossible to wade through so much rambling for a little art.

That’s harsh.

That’s harsh.:

This is likely the most self-indulgent, grating dumpster fire of a “film” I’ve ever had the displeasure to sit through in its entirety. There might be worse movies out there, but I shudder to imagine the possibility of their existence.

Heh heh.:

This makes me even more confused as to why Criterion rejected my Eclipse set proposal Matt and His Friends Dicking Around.

Heh heh:

After the actors quickly exhaust their capacity for concocting witty repartee from thin air (Sacha Guitry, these guys are not!) there isn’t much else left when it’s just the three of them except resorting to primal caveman antics and locker-room level horseplay.

You may surmise from the amount of googling going on in this here blog post that I’m not riveted by this movie.

Hey! A hammer! Rip Torn should have been in this movie, too, and beaten some sense into Mailer.

Mailer’s so drunk.

I feel like Peter Falk and Ben Gazzarra should have been in this movie instead.

Halfway through the movie, it seems like they realise that they have nothing, so they start bringing in a series of … “characters” … none of whom can keep themselves from smirking at appearing in the movie.

Except for the dog. The dog’s the best actor here.

Finally! Somebody interesting.

The final ten minutes are kinda good? After that woman with the beehive arrived, things finally took off and became interesting.

The preceding 70 minutes aren’t very good. So:

Wild 90. Norman Mailer. 1968.

This blog post is part of the Eclipse series.

Eclipse 1938: Quadrille

Chirp chirp.

OK, I’ve totally been slacking off on this blog series, and it’s mainly because I’ve been completely busy with other stuff. But it’s also because the Eclipse movies aren’t quite what I imagined they would be.

Criterion touts these movies as lost gems, and the Eclipse box sets I’d seen before this (the Chantal Akerman box, for instance) had indeed been incredible. So I thought these movies were basically movies that weren’t commercial enough to receive the full Criterion Collection treatment: Full restoration, 40 page booklets, interviews with everybody, extras…

Instead I suspect it’s basically movies that Janus Films had the rights to, for some reason or other.

Why Janus? Because Janus Films and Criterion have the same owners.

The selection criteria Criterion is using are basically: Does Janus have it? Nobody else wants it? Let’s put it in an Eclipse box.

It’s not that there haven’t been fantastic movies here and there… but they’re hidden between a lot of stuff that’s only of vague interest.

Like Guitry’s movies, which are quite interesting in many ways, and were huge box office smashes in France at the time… but…

They aren’t so much lost treasures as amusing artefacts.

I do covet those glasses.

This is definitely the least (and the last) of the Guitry movies in this box set. It’s basically a filmed theatre play… an extremely chatty one. They prate and they prate, but about nothing interesting. It’s occasionally quite amusing, but most of the scenes lack zip.

That’s a good ending, anyway.

Quadrille. Sacha Guitry. 1938.

This blog post is part of the Eclipse series.

Faster Music

Since 1997, I’ve been using Emacs as my music playing interface. The albums I have are arranged in a simple .../artist/album/track structure with no database beyond that. The metadata is just stashed in files called “stats” inside each album directory.

This works fine — I don’t have to worry about a database getting out of sync with the data, for instance, when ripping/incorporating new albums into the structure.

But as you can guess, this isn’t the speediest possible setup.

In normal usage, this doesn’t matter — if I want to look at Joni Mitchell’s albums, there’s only a couple dozen of them, and displaying them is pretty much instantaneous.

But some more obscure actions, like if I want to, say, list all albums by release date, it… can take a while.

I currently have 9359 albums. They’re stored on SSDs over USB3 using RAID5, and accessed via NFS from my music playing computer (which has an RME HDSPe Multiface II DAC). So how long do you think it takes to display all those albums chronologically?

(benchmark-run 1 (jukebox-display-directory "/music/chronology/" t t t))
=> (119.39481636800001 107 5.3514199719999995)

That’s 120 seconds.

Which might explain why I rarely do that.

So this is one of those things where I’m going like — should I just bite the dust and put everything into a database? Emacs has SQLite support these days, so writing something that puts all the metadata into a database, and then adding some hooks to my import scripts to update it, wouldn’t be a lot of work. And then generating a buffer like that would be like *snap*.

On the other hand, that’s just kinda boring. Why not throw more hardware at the problem instead?

Reader, I threw more hardware at the problem.

(Besides, that USB3 setup is getting a bit long in the tooth, and will probably need replacing within a few years anyway.)

For max speed, having the music on NVMe SSD in the music playing would obviously be the best — this removes the NFS latency, and NVMe has a lot less latency than USB3.

But I need 8TB storage, and I want RAID1, so that seems unrealistic to find in a machine that’s stereo stack size. But then QuietPC got this nice chassis and motherboard and I went “hm”. What if I could have one 8TB SATA SSD, and one 8TB NVMe SSD, and then use a write-mostly strategy (so that all reads only come from the NVMe)? Would that be… good?

Side note: I’m not sure whether RAID-ing over an M.2 disk is a good idea in general, because we’re using RAID, after all, because we assume that the disks are unreliable, and replacing an M.2 disk is a lot more work than replacing a SATA disk. And using write-mostly means that we won’t be reading from both disks, so it’s not immediately obvious that the lower NVMe latency and higher bandwidth will necessarily be better than the combined oomph from two SATA disks. And finally, it’s not obvious that if, say, the NVMe disk freaks out, that the RAID containing it will continue to work, anyway, so using RAID here doesn’t give me higher availability, and I might perhaps just use a single disk and run a nightly rsync process over to a different disc in the same machine to achieve pretty much the same effect.

I mean, if I’m doing something as ridiculous as this, I might as well eke out absolutely max performance for the most satisfying result.

So… let’s test five different configurations: Single NVMe, single SATA, NVMe + SATA with write-mostly, NVMe + SATA symmetric, and SATA + SATA symmetric.

*phew*

The SATA disks are Samsung SSD 870 QVO 8TB — so the write performance is going to suck anyway: They max out at 160MB/s when doing sustained writes. But writes don’t really matter much (beyond the initial copying, which is running now and takes a while), because the disk is basically static. The metadata is updated now and then, but the .flac files themselves don’t change, and only a couple new albums are added per day.

So I won’t be benchmarking writes, but only reads. And while only one kind of read matters to me (reading directories and metadata), I’ll also be looking at sustained reads, because… I want to, even if it’s largely irrelevant.

Epic unboxing sequence:

Oh yeah… QuietPC uses shredded cardboard as part of the packaging, which is very environmental I’m sure, but means that I’ll be picking up microscopic pieces of cardboard for the next three months…

Hah! I’ll never read that!

A screw fell out. A good sign.

There it is.

Nice and sweet.

My expert lab setup for benchmarking.

I had the 8TB NVMe SSD already, because I’d been wanting to try something like this for a while. But the machine I tried this in didn’t have a free M.2 slot, so I bought a PCIe adapter…

… but the performance was really bad, so I didn’t proceed with that project. (But I didn’t really test a lot, and benchmarking these things is slightly trickier than usual because of the complicated caching behaviours.)

Kinda cool they can smush 8TBs of SSD into a little stick.

The insides of the machine is mostly air, but I especially like that they put the PSU inside the chassis, so that I don’t have to have a huge wall wart somewhere.

That mainboard is half the size of the previous stereo rack machine I had, and has twice as many features.

OK, time to get the Sabrent SSD installed… This mainboard is really nice — it has heat spreaders for the M.2 SSDs.

Installing it was the least fiddly installation process ever, so that’s nice.

Creating a RAID1 over the NVMe and the SATA started off really fast:

But then dropped down to 160MB/s, as expected:

QLC SSDs are kinda slow for mass writes, eh?

OK, benchmarking time! To get a baseline (just to see how much time Emacs is taking doing 10K create-image and stuff, I’ve also copied all the metadata/etc to a RAM disk.

I’ve added two benchmarks in addition to “generate the Chronology buffer”: The “Cat Flac” is the FLAC reading speed, and “Meta” is Emacs reading all the metadata without generating any buffers.

 ChronologyCat FlacMeta
RAM

6.7s

N/A

1.2s
Single SATA

17.1s

487MB/s

9.2s
NVMe

46s

980MB/s

Uhm… this can’t be right. Is the Q NVMe really that bad? It’s said to have a 2TB SLC “cache”, and I’ve just written 4TB worth of data to it, so perhaps it’s… tidying up, distributing the data to the slower QLC parts of the disk?

I’ll give it until tomorrow, and then try again.

 ChronologyCat FlacMeta
RAM

6.7s

N/A

1.2s
Single SATA

17.1s

487MB/s

9.2s
NVMe

15.5s

1080MB/s

6.3s
NFS

99s

N/A

21s
NVMe+SATA

15.6s

950MB/s

6.1s
write-mostly

17.7s

890MB/s

8.4s

So… for generating the Chronology, there’s basically very little difference between the various SSD solutions. The “Meta” shows that the NVMe does, indeed, have lower latency than SATA, but the only surprising thing is how little difference that makes in practice — it’s one third faster.

And write-mostly only makes things slightly slower, and there doesn’t seem to be much point in testing a SATA/SATA RAID1, so I ditched that.

(Take these benchmarks with all the grains of salt that you have — I’m not using a very scienterrific methodology here.)

So there you go: I’m now listening to music coming from a symmetrical NVMe+SATA RAID1, and you can totally hear that. The bass is much more defined, and the trebles have much less jitter.

OH! I’m not writing this for What Stereo? now, am I? Sorry! It sounds the same, of course, but it’s… faster.