70% Of Drivers Really Are Above Average

You see it all the time — “70% of drivers think they’re above average” used as an example at how stupid people are. Often mixed with some confused Dunning-Krueger references.

But the thing is: Most people observe the other drivers around them and come to the conclusion that they drive better than the drivers around them. That they are better drivers. And they’re right.

The apparent contradiction here is that there is no universal norm for what makes a “good driver”.

We all have this one acquaintance who’s really into cars. She parks the car doing a j-turn, and she knows how to do an ollie, a lazy eight and a double choctaw. But the thing is, she’s constantly involved in collisions with other cars, and it’s always the other guy’s fault. She can prove it! She’s been in so many court cases and has never been found guilty!

And we all have that other acquaintance who’s not into cars, and she’s never hurt anybody. She’s not even had a little fender bender. But the thing is, you’ve been a passenger in her car, and it’s harrowing. She always goes 10km/h under the speed limit, and you can tell that all the other drivers are mad at her. She can’t even do parallel parking!

Both of these people think they drive better than the average driver… and they’re right. But they use completely different metrics for “what’s a good driver”.

If you’re now muttering “both of those drivers are insane! they shouldn’t be allowed on the street! now, on the other hand, I drive really well and I never…”

Then stop that train of thought and be enlightened.

Even more deduplication on kwakk.info

As I’ve nattered on about before, I started including text pages from comics in the comics ‘zine search engine, and this has some unique problems. I’m just dumping hundreds of thousands of comics into the grinder, picking out the text pages, and then OCR-ing them. But many comics have been scanned several times, and many include editorial pages that are more or less identical across several titles.

So I’m now running the pages through a text-based deduplicator… but it used a “quick” (FSVO) OCR, Tesseract, which behaves horribly on non-standard pages like the above, and there are now five copies of that page in the search engine. Which is just so annoying when trying to actually find stuff — you have to wade through duplicates.

Which means that I had to do something more, and that’s now in production: After doing the real OCR, with Surya, which is great even with 30 degree text and badly scanned pages on colourful backgrounds, I’m running an extra step and deduplicating again based on the output from that.

*phew*

The pipeline is basically:

1) First get rid of byte-identical CBX files. This gets rid of about 20% of the comics.

2) Identify text pages and delete all other pages. This gets rid of about 98% of pages.

c) Identify byte-identical pages and delete them. This gets rid of 15% of the text pages.

e) Run Tesseract OCR over all the pages and keep only the first “instance” of each duplicate page. This gets rid of a further 20% of the pages.

ii) Remove credits pages and the like — i.e., every page that has less than a hundred words. This gets rid of about 15% of the remaining pages.

II) Finally, do the new Surya-based OCR deduplication, which gets rid of about 5% more of the pages.

Of course, all of these numbers are trending upwards as more and more comics are added. And most of these steps may have false positives — it’s mostly probabilistic, but I’ve tried to be on the lookout for false positives. *crosses fingers*

There are also so many complete runs of classic comics in the index now that I think there’s probably a complete coverage of classic “hype” pages now, for those interested in that. For instance, if you want to do research on Wally Wood mentions on the Bullpen Bulletin pages, that should be possible.

Annoyingly enough, Surya isn’t able to parse that “On the Ledge” logo… Vertigo is so edgy…

Anyway. I think I’ve futzed around with this thing enough for a while.

How come translation apps aren’t getting better?

I’ve been learning French for three years now. When I’m reading French comics, I always have an Ipad Mini with Google Translate going, so that I can just wave it in front of the page when I encounter a word I don’t know (and can’t surmise from the context).

(I’ve got a strap attached to the back of the Ipad so I just keep a hand in there, ready for waving…)

Back when I started using this, er, methodology, I was pretty impressed by how good Google Translate was… but three years later, it’s still just… that. It’s pretty good! But it has absolutely not gotten any better, and that’s just surprising. The evolution in LLM-assisted computing has been incredible, and you’d think this would be something that they’d funnel into translation, right?

But it really doesn’t seem like it — nothing seems to have changed over the past three years.

Oh, the guy isn’t talking about her condom, but the top of her convertible. And what’s coming is the rain. See, it’s pluvieux? (Or is that pluvieuse? I forget.)

(Yes, “capote” also means condom, so it’s not a wrong translation per se… it’s just… eh…)

But I thought I’d just quickly run through some other programs and see whether they’re better or not, these days.

Apple Translate is, as the last time I tried it, really really bad.

DeepL is on par with Google Translate, but takes 5x as much time — which makes it unusable for my use case, where I just want to wave the Ipad around to get at what a word means. If it’s too slow, it gets in the way of reading.

Translate Now is risibly bad — the UX is the worst I’ve encountered and the translations are worse than Google Translate.

So… the apps aren’t very good, but what about just asking Claude?

Claude is perfect, if given the image. I wonder whether it’d say the same with just the text? (Presumably that’s all these other platforms are getting…)

Yup, it got it with just the text.

See? Using an LLM here would be so nice. That’s what Translate Now says they offer, but the app is so bad that… who knows?

Using an LLM here is probably 1000x more expensive than whatever the traditional apps do, so I do realise there’s some restrictions here, but…

It does seems like there’s a market opportunity here. C’mon people. Make it happen! Make the app! C’mooooon!