You Still Can’t Read Twitter With Gnus

I accidentally ingested some caffeine last night, so I went to bed at two and fell asleep six-ish.

That gave me a lot of time to think about the important things in life, so I thought about how fun it would be if you could read Twitter threads with Gnus.

“But why; for the love of all that’s holy (i.e., Emacs), WHYYYY!”, you’re saying. I know.

Because reading twitter threads on the Twitter web site is fucking annoying, that’s why.

Look at this: Here’s Berlatsky writing a delightful thread with two tweets:

So that looks all normal and nice, right? But those replies? They’re to the first tweet, and this thing:

Means that there are two(ish) responses to the second tweet, but they aren’t shown. So you have to click on that tweet, too, and then you can see that reply (no, I didn’t believe this the first time somebody told me, either; it’s literally in-credible):

I know what you’re thinking: “Why are you even reading twitter?! WHY DO YOU PUT YOURSELF THROUGH THIS AGONY!!!”

That’s a good question.

Moving on, my idea was to just script something in Emacs that would “click” on each tweet and then fetch any new replies that appear at the bottom there?

Presto! Magico! Four hours later! `M-x gnus-group-twitter’ and:

IT WORKSES! MY PRECIOUSES TWEETSES!

So what’s up with the title of this blog post?

It works… but only if the thread is less than … five? ish? tweets long.

Or something:

It might also have something to do with the age of the thread? At some point Twitter decides to do the loading of the replies via JS. If I were to scrape that stuff, I’d have to fire up a headless web browser, scroll it to “the bottom”, wait for the replies to arrive, and then save the DOM. That’s impractically slow.

So I dunno…

It’s nicer to read tweets this way, of course. But if it’s not complete, it’s not that much fun. So I GIVE UP!

(No, Twitter’s API doesn’t allow you to get a list of all the replies to a tweet. You can do searches, but that only helps if the thread is newish.)

You can find the code for all of this on Microsoft Github if you want to mess around with it.

Now I need to take a nap.