Over on the movie blog I’m still trying to answer the question whether 80s arthouse movies suck or not.
And I knew from that dataset that Querelle had one single vote in the Sight & Sound 2022 poll, but I wondered who had voted for it.
I tried googling, and I tried binging, and both came up short. So I thought: The advertised use case for LLMs is that they’re supposed to be able to use more context to answer queries. The search engines aren’t able to find solution possibly because there’s no single web page that says both “querelle” and “sight and sound 2022 poll”?
So I tried ChatGPT:
Huh! Really? The data set was wrong? There were two people who voted for Querelle in the Critic’s poll — Michael Koresky and Mark Cousins?
OK, let’s look at Mark Cousin’s response:
Err… No Querelle.
Er… Nothing in Michael Koresky’s list either.
So ChatGPT was just doing it’s usual thing — straight up lying when it didn’t know the answer.
When these “hallucinations” come up, people often recommend using Perplexity instead:
Bla bla bla that’s a long way to say “fuck off; I have no idea”. Which is a better response for sure, but it’s not actually useful.
But restricting the google to this, however: site:https://www.bfi.org.uk/sight-and-sound/greatest-films-all-time/all-voters/ querelle
Yes!
It was Bing Wang! (And I have to say that that’s a quite bad top 10 list — two of those movies are awful, and a couple is about the worst movies that director did. But I digress!)
In conclusion: LLMs still suck. And waste our time. Sure, LLMs are just spicy autocomplete — if there’s a clear answer to anything, a simple search will give you the result (i.e., the Wikipedia page where the topic is discussed). If it’s an obscure question, the LLMs will either lie to you or tell you to fuck off.
You’d think that these plagiarism machines would be better by now… but I guess they work well enough for their primary purpose: Generating essays for college students. But as somebody wise once said: “this is only possible to argue if you are foolish enough to believe that the purpose of assigning essays to students is to increase the number of essays in the world”.