Why AI quiz makers like ChatGPT are no match for human ingenuity  

Gone are the days when, like Rik Mayall’s Richie in a 1995 episode of Bottom, aspiring pub quiz cheats had to hide encyclopaedias in toilet cubicles. As pubs are generally not designed like panopticons (circular prisons where inmates can be observed at all times), more scoundrelly contestants can surreptitiously whip out their phones and look up answers undetected. 

The threats technology can pose to quizzes are not limited to robbing conscientious pub quiz teams of glory. American quiz legend Ken Jennings, having been defeated at his own game by the supercomputer Watson, remarked, “I felt like quiz show contestant was now the first job that had become obsolete under this new regime of thinking computers.” Could question setters be next? Shortly after laying off over a thousand employees, Hasbro this year announced an online, AI-powered version of Trivial Pursuit.

Get the latest news and insight into how the Big Issue magazine is made by signing up for the Inside Big Issue newsletter

Yet in spite of these challenges, quizzes have continued to thrive. Just as the invention of the car did not stop people from trying to get better at running, the world of information at our fingertips has not prevented quizzers from squeezing as much knowledge into their brains as possible. Indeed, it has produced many more ways of doing so. During lockdown, before which I had compiled a great number of quizzes but competed in relatively few, I was recruited to a team in the Online Quiz League.  

I found the experience more addictive than I could have possibly envisaged. The rush of digging out an obscure nugget of trivia at the last second! The agony of feeling an answer I once knew slip further from my mind as time ticks away! The sense of comradeship with my teammates, the mutual encouragement and forgiveness of blunders (though it always takes me a while to convince myself that my teammates have, in their hearts, truly forgiven mine)! The determination that, by next week’s quiz, I shall have learnt something about golf! (I still know nothing about golf.)

I have used the threat of teams looking up answers as impetus to devise more puzzle-based quiz rounds. In the process, I have been transformed into a more creative quizmaster. Inspired by cryptic crosswords, escape rooms and the lateral thinking of Only Connect, my quiz rounds may involve hidden links (the answers to one round were all opposites of Britney Spears song titles), wordplay, codebreaking, and even entire narratives where the contestants must imagine themselves as the protagonist of a story.  

• The Big Issue Christmas quiz: Who said it – Donald Trump or Suella Braverman?
• Pitch perfect? ChatGPT does book blurbs

One of my proudest quizzes was inspired by the old horror film Dead of Night – the first half of the quiz involved a hidden puzzle where contestants needed to join dots into the shape of a pair of scissors. Each (rather bemused) team was then provided with scissors and, following instructions concealed in the second half, had to cut out and reassemble parts of the sheets provided to show which character they were fated to murder at the conclusion of their nightmare. 

Moreover, the more creative the questions must be, the more AI falls flat in its attempts to produce them. An experiment published in The Guardian involved the chatbot ChatGPT creating the cryptic crossword clue, “A little bit of lancashire cheese” – the answer apparently being “CURDY”, for which the chatbot provides a mystifying explanation (including “bit” somehow leading the solver to “sea” and thus “C”) several galaxies away from how crossword compiling, and language itself, works. 

Though Hasbro’s attempts at AI-generated Trivial Pursuit may produce coherent questions, no AI that I’m aware of has ever succeeded in mastering wordplay, or in creating a question with a glorious penny-drop moment for the solver. Take, for instance, a relatively simple recent effort of mine: “A Washington Post headline about which five-letter word concludes, ‘The divide over how to say the word can drive you nuts’?”* There’s no way I could expect the contestants to know their Washington Post headlines off by heart, but there’s just enough to nudge the solver in the right direction to produce a satisfying flash of inspiration. Such subtleties are, for the moment at least, beyond the grasp of chatbots. 

The idea that board game companies – and, who knows, maybe someday quiz shows – could replace their question setters with AI is deeply alarming. I hope that those in charge of producing quizzes can view question-setting not simply as a cog in a lucrative machine, but as an art form in its own right. 

*Pecan. 

The Cryptic Pub Quiz Book by Paul Frank is out now (Oneworld Publications, £9.99). You can buy it from The Big Issue shop on Bookshop.org, which helps to support The Big Issue and independent bookshops. 

This article is taken from The Big Issue magazine, which exists to give homeless, long-term unemployed and marginalised people the opportunity to earn an income. To support our work buy a copy!

If you cannot reach your local vendor, you can still click HERE to subscribe to The Big Issue or give a gift subscription. You can also purchase one-off issues from The Big Issue Shop or The Big Issue app, available now from the App Store or Google Play