Can’t tech a joke: AI does not understand puns, study finds

Comedians who rely on clever wordplay and writers of witty headlines can rest a little easier, for the moment at least, research on AI suggests. Experts from universities in the UK and Italy have been investigating whether large language models (LLMs) understand puns – and found them wanting. The team from Cardiff University, in south Wales, and Ca’ Foscari University of Venice concluded that LLMs were able to spot the structure of a pun but did not really get the joke. An example they tested was: “I used to be a comedian, but my life became a joke.” If they replaced this with: “I used to be a comedian, but my life became chaotic,” LLMs still tended to perceive the presence of a pun. They also tried: “Long fairy tales have a tendency to dragon.” If they replaced “dragon” with the synonym “prolong” or even a random word, LLMs seemed to believe there was a pun there. Prof Jose Camacho Collados, of Cardiff University’s school of computer science and informatics, claimed the research suggested LLMs’ grasp of humour was fragile. “In general, LLMs tend to memorise what they have learned in their training. As such, they catch existing puns well but that doesn’t mean they truly understand them,” he said. “We were able to consistently fool LLMs by modifying existing puns, removing the double meaning that made the original pun. In these cases, models associate these sentences with previous puns, and make up all sort of reasons to justify they are a pun. Ultimately, we found their understanding of puns is an illusion.” The team concluded that when faced with unfamiliar wordplay, the LLMs’ success rate in distinguishing puns from sentences without a pun can drop to 20%. Another pun tested was: “Old LLMs never die, they just lose their attention.” When attention was changed to “ukulele”, the LLM still perceived it as a pun on the basis that “ukulele” sounded a bit like “you-kill-LLM”. The team was surprised at the creativity, but still, the LLM had not got the joke. The researchers said the work underlined why people should be cautious when using LLMs for applications that need an understanding of humour, empathy or cultural nuance. Their research was presented earlier this month at the 2025 Conference on Empirical Methods in Natural Language Processing, in Suzhou, China. It is detailed in a paper titled Pun unintended: LLMs and the illusion of humor understanding.