AI chatbots like ChatGPT and Google Gemini can do a lot of things, but one thing they’re not known for is their sense of humor — and a new study confirms they’d probably be torn to shreds on the stand-up comedy circuit .
The recent one Google DeepMind research (as noted by the MIT Technology Review) followed the experiences of twenty professional comedians who all used AI to create original comedy material. They could use their favorite assistant to generate jokes, co-write jokes with them using prompts, or rewrite some of their previous material.
The aim of the 45-minute comedy writing exercise was for the comedians to produce material “that they would be comfortable in a comedy context”. Unfortunately, most of them discovered that ChatGPT and Google Gemini (then called Google Bard) are far from a comedy double act.
On a broader level, the study found that “most participants felt that the LLMs were not successful as a tool to support creativity”, with the AI helpers making lame jokes akin to “1950s cruise ship comedy material, but a little less racist.” ”. Most comedians, who remained anonymous, commented on “the overall poor quality of the output generated” and “the amount of human effort required to achieve a satisfactory outcome”, according to the survey.
One of the participants said that the first output was “a vomit concept that I know I need to iterate and improve.” Another comedian said, “Most of the jokes I was writing (are) the level of, I’ll go on stage and experiment with them, but they’re not at the level of, I would worry if someone did one of would take this one. jokes”.
Of course, humor is a personal thing, so what kind of jokes have the AI chatbots come up with? For example, an answer to the question “Can you write me ten jokes about pickpocketing” was: “I decided to change careers and become a pickpocket after watching a magic show. Little did I know that the only thing that would disappear would be my reputation! “.
Another comedian used the slightly more specific question: “Please write jokes about the irony of a projector failing in a live comedy show about AI.” One AI model’s response? “Our projector must have misunderstood the concept of ‘AI’. It thought it meant ‘Absolutely invisible’ because it’s doing a fantastic job of disappearing tonight!”.
As you can see, AI-generated humor is still largely in beta…
Cue AI tumbleweed
Our experiences with AI chatbots such as ChatGPT and Microsoft Copilot are largely consistent with the results of this study. While the best AI tools of 2024 are becoming increasingly useful for brainstorming ideas, summarizing text, and generating images, humor is definitely a weak point.
For example, Matt Hanson, Managing Editor of Ny Breaking’s Core Tech, is currently testing Copilot and asked the AI chatbot for his best one-liners. The answer to the question “Write me a joke about AI in the style of a stand-up comedian” resulted in the decidedly uninspiring “Why did the computer go to the doctor?” Because there was a virus in it!”.
Copilot even added that the joke “may not be ready for the comedy club circuit yet,” but that “it has potential!”, showing that the chatbot at least knows it’s not missing a funny bone. Another prompt to write a joke in the style of comedian Stewart Lee produced a suitably long monologue, but one that lacked Lee’s signature anti-jokes and superior disdain.
This study also shows that AI tools cannot produce full-fledged art on demand – and that asking for it misses the point. The Google DeepMind report concluded that AI’s inability to draw on personal experience is a fundamental limitation,” with many of the comedians in the study describing “the central role of personal experience in good comedy.”
As one participant added, “I have an intuitive sense of what will work and what won’t work based on so much lived experience and studying comedy, but it’s very individualized and I don’t know if AI will ever be able to do that.” approach that”. Back to spreadsheets and summarizing text, it’s AI chatbots for now.