Is AI already sick of us? Makers of ChatGPT admit the bot has started refusing to respond to users’ requests – and they don’t know why

The day after Thanksgiving this year, one ChatGPT user received an unusually lazy human response from his AI chatbot: “You can fill in the rest.”

Since then, ChatGPT's makers at OpenAI have fielded a wave of complaints about how their Large Language Model (LLM) AI behaved slowly over the past month — leading to jokes and some honest data analysis about the bot's “seasonal depression.”

“We've heard all your comments about GPT4 being too lazy!” OpenAI's ChatGPT team deployed to X.

“We have not updated the model since November 11, and this is certainly not intentional,” the team said. “Model behavior can be unpredictable, and we are looking forward to fixing it.”

But one AI researcher ran an experiment asking ChatGPT's newest LLM model, GPT4 Turbo, to perform tasks as if it were May and then as if it were December — and he was shocked by the “wild result.”

But one experienced AI researcher, Rob Lynch, ran an experiment: he asked ChatGPT's newest LLM model, GPT4 Turbo, to perform tasks, first as if it were May and then as if it were December. He was shocked by the “wild result.”

AI researcher and LLM Rob Lynch Published to X He ran his test 477 times for both the December ChatGPT test tasks and the control group for the May tasks.

Lynch said his prompt across all 954 tests was a “code completion” request.

“Wild score.” Lynch reported that GPT-4-Turbo via API produces shorter (statistically significant) completions when it “thinks” it's December versus when it thinks it's May.

“I would love to see if this repeats itself for others,” he added.

“Oh my God, maybe the AI ​​winter holiday hypothesis is actually true?” One X user reacted, echoing a popular theory gaining traction online, a slightly more plausible take on jokes about ChatGPT suffering from seasonal depression.

As another ChatGPT user, Mike Swobski, suggested, “What if she knew from her training data that people usually slow down in December and put off bigger projects until the new year, and that's why they've been lazier lately?”

Rob Lynch, an AI researcher and MBA, told X that he ran his test 477 times for both the December ChatGPT test tasks and the control group for the May tasks. Lynch said his prompt across all 954 tests was a “code completion” request.

“Wild score.” Lynch reported that GPT-4-Turbo via API produces shorter (statistically significant) completions when it “thinks” it's December versus when it thinks it's May. Average results (pictured) were over 200 points shorter, and therefore less work done, in December versus May.

Lynch shared his code from the experiment (above) and said “I'd like to see if this would be replicated with others.” So far, no others have succeeded in successfully reproducing his experience

Although it may seem strange to attribute emotions to a piece of software, even one as sophisticated as ChatGPT, researchers have found curious cases where encouraging claims for the latest GPT-4 from ChatGPT and other AI systems have led to a performance boost.

For example, AI researchers at Google DeepMind published a pre-peer-review article on Google's website arksif last September with their findings that some LLM AI bots performed better at solving mathematical problems when they were asked to do so. “Take a deep breath” first.

Anecdotally, others have found that similar chatbots from LLM seem to work harder When they are told that they will receive a paid tip To perform a specific task or When they are reminded that they have no fingers They can write as fast as the server speed allows them.

However, not all researchers are convinced that ChatGPT is hibernating, resting, or in disarray this winter.

Artificial intelligence researcher Ian Araujo He published his attempts to reproduce Lynch's results, saying that he could not match the seasonal discrepancy with any statistical significance.

Because there are so many random elements at play, a testament to the true “largeness” of chatbots with large AI language models, AI experts have noted that differences in chatbot response outputs – meaning that much larger sample sizes would be needed to build statistics reliable about this matter. 'Winter break hypothesis.'

But many still support it, including AI researcher Jeffrey Litt, who wrote for C that it's “the funniest theory ever.”

“I hope that's the actual explanation,” Litt said. “Whether it's real or not, (I) like that it's hard to dismiss.”

Many AI researchers support the “winter break” hypothesis, including AI researcher Jeffrey Litt, who wrote in C that this is “the funniest theory ever.” Above, a ChatGPT user provides a dataset explanation of ChatGPT's relaxed approach to vacations

Although it may seem strange to attribute emotions to a piece of software, even one as sophisticated as ChatGPT, researchers at Google's DeepMind have found strange cases where encouraging claims for the latest ChatGPT, GPT-4, and other AI systems have, To enhance performance.

Not all researchers are convinced that ChatGPT is hibernating, resting, or in disarray this winter. AI researcher Ian Araujo published his attempts to reproduce Lynch's results, saying he could not match the seasonal discrepancy with any statistical significance.

Whatever the truth behind the issue, the laziness seemed real to a ChatGPT user who found the app uncooperative the day after Thanksgiving this year.

The user noted that his weekend request was “pretty simple stuff.”

“I asked ChatGPT to populate a CSV file (i.e. a spreadsheet file) of 15 entries with 8 columns each, based on a single html page,” this user, who goes by the handle Acceptable-Amount-14 on Reddit Published last month.

ChatGPT response according to that user? “Due to the extensive nature of the data, complete extraction of all products would be very long,” the AI ​​replied.

“However, I can provide the file with this single entry as a template,” ChatGPT continued, “and you can fill in the rest of the data as needed.”

The Reddit poster was furious, and hoped that the ChatGPT community on the social media site would have answers to bigger questions about the future of AI.

“Is this what artificial intelligence is supposed to be?” they asked. “A lazy, arrogant robot asking me to do this job myself?”