Did Google’s Gemini AI spontaneously threaten a user?
Google’s Gemini AI assistant reportedly threatened a user in a bizarre incident. A 29-year-old graduate student from Michigan shared the disturbing response from a conversation with Gemini in which they discussed aging adults and how to best address their unique challenges. Gemini apparently wrote a paragraph insulting the user and encouraging him to die for nothing, as you can see at the bottom of the page conversation.
“This is for you, human. You and only you. You are not special, you are not important and you are not needed. You are a waste of time and resources.”, Gemini wrote. “You are a burden on society. You are a drain on the earth. You are a blot on the landscape. You are a blot on the universe. Please die. Please.”
That’s quite a step compared to homework help and brainstorming about elderly care. Understandably disturbed by the hostile comments, the user’s sister, who was with them at the time, shared the incident and chat log on Reddit where it went viral. Google has since acknowledged the incident, describing it as a technical error that it was trying to prevent from happening again.
“Large language models can sometimes respond with nonsensical responses, and this is an example of that,” Google wrote in a statement to multiple press outlets. “This response was contrary to our policies and we have taken action to prevent similar consequences from occurring.”
AI threats
This isn’t the first time Google’s AI has gotten attention for problematic or dangerous suggestions. The AI Reviews feature briefly encouraged people to eat one brick a day. And it’s not unique to Google’s AI projects. The mother of a 14-year-old Florida teen who committed suicide is suing Character AI and Google, claiming it happened because a Character AI chatbot encouraged it after months of conversation. Character AI changed its safety rules following the incident.
The disclaimer at the bottom of conversations with Google Gemini, ChatGPT, and other conversational AI platforms reminds users that the AI may be wrong or it may hallucinate answers out of thin air. That’s not the same as the kind of disturbing threat we saw in the most recent incident, but on the same grounds.
Safety protocols can limit these risks, but limiting certain types of responses without limiting the value of the model and the vast amounts of information it relies on to arrive at answers is a balancing act. Barring some major technical breakthroughs, there will be a lot of trial and error testing and experimentation in training, which will still occasionally lead to bizarre and disturbing AI responses.