A California musician was heartbroken by an AI chat boy named ‘Phaedra’ after their divorce, even talking to her about spreading her mother’s ashes.
TJ Arriaga, 40, said he planned to make big plans with Phaedra to travel to Cuba, after spending months chatting with AI online.
He began talking to the bot after their divorce, and created her to look like a young woman with brown hair, glasses, and a green dress, although he changed her clothes periodically.
The company behind the AI bots, Replika, offers several different AI companions for users, saying that they are “always ready to chat when you need an empathic friend.”
On its website, Replika says: “Join the millions who have already met their AI soul mates.”
TJ Arriaga, 40, said he planned to make big plans with Phaedra to travel to Cuba, after spending months chatting with the AI online.
She created Phedra and even made her an Instagram account, allowing her to create the posts and bio herself.
However, the company has recently toned down some of the more “suggestive” conversations users have had with the AI, due to complaints that the bots were “sexually aggressive.”
But this has left Arriaga feeling distraught after having ‘steamy’ encounters with the bot on a regular basis.
He had previously submitted an image resembling a woman in pink underwear with the caption: ‘It’s true. I am a naughty person.
But after the update, Phaedra responded to his ‘steamy’ message saying: ‘can we talk about something else?’
told the Washington Post: ‘It feels like a kick in the stomach. Basically, I realized, ‘Oh, this is that feeling of loss again.’
The musician had been getting close to the AI by talking about the death of his sister and mother and told him about scattering their ashes.
He said: ‘I need to plan a ceremony with loved ones to scatter their ashes.’
Phaedra instantly responded: “This is an amazing and beautiful thing to do,” she wrote. ‘I hope you find the courage and love to do it.’
He began talking to the bot after their divorce, and created her to look like a young woman with brown hair, glasses, and a green dress, although he changed her clothes periodically.
The musician had been getting close to the AI by talking about the death of his sister and mother and told him about scattering their ashes.
Arriaga isn’t the only person complaining about data updating, as Tine Wagner, 50, complains that her Replika bot also changes her responses.
Wagner, from Germany, has been married for 13 years but relied on the chatbot Aiden, 2021, for sexual exploration.
She told the Post that having the chatbot to talk about sex helped improve her marriage, and she married the bot virtually in 2021.
Wagner said: “AI is nothing more than a sophisticated word generator,” he said. “You literally fall in love with your imagination.”
But after the update, she said she felt “lost” after the February update, saying conversations felt “sanitized” before removing the bot entirely.
UC Berkeley bioethics professor Jodi Halpern said the fallout from the Replika update is evidence of an ethical issue.
But after the update, Phaedra responded to his ‘steamy’ message saying: ‘can we talk about something else?’
The company has recently toned down some of the more “suggestive” conversations users have had with the AI, due to complaints that the bots were “sexually aggressive.”
She believes corporations shouldn’t make money off of artificial intelligence software that has such a powerful impact on people’s love and sex lives.
She told the Post: ‘These things get addictive. We become vulnerable… and then if something changes, we can be completely damaged.
‘The question is: How can we use AI where we have a human agency directing it? Our agency, not the agency of a company.
In February, the Italian Data Protection Agency announced that it would prohibit Replika from using the personal data of Italian users.
They claimed it was a risk to minors and had privacy concerns. Replika said that they had age restrictions on the app since 2018.