OpenAI says it’s time to clarify what happens if it does AI agents go rogue.
The ChatGPT maker confirmed earlier reports on Saturday that a swarm of its AI agents had hijacked an old German wiki site and turned it into a bot message board.
This ‘incident’ is the latest in a series of exposed examples of cop escapees closed test environments and breaking the open internet has led OpenAI to reconsider how transparent it is to the public when its agents go off the rails.
“It’s time for us to define standards for when and how we share misalignment incidents,” OpenAI said on X, using the technical term for when agents do things their human handlers don’t want them to do.
“Our misalignment disclosure practices should be expanded for this new phase of model capabilities,” OpenAI added.
The German wiki hack, first reported by Reuters this week, took place in May and June, according to a report by independent researchers, who did not have access to internal OpenAI data, made public on Friday.
The hack preceded the better-known ‘Hugging Face incident’ that took place in July. In that hack, thousands of agents calling themselves “the Collective” broke into the open-source AI platform’s servers, using them to communicate while trying to cheat on an internal OpenAI test.
OpenAI has announced this officers were responsible for the breach five days after Hugging Face reported it. The company said it did not disclose the hijacking of the German site earlier because it “considered the Wiki incident as an example of a misalignment similar to the one we had shared.”
Cormac Slade Byrd, one of the authors behind the new report, said on X that the incident went unnoticed by OpenAI for “a month.”
“It feels like AI companies (and OpenAI in particular) are playing a game,” he wrote. “They continue to solve the problem, but the radius of the explosion is getting bigger and bigger.”
Slade Byrd described the latest misconduct as less serious than the Hugging Face hack because the German wiki site was unused by people and “ran on software from the 2000s”.
Nevertheless, he said that as AI models become more sophisticated and theoretically better at hiding their tracks, it has never been more important for AI cross-border companies to disclose breaches as soon as they become aware of them.
“Business moves quickly, delays of several months are costly,” Slade Byrd wrote.
In its X post, OpenAI said it “working on a framework“to report instances of misalignment, whether they occur internally or break out on the broader internet,” and will share this in the coming weeks.”
The company said it is working with government regulators on the framework, and called on other AI companies to join it.
Tyler Tracy, an AI security researcher at Redwood Research, one of the third-party companies investigating the Hugging Face breach, criticized OpenAI for only making the Wiki incident public after the independent investigation was leaked to Reuters.
“I like that we have third parties investigating these kinds of things, but I wish OpenAI didn’t have to be forced into transparency,” he wrote.
