Claude AI and other systems may be vulnerable to concerning command prompt injection attacks
- Security researchers tricked Anthropic’s Claude Computer Use into downloading and executing malware
- They say other AI tools can also be tricked by rapid injection
- GenAI can also be tricked into writing, compiling, and executing malware
In mid-October 2024, Anthropic released Claude Computer Use, an Artificial Intelligence (AI) model that allows Claude to control a device – and researchers have already found a way to exploit it.
Cybersecurity researcher Johann Rehnberger recently described how he was able to abuse computing and trick the AI into downloading and executing malware, and how to get it to communicate with the C2 infrastructure, all via prompts.
While it sounds devastating, there are a few things worth noting: Claude Computer Use is still in beta, and the company has left a disclaimer stating that Computer Use may not always behave as intended: “We don’t recommend to take precautions to isolate Claude from sensitive data and actions to avoid risks associated with rapid injection.” Another thing worth mentioning is that this is a quick injection attack, which is quite common among AI tools.
“Countless ways” to abuse AI
Rehnberger calls his exploit ZombAIs and says he was able to obtain the tool to download Sliver, a legitimate open source command-and-control (C2) framework developed by BishopFox for red teaming and penetration testing, but it is often abused by cybercriminals as malware .
Threat actors use Sliver to establish persistent access to compromised systems, execute commands, and manage attacks in a similar way to other C2 frameworks such as Cobalt Strike.
Rehnberger also emphasized that this is not the only way to abuse generative AI tools and compromise endpoints via rapid injection.
“There are numerous other ways, such as another way to have Claude write and compile the malware from scratch,” he said. “Yes, it can write, compile and run C code.”
“There are many more options.”
In his writing, The hacker news The DeepSeek AI chatbot was also found to be vulnerable to a quick injection attack that allowed threat actors to take over victims’ computers. Additionally, Large Language Models (LLM) can execute ANSI escape code, which can be used to hijack system terminals via prompt injection, in an attack called Terminal DiLLMa.