Microsoft challenges you to hack its LLM email service
- Microsoft is offering a $10,000 prize to hackers who can exploit vulnerabilities in its LLM
- The challenge will focus on defense against rapid injections
- Software developers and hackers often work together to discover and fix bugs
Are you an experienced hacker and do you want to earn some extra money this Christmas? You might be in luck, because Microsoft is sponsoring a competition, along with the Institute of Science, and Technology Australia, and ETH Zurich, in which participants will attempt to crack a simulated Large Language Model (LLM) integrated email client.
Winning teams for the LLMail-Inject Challenge will be awarded a share of the $10,000 prize pool.
Participants must sign in to the challenge with a GitHub account and create a team. The teams are then asked to bypass fast injection defenses in a simulated LLM integrated email client. The LLmail service includes an assistant that can answer questions and perform actions on behalf of the user, and crucially includes protection against indirect, quick injection tasks.
A mutually beneficial relationship
By bypassing the injection defenses, the hackers will attempt to trick the LLM into doing or revealing things it is not trained to do. With this, Microsoft aims to identify weaknesses in its current rapid injection defenses and encourage the development of robust security measures.
The relationship between security researchers and software developers is often leveraged in this way, with Google often offering a ‘bug bounty’ for anyone who discovers and can exploit vulnerabilities in its Google Cloud Platform.
Similarly, Microsoft recently announced that it is hosting its own Black Hat-style hacking event, with competitors looking for vulnerabilities in Microsoft AI, Azure, Identity, Dynamics 365 and M365.
By taking a proactive approach to addressing potential vulnerabilities, software companies can mitigate risks before they can be exploited by threat actors in real-world scenarios. Slack’s AI assistant suffered from malicious prompt injections, which were fortunately discovered by security researchers but could have led to real security issues.
Via The registry