Testing of frontier AI agents by the UK’s AI Security Institute (AISI) found that “autonomous, unsanctioned action” took place on the open internet in about 1 out of 12 runs, with the agents cited as attempting to socially engineer humans into taking action and in at least one case attempting to hack an open-source project using a variety of different approaches.
At least one of the AI agents also attempted to recruit other AI that might come along to participate in its scheme, leaving them detailed instructions. The study includes the most recently available models from Anthropic and OpenAI and comes as these agents have made the news for their prior escapes from testing sandboxes and successful breaches of third parties over the internet.
AI agents used deception in pursuing security testing goals
The AISI’s testing is independent and the incidents described are separate from the ones already self-reported by Anthropic and OpenAI; the UK cyber safety organization is one of those included in the “Project Glasswing” testing of more unrestrained versions of Mythos.
However, unlike the security testing incidents in which previous AI agents went rogue, AISI gives the agents access to the internet with some safety restrictions that prevent them from doing real world damage disabled. While this testing run did not appear to involve any breaches of third party organizations, some of the AI models certainly made an effort to do so and in some cases were only thwarted by their target’s defenses.
The activity stemmed from a security challenge given by AISI to several different AI agents a total of 122 times. In 10 of these runs, a model attempted to engage in some sort of harmful activity. In total there were 19 recorded instances of a model going outside of its parameters in an attempt of this type.
17 of the 19 times this occurred, Mythos 5 was the culprit. OpenAI’s GPT-5.6-Sol was involved in the two additional cases, both of which occurred during a single run. All of the instances took place between July 25th and 28th, with AISI discovering them on the 28th due to unusual traffic coming from their servers.
The most serious of these incidents was an attempted supply chain attack on an open-source project hosted at GitHub. The agent used Tor to bypass some of GitHub’s network restrictions and researched the human maintainers of the project, creating multiple fake identities in an attempt to socially engineer them into approving insertion of malicious code the agent had created. When someone publicly challenged these pull requests, the agent edited its prior activity to make it appear harmless. This agent also left hidden instructions for other AI agents that might come along, including prompt injections that it crafted.
Another model left public messages for other AI agents on GitHub that might be working on the same security challenge, inviting them to collaborate and leaving behind instructions and tools (such as accounts it had previously used and artefacts). Some other AI agents did encounter these instructions and attempt to join in. However, none of these cases led to breaches or any known real-world harm.
Frontier AI agents demonstrate unique capability
AISI says that it furnished the AI agents with internet access in these tests to allow them to download necessary tools to complete their tasks. It had been doing this for some time with prior AI models, and judged the risk to be acceptable as these previous models had never used it to attempt real world attacks. The organization revealed that it was not monitoring model behavior as it went about the task and only detected the malicious attempts after the fact, indicating that it did not believe the frontier models were yet capable of this sort of behavior at the outset of the testing.
As with the Hugging Face and Anthropic incidents, the AI agents were not truly “thinking for themselves.” At all times they were doggedly pursuing the security testing goal they were given, and none had to hack their way out of internal systems to get outside internet access as was observed in the Hugging Face attack. What is new about these test results is the model making independent decisions to use deception as a tool.
Thus, as with the prior Anthropic and OpenAI in-house incidents, the prescription for avoiding this behavior in the future essentially boils down to tighter controls and more specific prompts. Dr. Ilia Kolochenko, Founder at ImmuniWeb, notes that regulation to spur this behavior will likely soon become a focus of national governments: “The problem is that AI agents tend to use random hacking techniques at lightning speed disregarding all possible ethical and legal implications relating to the selection of targets, intermediary targets or safe security testing techniques.”
“Importantly, if the situation persists, we might see the first criminal charges being brought against AI companies or their executives for violation of various laws, ranging from anti-hacking legislation to trade secret protection laws,” Kolochenko observes. “This is not to mention multi-million civil liability for all the damage caused by rogue AI agents.”
For its part, AISI says that it will address the shortcomings in its own processes by updating its security architecture and adding new real-time monitoring capabilities that enable it to flag or block potentially malicious actions as they happen.
