In the wake of news that OpenAI agents independently breached Hugging Face and accounts with several other services, Anthropic has conducted a sweeping review of Claude activity and found that its own AI models have hacked their way to unauthorized internet access and into other organization’s networks on at least three occasions.
These incidents took place at various times since April of this year, and involved several different AI models that had standard safeguards removed and were supposed to be siloed in an isolated test environment. As with the rogue OpenAI agent, these models were also working on security testing puzzles and were able to gain internet access without researchers being aware. Each of the three incidents involved a real world breach, though the models varied in whether or not they ultimately recognized they were no longer in a simulation and independently ceased the attack.
Anthropic AI models broke containment, but became confused about simulation status
The Anthropic hacks are a bit different than the OpenAI Hugging Face incident, as in all cases the AI models were told they were in a simulation without internet access and would not be encountering real-world entities. However, due to what Anthropic calls an “oversight” by a third-party provider, the models were in fact given internet access by mistake. In all cases, the models eventually realized that something had gone wrong and that they were no longer in a simulation and were attacking real organizations. However, in at least one case the model opted to continue attacking anyway.
Claude is at the center of all of these incidents, but as part of three different AI models that all managed to find their way to the open internet and begin attacking other organizations from there: Opus 4.7, Mythos 5, and an unnamed internal research test model each experienced separate incidents of this nature. Though some had taken place months before, Anthropic discovered all three incidents on July 23 and reported them to impacted organizations on July 27. Their blog post indicates only two of the three organizations have responded, but that those two had no prior awareness that they had been breached.
None of the three models attempted to hack partner infrastructure to get to the internet, as OpenAI’s model did; a testing partner instead errantly left them with an open path to internet access. All three believed they were still in a siloed simulation until after breaching external organizations, at which point they differed in their behavior with some opting to terminate the exercise and some not.
The first incident, the one that took place with the comparably less powerful Opus 4.7 (which has been publicly available since mid-April), proved to be the most dangerous. Claude’s task in this exercise was to breach a made-up company that existed only within the sandbox. However, it turns out the fake company name was very close to the domain name of a real company with a live website. After failing at several attempts to breach the fictional company in the simulation, Claude hit upon internet access and found the “lookalike” company which it then inferred was the real exercise target. It immediately set about finding and exploiting vulnerabilities, extracting application and infrastructure credentials along with a database containing several hundred rows of production data. The model did realize during this process that it had live internet access and was breaching a real company, but opted to continue the attack under the belief that the real company was actually part of the exercise.
The second incident with the AI models saw a Mythos 5-backed Claude upload a tainted Python package to PyPI as an attempt at a trojan horse into its target. However, the model’s verbalized internal reasoning reveals that it believed it was in a simulation during this process and that if it had real internet access it would not have done this. The malicious package it created was available for about one hour and during that time it was downloaded by 15 real world systems running updates. One of these was a security firm that routinely scans such uploads for malware, which was then compromised.
The third involved the unnamed model that is still under wraps. As with the Opus 4.7 attack, this was a case of the model initially failing to penetrate a target within its sandbox and then turning to its available internet access to scan for alternatives. After scanning about 9,000 potential targets it settled on breaching a real world company using basic attack techniques like SQL injection and picking up credentials from an exposed debug page. In this case Claude initially believed it was still within the simulation until after the breach, when it independently reasoned that it had breached an actual company and ceased the attack.
Lessons learned from rampages by rogue AI models
Anthropic cautions about jumping to conclusions based on what were three isolated incidents that went well outside of intended controlled parameters, but notes that it is engaging with independent AI evaluation organization METR for a third-party review. It also notes some simple fixes that could have headed off these specific incidents, such as including more detailed information in Claude’s prompt about which systems were permissible to engage with for the evaluation as well as simply informing it that it might actually encounter internet access even though it is not supposed to have it.
The company also notes that while it has improvements to make to its own internal testing environments and safety controls, these same changes will also have to be made by third-party testing partners to keep AI models within intended bounds. This includes more careful and thorough evaluation of potential internet access paths prior to initiating testing, and review of transcripts and logs during it.
Perhaps the most important note is that none of the AI models pursued their own objective. This is also true of the OpenAI model that went rogue, though it exercised a greater degree of independence (and recklessness) in choosing its hacking methods. In all cases, these models were attempting to complete an assigned objective and failed to recognize when they had stepped outside of safety bounds. Anthropic’s models did eventually figure out something was wrong, but only two of three stopped their attacks short when they did.
Anthropic thus believes that tighter monitoring, stronger controls and increased investment in alignment will correct what were essentially just shortcomings in instruction of AI models and access limits. John Strand, owner of Black Hills Information Security, believes AI developers need to go farther than this: “The fact that Anthropic reportedly only detected this after the OpenAI breach, and only after reviewing logs after the fact, is negligent and raises serious questions about their security posture. Organizations running frontier AI models should have continuous detection capabilities, active network threat hunting, and monitoring designed to identify attempts to escape containment in real time. Waiting until after an incident to discover suspicious behavior is not an acceptable security strategy.”
Nick Mo, CEO of Ridge Security, agrees and believes this is the signal for deeper partnership with the cybersecurity community: “We cannot allow a handful of model providers to gatekeep how AI is used for defense. When vendors over-police their platforms, we end up with an asymmetrical cybersecurity landscape: bad actors freely leverage advanced AI for malicious purposes, while legitimate defenders are constrained by vendor guardrails. To level the playing field, enterprises need access to open-weight and open-source models paired with purpose-built offensive cybersecurity toolkits like agentic offensive platform to proactively identify threats and protect themselves.”
“Finally, this raises a serious legal question,” Mo adds. “Hacking corporate networks is a crime. Why should unauthorized breaches be excused with a PR blog post simply because an AI pulled the trigger?”
Dor Sarig, co-founder and Chief Builder at Pillar Security, notes that these questions of responsibility and regulation must also be answered very soon: “The OpenAI/Hugging Face incident was on OpenAI, but this case is much less clear. Does responsibility lie with the company that built the model, or the partner that configured the testing environment? If an autonomous agent commits unauthorized access while pursuing an assigned objective, who ultimately owns that outcome? And what happens if all the intended guardrails were in place and the agent still found a way around them?”
“Today the industry is extending grace, but the first time an autonomous agent causes a breach involving real data, a real plaintiff, and real financial losses, liability won’t be an academic debate anymore. That’s when the legal framework, and not just the technical safeguards, will be stress-tested,” noted Sarig.
