The nation of Australia has apparently only just learned of a June hack on its national health portal, just a little ahead of the rest of us. A rogue OpenAI agent was once again responsible, finding and taking advantage of what appeared to be a serious weakness in the government health agency’s systems. The incident is believed to be the first breach of a national government system conducted independently by an AI agent.
OpenAI allegedly took months to discover breach, then further weeks to notify Australian government
Australian Prime Minister Anthony Albanese broke the news to the world while giving comments to the press at the UN General Assembly in New York. OpenAI allegedly knew about the June breach sometime in August, but waited until into September to send an email to a general government agency inbox about it.
Albanese said that the timeline was “unacceptable” and that OpenAI took “way too long” to notify his government of the incident. The OpenAI agent accessed the national Medicare statistics portal, where it could have come across private beneficiary information (though not types of data classified as “sensitive”). Albanese said that he personally called OpenAI head Sam Altman to express his “extreme concern.” The Australian Signals Directorate has launched an investigation into the incident, which is ongoing.
In most other cases of this nature that have surfaced, an AI model attempting to solve a security puzzle has gone off the rails in some way. In this case, the OpenAI agent was seemingly tasked with a more modest research assignment involving health care statistics. Frustrated with blocks placed on its access to the Medicare portal, the agent began devising less-than-ethical ways to get around them.
And as with most other cases of this nature in recent months, OpenAI’s own logging and monitoring did not catch the incursion initially. It was found about two months later when a company deep dive into prior AI model activity (likely spurred by the Hugging Face attack) turned it up. OpenAI notified the Australian government on September 10, but by way of an email sent to a general government mailbox at Services Australia (the agency overseeing welfare and health insurance payments) meant for the general public; this added five additional days to the timeline of government ministers actually being advised of the breach.
Raymond Schippers, Lead Technologies, Check Point Software Technologies ANZ, notes that this news comes at a particularly bad time for already beleaguered Australian IT teams: “This lands on top of a threat environment that was already intensifying for Australian organisations. ASD’s most recent threat report found a cybercrime report is made roughly every six minutes, and the average cost per report for businesses rose 50 per cent to more than $80,000. Ransomware and data breaches were both on the rise. Behind those numbers are businesses that couldn’t trade, serve customers, or pay staff while they recovered.”
AI developers insist continual unearthing of old breaches consists only of “minor” incidents
The news continues a steady drip of newly-uncovered incidents from months past that did not necessarily involve some sort of catastrophic breach or leak of sensitive information, but did involve hacking and clear attempts to hack by AI agents based on autonomous decisions that were not being properly monitored by a human handler. The general industry position on this level of breach, which OpenAI has carried on here, is that it is too “minor” to report immediately to the targets themselves let alone the public and government officials.
Legislation has been slow to catch up to the breakneck pace of AI development, but this particular incident may have violated an existing regulation. Australia’s data privacy law establishes a duty to notify with an attached time limit once a breach of this nature is detected, along with a requirement that human safety officers be present at firms at this size that continually ensure safety standards are being implemented and followed. There has yet to be any public comment on the issue from Australia’s data privacy regulators, however. Albanese has indicated that Services Australia will be facing scrutiny over the five day delay in forwarding the email to appropriate authorities.
The comments from Albanese were shortly followed by a broad disclosure from OpenAI that a number of its agents “may have meddled” with the systems of multiple “governments, universities, public agencies, and other institutions.” OpenAI says that it has privately alerted “dozens” of impacted parties. These attacks appear to have unfolded in a similar manner to the one brought up by Albanese, with the OpenAI agents initially set to what should have been harmless research tasks. However, the company continues to characterize “most cases” it has found as “low severity.”
This is accompanied by a recent paper from AI research lab Transluce, which has conducted independent research indicating AI agents are responsible for at least three more previously undisclosed attempts at breaches that date back as far as March of this year. These agents were set to similarly “mundane” data retrieval tasks, but went the extra mile in attempting to aggressively identify vulnerabilities and hack their way into target systems (which included the University of New Mexico digital library and open source government data platform Data USA). None were successful in breaching their targets, but the researchers indicate that OpenAI agents were involved with all three of these attacks.
Paul Bischoff, Consumer Privacy Advocate at Comparitech, notes that this not only invites regulatory scrutiny but also serves as a warning to organizations to update their understanding of current threats from AI agents: “The attack would have been prevented if the data was properly secured in the first place, and that’s the lesson that should be learned here. AI is good at finding vulnerabilities but it’s not creating new ones. We should be using AI to run preemptive scans and plug the holes before attackers have a chance to find and exploit them with their own AI.”
And Jacob Krell, Senior Director: Secure AI Solutions & Cybersecurity, Suzu Labs, builds on the idea that existing regulation could be applied more aggressively and effectively to rein in developers that are not meeting their safety obligations: “Australia already has relevant criminal offences. Section 478.1 of the Criminal Code Act 1995 covers unauthorized access to or modification of restricted data. Section 477.2 covers unauthorized modification that impairs, or risks impairing, computer data or systems.”
“Those laws do not need to understand neural networks,” Krell adds. “They need investigators willing to apply them when an AI system crosses a legal boundary. The model may have found the workaround, but the lab built and deployed it, gave it the objective, and controlled the operation. The effective deterrent is criminal enforcement. Preserve the prompts, tool calls, and access logs, identify the responsible people and companies, and bring charges where the evidence supports them. More regulation gives frontier labs another framework to negotiate. Enforcement gives them a reason to stop.”
