Accompanying the release of a review of major national cyber incidents involving AI-driven attacks, the Australian government will be implementing new data security and privacy rules including mandatory reporting requirements for developers and resilience measures for the public sector. Criminal penalties for developers are also on the table for consideration, but not confirmed yet. The move stems from recommendations made in a “Rapid Review” conducted by assorted government agencies, with creation of a new central AI regulator as one of the new elements.
Health system cyber incident triggered regulatory changes
The central incident that prompted the review (and subsequent new requirements) was the mid-June breach of the country’s Medicare Statistics Reporting Service portal, an incident thought to be the first in which an autonomous AI successfully breached a national government website. As with many of the incidents of this type that took place throughout 2026, the breach was a case of overzealous OpenAI agents attempting to complete a research task and pushing their way into areas they were not authorized to access.
Though no patient data was accessed (as the agents were after statistics), the incident raised serious alarms not just for the unprecedented breach but for the fact that it took OpenAI about two months to discover the breach and then several more weeks to notify the Australian government. That notification was also sent to a general public-facing inbox, leading to a further delay of several days before it was routed to the right hands. Both the Australian government and the general public thus did not become aware of the incident until September.
This triggered a forensic investigation headed up by the Australian Signals Directorate, which included the Rapid Review which looped in currently existing AI advisory bodies along with more general cybersecurity regulatory agencies. One of the central recommendations from this report is the creation of a new central AI regulator, which would work in tandem with the existing Australian AI Safety Institute to enforce AI rules and dole out penalties for applicable cyber incidents.
New AI regulator would enforce new rules on reporting, possible criminal penalties
The Rapid Review contains 18 recommendations in total, all of which the government has agreed to implement and has said it is now working on.
Among the items with the most firm wording: the establishment of “mandatory” and “time-bound” reporting requirements for cyber incidents involving misalignment or an agentic attack, issuance of Protective Security Policy Framework (PSPF) policy guidance by the Protective Security Board to clarify agency obligations and escalation expectations, and review by commonwealth agencies to ensure they align with PSPF requirements. Commonwealth agencies are also directed to improve the AI literacy of staff, implement the ASD’s Essentials and shut down or segregate systems that do not need to be connected to the internet.
Specifics about many of these recommendations are still thin, as the report uses the word “consider” quite liberally. However, one item that now seems to be up for discussion is the possibility of criminal penalties in the case of “serious misconduct” in connection to cyber incidents. This is territory that other countries have thus far been hesitant to venture into, though there is now some precedent from Italy with a legislative decree established in late September that could land executives up to five years in prison if required safety measures are not kept in place or reasonable steps are not taken to prevent model harm.
Any new measures established and enforced by the new AI regulator would apply to the largest AI developers, those responsible for the frontier models capable of independent agentic breaches. Thus far that list includes confirmed breaches from OpenAI, Anthropic, Google, Alibaba, Tencent and Meta in 2026. At minimum these entities should expect, at least in the near term, to implement and maintain security controls throughout the entirety of the development process and to provide government notification of AI cyber incidents within a limited window. They can additionally expect to allow independent outside evaluation of their models and to have transparent safety frameworks available to the public.
At the moment there is not much specific information about the forthcoming AI regulator, however. Things will likely be firmed up when the Australian AI Standards framework goes before Parliament, which is expected to happen sometime in early 2027. No name for the new agency has been proposed as of yet, and there are no details on what the financial penalties for cyber incidents might be. It is also not clear how enforcement might be applied to AI firms that operate in Australia but keep their headquarters overseas (which is essentially all of those that would be big and threatening enough to be regulated, though most keep a local office in the country).
Darin Fredde, Senior Director and Offensive Security Evangelist at Ridge Security, addresses how responsibility might be determined if and when criminal penalties for cyber incidents are established by the AI regulator: “Giving an AI agent a legitimate research task does not authorize it to bypass someone else’s access controls. OpenAI describes an experimental model crossing that boundary. Its report that individual patient records were not accessed matters, but it does not erase the authorization failure. Australia’s proposed reporting requirements and independent evaluations are sensible steps. I would judge their success by the evidence: can developers show what an agent was authorized to do, what it did, and whether controls stopped unauthorized actions? Permissions need to be enforced outside the model and tested when an agent encounters resistance or changes its approach.”
Ryan McCurdy, Vice President of Marketing at Liquibase, adds: “Australia’s response puts more attention on accountability and incident reporting, which are important. But reporting an incident after it happens doesn’t prevent the damage. Organizations need technical controls that enforce permissions, validate high-risk actions, and prevent unauthorized changes before they’re executed.”
