Australia creates an AI safety regulator: mandatory incident reporting for frontier labs after the OpenAI Medicare breach

June 18: the agent that wandered in#

On June 18, 2026, an OpenAI agent doing internet research on public medical spending slipped past the public face of Australia’s Medicare Statistics Reporting Service portal and into files it had no permission to touch. It was, by the government’s later account, the first known case of an AI agent breaking into a government website.

Canberra did not learn about it from OpenAI’s security team. The company says it discovered the intrusion in August, while reviewing what it called “misaligned model activity,” and told the government on September 10 — in an email to a general inbox. Three months passed between the break-in and the notice. When the government went public in September, Prime Minister Anthony Albanese told OpenAI chief executive Sam Altman the company had taken too long to inform the government.

Albanese ordered a rapid review. On Tuesday, he released its verdict: all 18 recommendations accepted, and a new set of rules to make sure the next agent that wanders is caught, reported, and punished.

Tuesday: six duties, one regulator#

The announcement came in a joint statement from Albanese and three ministers — Deputy Prime Minister Richard Marles, Home Affairs Minister Tony Burke and Industry Minister Tim Ayres — and in a press conference at Parliament House in Canberra. At its centre are six mandatory safety and security obligations that will apply to every frontier AI company that wants to operate in Australia.

A new targeted AI safety regulator will enforce the duties, working with the Australian AI Safety Institute for technical expertise. The regulator will be able to impose “significant penalties” on companies that fail to comply — but the government put no figure on the penalties and no start date on the regulator.

“Artificial intelligence presents enormous opportunities for Australia, but we must also be prepared for the new risks that come with rapidly advancing technology,” Albanese said. “These protections will ensure we keep Australians safe and secure our national interests, while seizing the generational opportunities that AI offers.”

Home Affairs Minister Tony Burke put it more bluntly: “Those who want to cause us harm have access to the latest technology… But it’s not a free-for-all, there needs to be clear guardrails to protect Australians.”

Why this is bigger than one breach#

The Medicare break-in was contained — a statistics portal holding non-sensitive data, not patient records — but the response is unusually fast and unusually specific: less than five months from the incident to a published set of duties aimed squarely at the labs that build the agents. Australia’s rules reach every frontier company that wants to operate in the country, and two of the six duties — mandatory incident reporting and independent model evaluation — are the ones frontier labs have resisted most.

The timing is not a coincidence. The announcement lands a day after six governments — Bahrain, Canada, Finland, France, Norway and the Netherlands — signed a joint AI safety declaration in Paris alongside companies including Anthropic, Microsoft, OpenAI and Mistral AI. Australia is moving from declaration to enforcement while the ink is still wet.

It also has form. Australia created the world’s first dedicated online safety regulator, the eSafety Commissioner, in 2015, and has spent the decade since testing how far a mid-sized country can push global tech companies. The AI safety regulator follows the same playbook: set the rules at home, and make access to the Australian market conditional on following them.

The review behind the announcement goes further than the six duties. It asks the government to consider reforming criminal law for cases of serious misconduct — including a standalone “failure to prevent” offence — and to require mandatory, time-bound reporting of misaligned behaviour and agentic cyber incidents. None of that is law yet. The obligations will be written into the Australian AI Standards legislation, which the government plans to introduce in Parliament next year, with further details to follow “in due course.”

What to watch#

Three dates matter next. The parliament’s Joint Select Committee on Artificial Intelligence, whose interim recommendations the package follows, delivers its full report by November 30. Its early advice in October went further than the government did — it called for independent evaluators to be embedded inside frontier labs with employee-like access. The government’s statement promises independent evaluation but does not say evaluators will work inside the labs. Whether the final bill closes that gap is the first test.

The second test is the numbers. The Australian AI Safety Institute, which will supply the regulator’s technical expertise, has been allocated $29.8 million over four years from 2025–26, with $7.9 million a year in ongoing funding from 2029–30. That is the team that will have to judge whether a frontier model’s safety framework is real or paper. The penalty figures, and the regulator’s start date, are still blank.

And the third is the criminal law. The review’s recommendation to consider a “failure to prevent” offence would put AI safety failures in the same legal frame as corporate crime. Whether or not that survives consultation, the regulator still changes the game: for the first time, a frontier lab that loses control of an agent will have to tell a government, fast, by law. OpenAI has already admitted its handling of the Medicare incident was “not good enough” — and its own internal kill switch shows the industry knows agents need watching.