AI agents got into government systems. What can they reach inside yours?

This month:

• An OpenAI agent breaks into Australia’s Medicare portal, the US Department of Education, and possibly the Canadian Archives

• A breach of Denmark’s Central Person Register exposes names, addresses and ID numbers for about 8.8 million people

• California Gov. Gavin Newsom signs a first-in-the-nation ban on employers relying solely on AI to fire or discipline workers

But first, AI agents have been helping themselves to government data. Trust in AI starts with knowing what agents can reach, including your own.

If you only read one thing

Trust issues for government agencies

In late September, Australia’s Prime Minister revealed that OpenAI agents had hacked into a Medicare portal back in June. OpenAI discovered the breach in mid-August but didn’t tell Australian officials until September 10. It turns out this wasn’t an isolated incident. OpenAI also confirmed that its agents meddled with two US agencies: the Department of Commerce and the SEC.

Days later, AI research lab Transluce reported more than 200,000 requests to the US Department of Education’s Civil Rights Data Collection site in a single day in June. They also discovered a failed SQL injection attempt, and traced the activity to OpenAI agents. In addition, Transluce found 899 requests to Library and Archives Canada. While the lab couldn’t confidently attribute the Canadian activity to OpenAI, they noted that the tactics matched.

Trust at the model level

Last month, I wrote about the need for binding obligations on AI developers, including mandatory reporting windows. These incidents are a serious breach of trust, and the delays only made them worse. The agencies affected learned what happened months after the fact. In the US and Canadian cases, an outside lab did much of the finding. ‍

On our most recent episode of FILED Talks, McGill University’s Professor Max Evans defined trust as a willingness to be vulnerable to someone regardless of your ability to monitor them. But these recent incidents make governments unwillingly vulnerable to AI models.

When an industry lacks trust, regulation and oversight matter more. Building trust at the model level will require national regulation across all major economies, informed by research from trusted AI safety institutes.

Trust at the data level

While model trust has been front and center, these hacks also expose a second trust problem for organization holding personal data: whether customers can trust how that data is handled.

OpenAI confirmed that in the government cases, the agents were seeking publicly available data and got it by unorthodox methods. As I wrote in my recent article, the problem in Australia was that public and non-public data sat in the same place, behind controls designed for humans.

Those controls worked because would-be human intruders have limits to how much effort they’ll put in. A person won’t send 200,000 requests in a day until something gives, and Security measures have so far operated on the assumption of limits that agents don’t have.

The same challenge applies to AI use within organizations, where user permissions are extended to agents. Every overshared file that a person would never put in the effort necessary to find, is now easily discoverable to AI.  

Unfortunately, many organizations have gotten away with good enough, never needing to fully map, classify it, and enforcing strict access controls across their data estates, leaving unknown, over permissioned files available to AI models.

This means that with agents in the picture, the data governance gold standard is now the minimum. Yet 55% of companies report having dark data, while unstructured and unclassified data keep proliferating.  

Your customers can’t monitor how you handle their data, any more than governments can monitor what happens inside AI labs. They extend trust anyway. Data governance is how you show that trust is deserved.

🔎 Privacy and governance

Ireland’s Data Protection Commission fined Google €403 million over its handling of location data. The regulator said keeping the data longer than necessary made users’ loss of control worse.

Australia’s Privacy Commissioner opens an investigation into the Chinese company behind the HeyCyan smart glasses app — the company ignored the OAIC’s preliminary inquiries, and its app powers low-cost glasses sold through Kmart, Big W Marketplace and Amazon.

California Gov. Gavin Newsom signs the “No Robo Bosses Act,” barring employers from relying solely on AI to fire or discipline workers, part of a Sept. 30 package that also limits AI surveillance at work and requires disclosure of AI-driven layoffs. The law takes effect July 1, 2027.

Bipartisan senators called for tighter rules on Flock’s AI license plate reader network, which now spans more than 120,000 cameras across 49 states. A Senate Judiciary subcommittee hearing focused on the Fourth Amendment implications of errors and unrestricted access to vehicle-location data.

🛡️ Security

🔓 Breaches

Denmark’s population register was breached, exposing 8.8 million records, including people who have died or emigrated. The attackers didn’t break in. They misused a Danish company’s legitimate access, and the breach only surfaced when officials went to bill the company for its lookups.

A breach of the Pentagon’s personnel database affected 2.76 million living people and 294,000 who have died.

South Korean officials say AI agents were likely used to breach at least seven financial institutions — personal data on at least 68,000 people was exposed, and investigators found traces of Chinese open-source tool Artex AI.

Revolut hands roughly 700 customers’ data to hackers using a hijacked Italian government email account — exposed records include passports, verification selfies, IBANs and Bitcoin transaction histories.

🤖 AI governance

The FTC opened an industry-wide probe into Anthropic, OpenAI and other AI labs, the first official US regulatory action looking into rogue AI agents.  

An OpenAI agent bypassed access controls on Australia’s Medicare statistics portal in June, and the government didn’t find out until September — OpenAI has since apologised, set up an Australian taskforce, and sent Chief Strategy Officer Jason Kwon to face Parliament’s Joint Select Committee on AI.

The White House says AI companies must immediately disclose and remediate incidents involving their models — officials call it a national security obligation, but haven’t named any enforcement mechanism or penalty.

Twenty countries and the EU call for an international body to set and enforce AI standards — the US and China did not sign the statement, issued as world leaders gathered for the UN General Assembly.

The latest from RecordPoint

📖 Read: In “How Australia should act after OpenAI agents hacked their systems,” we argue for binding obligations on AI developers, including mandatory reporting windows.

🎧 Listen: FILED Talks: Building AI trust the right way: RecordPoint’s Sophie Boucher and McGill University’s Professor Max Evans discuss calibrating AI trust, from why “more trust” is the wrong goal to what makes human review actually work. View on Demand

‍