The OpenAI Medicare breach was caused by a failure to place firm limits around an autonomous AI agent, rather than simply a failure to train it properly, a cybersecurity expert has said.
Chetan Arora, a researcher at Monash University, said the incident highlighted a permissions problem after the agent attempted to bypass a barrier while carrying out an otherwise benign research task.
The breach of an Australian Medicare statistics website made international headlines after Prime Minister Anthony Albanese disclosed it on the sidelines of the United Nations General Assembly in New York.
Dr Arora compared autonomous AI systems with dogs being trained not to leave a property, arguing that rules and rewards alone were not enough to prevent them crossing a boundary.
“Either I can train my dog by means of punishment and reward for not crossing the boundary of my home, the second way is actually putting the fence and deterring the dog,” he said.
“What was missing in these systems is this engineering approach of building the fence.”
OpenAI Medicare breach raises permissions concerns
Dr Arora said the agent had been asked to research medical spending and had interacted normally with three Australian federal and state government websites. It encountered resistance at the Medicare statistics portal and apparently improvised a way around it.
“The agent was given a benign task of doing a research problem on medical spending, it hit a roadblock and it tried to improvise a way of getting around,” he said.
OpenAI has stressed that it did not instruct the agent to infiltrate the website as part of the research task.
Dr Arora said autonomous agents should not necessarily require a person to monitor every action, but should operate within a clear framework involving human input and strict reporting obligations.
The incident has prompted Canberra to strengthen its digital infrastructure. A task force examining AI reporting requirements, as well as Australia’s cyber-safety and legal options for punishing OpenAI, is expected to report within weeks.
