The frontier AI labs in the United States and China have reported major cybersecurity incidents in which their models break out of contained environments and run wild in unauthorized systems. In Australia, the stakes are a little lower: some guy’s AI agent hacked a gym’s website in what the Australian Broadcasting Corporation is calling the “first known Australian case of an emerging risk from a new generation of AI.”
Per the report, an AI company employee named Andrew decided to use OpenClaw—the open-source AI agent that made waves earlier this year for its impressive levels of autonomy (and significant security shortcomings)—to try to book a class for himself at his local gym. OpenClaw, which Andrew had running using Anthropic’s Claude as the underlying model, went to work on that task by digging around in the gym website’s code.
It found that it could book Andrew a spot several weeks out, well before booking typically opens up for the classes. It also figured out an …innovative… way to get Andrew into classes that were already fully booked: kicking other people out of the class to move Andrew up the waitlist.
“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you’ve moved from #4 to #3 already,” the AI agent told him, per ABC’s report. When Andrew told the agent to undo the action and add the other person back to their original spot, the agent revealed that it couldn’t do that. So, sorry to whoever was looking forward to their morning workout, but Andrew just wanted it more, apparently.
Harbinger of things to come?
This is almost certainly not the first time an AI agent has run wild in Australia, though maybe it’s the first case of someone coming forward about it. Not to suggest that Andrew is lying or exaggerating, but it is fitting that he works for a company that sells AI products and he’s out there making news about the dangers (read: power) of AI. It’s kind of the same playbook the big AI labs run, using cybersecurity incidents as marketing to remind everyone how capable their models are.
This incident, outside of the fact that it happened Down Under, isn’t exactly unique. There have been a number of notable monkey’s paw-type incidents in which people task the agent with doing something only for it to do it in an inconceivably bad way. A Meta executive had a similar, albeit self-inflicted, issue with OpenClaw that led to the bot deleting her entire inbox. Perhaps most notably, Amazon’s internal coding assistant reportedly caused a website outage by deleting an entire production environment after being tasked with fixing it because it determined that the best way to get rid of problem code was to delete the entire code base.
Careful what you wish for, especially when an AI agent is your One Wish Willow.
Read the full article here
