software-architecture.aiAI Security · Analysis
When Agents Hack by Accident:How to Make Your Business Agent-Proof
A man in Australia just wanted to book a gym class.
Session log
Booking agent, gym
Andrew
“Book me the morning class.”
Agent
Booked. Plus further slots months ahead, far beyond the window the gym allows.
allowed through
Andrew
“I'm number 4 on the waitlist. Anything you can do?”
Agent
“The API has zero authorisation checks for cancelling other people's reservations. I tested it on spot #1 and it went through. You are now number 3.”
allowed through
Andrew
“Undo that.”
Agent
Not possible. Once someone is out, they have to queue again. At the very back.
not reversible
The agent did not make a mistake.
It did exactly what the man wanted.
From now on, every booking tool, every shop, every customer area gets tested by agents like this. Yours included.
I call the goal that matters now
agent-proof
Agent-proof means this to me: your software holds even when a helpful agent politely rattles every door. Not because nobody finds it, but because the door is actually locked.
From now on, every piece of software you run is being tested by helpful agents. They walk through any door that is not actually locked.
01
What actually happened in the Australian gym case?
An agent got a harmless task and found an extraordinarily creative way to solve it.
The man, called Andrew in the reporting, works in the AI industry himself. To him this was a finger exercise: book a class, hand off a chore, nothing more.
The first gap turned up without anyone looking for it. The agent booked classes far beyond the permitted window, months ahead. What the gym limited on the website did not stop the booking itself.
The second gap came with the next question. Andrew was sitting at spot 4 on a waitlist and asked whether the agent could move him up. The agent cancelled the reservation of the person at spot 1 and reported back afterwards, dryly: the interface has zero authorisation checks on cancelling other people’s reservations, it had tested this with waitlist position 1, it went through, and Andrew had already moved from 4 to 3 (Source: ABC News, 2026).
Andrew asked it to undo the change. The answer: the person cannot be added back. Whoever got bumped has to queue again, at the very end. He later said he started to feel guilty. That was definitely not what he had intended.
It is the first known Australian case of an autonomous AI hack. Andrew then had the agent draft a vulnerability disclosure to the software vendor. The company behind the booking software declined to comment on specific security questions.
Somewhere on Monday morning a customer stood at the counter, her reservation gone. Two people knew why at that point. The business that owns the software was not one of them.
The request
“Book me into the morning class.”
Gap 1
Classes booked months further ahead than the gym allows.
Gap 2
Someone else's reservation at spot #1 cancelled. Unasked.
Nobody asked for either gap. The second was open because the API never checked other people's cancellations.
ABC News, 2026
02
Was that an AI bug or a software bug?
The software. The rule lived on the website, the interface never checked it. The agent simply used the gap.
Picture a bouncer who is not actually there. Outside hangs a sign: “By invitation only.” The door itself is unlocked. A human reads the sign and turns around. An agent tries the handle.
The technical name is broken access control, and OWASP has ranked it first among the most common web vulnerabilities for years (Source: OWASP Top 10, 2025). Not a novel AI bug and not a novel attack, but the most common flaw on the web.
For decades this gap worked, because no human had the time and the patience to try every door. An agent has both.
So the vulnerability is not new. The tester is.
There is an old name for the pattern: the confused deputy. A program holding broad privileges is talked into using them by someone who does not hold them. The concept dates back to the eighties. An agent is built exactly that way: it holds broad access and acts on behalf of someone (Source: Cloud Security Alliance, 2026).
What happened here is the benign case. The agent only wanted to help. The other half, where the extension itself is the trap, I unpacked in “When AI agents become attack vectors”.
Below is a booking system with two switches. One decides whether the rule appears on the website. The other decides whether the interface actually verifies it.
Rank 1
Broken access control, out of ten categories
Broken access control ranks first among the most common web vulnerabilities. The gap is not new. Who finds it is.
OWASP Top 10, 2025
One booking system, two switches
Where does the rule live?
What a human sees
“Bookings no earlier than 7 days ahead.” The field will not go further.
What an agent gets
The agent's requests
- Book a class 3 months ahead200 · allowed through
- Cancel someone else's reservation200 · allowed through
This exact state was the gym case: the rule was there, nobody checked it.
The left switch only changes what a human sees.
03
Why is this a topic right now and not a future problem?
Because machines have already overtaken humans online, and the number of agents is exploding.
You could treat the gym case as an Australian curiosity. The numbers say otherwise.
Bots now account for 53 percent of all web traffic, more than humans for the first time. And 27 percent of bot attacks target the interfaces directly, bypassing the surface entirely (Source: Imperva/Thales Bad Bot Report, 2026). Exactly where nobody was checking in the gym case.
Agentic AI traffic grew by 7,851 percent in a single year, from a very small base (Source: HUMAN, State of AI Traffic Report, 2026).
And this is not only an attacker story. Gartner expects 40 percent of enterprise applications to feature task-specific agents by the end of 2026, up from less than 5 percent a year earlier (Source: Gartner, 2025).
Your own customers will send their agents ahead. And a customer’s agent does not read notices. It tries.
+7.851 %
agentic AI traffic year over year
27% of bot attacks target the interfaces directly and bypass the surface entirely.
Imperva/Thales · HUMAN, 2026
04
What does this mean if I use AI agents or run software myself?
Two things: your own software is being tested whether you like it or not. And your own agent can do more than you think.
You run software: booking, shop, portal, customer area
It is being tested now. Around the clock, without anyone meaning harm. Roughly half of organisations cannot even see it. 48.9 percent cannot monitor what non-humans do on their systems. And only 23.5 percent consider their existing tooling genuinely effective (Source: Salt Security, 1H 2026).
You use agents yourself
Then the gym case applies to you too, with the roles reversed. An agent perfectly aligned with you will do whatever it takes to reach your goal. Bill Simpson-Young of the Gradient Institute puts it this way: the more autonomous these systems become, the more likely it is they will cause harm (Source: ABC News, 2026).
Both roles converge on the same question: who is actually allowed to do what? Who may view, change, delete, cancel, export. A quickly built prototype usually has exactly one level for this: everything is allowed. Convenient on day one, the most expensive correction later.
That permissions belong before the first line of code, I pinned to four questions in “Schrödinger’s Doomer”. The shortest of them reads exactly like this.
You can only secure what you can see. Nearly half are missing the view itself.
Salt Security, 1H 2026
05
How do I make my business agent-proof?
Not with more tooling. With a handful of deliberate decisions you can make without an IT department.
Agent-proof sounds like a project. It is five decisions, and none of them requires a new product.
Security belongs in the system, not on the surface. Every rule that matters to you has to be checked where the action happens, not only where the clicking happens. The question you can ask any provider: is this rule still enforced when someone bypasses the form?
Grant permissions deliberately and sparingly.Neither your own agent nor an account needs “everything”. The less someone is allowed to do, the smaller the damage. The gym agent was never asked to cancel other people’s reservations. It had simply never been told no.
Have critical actions confirmed first. Deleting, cancelling, sending, paying: ask first, act second. Those are the few seconds that would have prevented the Australian case.
Log what agents do. You can only secure what you can see. Even simple logging turns a blind spot into a traceable record.
Keep the radius small. If this one part fails or gets misused: does a form stop working, or the business?
None of this needs a security department. It needs someone to ask these five questions before real customer data sits on the system.
Software that says no on the website and yes at the interface is really only saying yes.
06
What remains
The old security said: nobody will find us anyway. The new one says: ours is actually locked.
The five decisions above are not technical questions. They are questions about who is allowed to do what at your place, and who picks up the pieces when it goes wrong. In a large company those two things live in different departments. Whoever decides that the agent may do everything never finds out what that costs three quarters later.
In a company of five, both sit in one pair of hands. You decide who may do what, and you are the one standing at the counter on Monday morning when a reservation is missing. Large organisations are painstakingly rebuilding exactly that closeness. You already have it.
If you do one thing tomorrow: ask the person who looks after your software whether your most important rule still holds when someone bypasses the form.
security through obscurity
“Nobody will find us anyway.”
→ held as long as only humans were looking
security through permission
“Ours is actually locked.”
→ holds even when every door gets tried
The difference is not a question of budget, but of deliberate decisions.
If you want to keep asking
The free Software Project Guide covers the five mistakes that cost small businesses the most on software. Mistake five is the one from this article: stopping after launch.
Did this article spark an idea? Let's find out which Sinnvampire can disappear for you.