A compromised human acts in minutes and hesitates at a strange instruction. An agent acts in milliseconds, in a loop, and does not hesitate. That speed difference changes which control matters: prevention still counts, but containment counts more.
Two failure modes, one mitigation
The first mode is malicious: a poisoned context tells the agent to iterate over records and send everything outward. The second is mundane: an orchestration bug puts the agent in a cycle calling the same tool thousands of times. The second is far more common and produces the same early symptoms — which is good news, because the control against error also works against attack.
Limits worth having
- Call budget per task. A task that needed 12 calls and has made 400 is wrong, whatever the reason. Cut it and ask for human intervention.
- Per-tool limits, not global ones. Fifty reads may be normal; fifty external writes almost never are.
- Data-volume limits, not just counts. A single call returning 2 GB is more alarming than a thousand small ones.
- Circuit breaker on repeated errors. The same failure three times running means the agent is not learning from the result; continuing only widens the damage.
- Per-identity kill switch. There must be a single command that revokes one agent's credential without taking down the whole platform — and it is only possible if each agent has its own identity.
The signal nobody uses: cost
Inference and API spend is security telemetry disguised as finance. A hijacked or looping agent shows up on the invoice before it shows up in the SIEM, because FinOps teams monitor cost anomalies by habit. It is worth wiring that alert through to security — free detection that is already running.
The first person to notice an agent out of control is usually the one paying the bill, not the one watching the logs.
Where to implement it
On the client side or at the gateway, never only at the tool server. The server sees isolated calls; the layer that understands task, budget and sequence is orchestration. Putting the limit only at the server is like asking each microservice to work out on its own that it is under distributed attack.
