9.10 Agent security

What this is and why it exists

Agent security starts from one assumption and everything follows: everything the model reads is a potential instruction channel. A retrieved document, a tool result, a web page, a file, a code comment — any of it can carry text intended to redirect the agent, and no wording of a system prompt reliably prevents that. So the defences that matter are not in the prompt. They are in what the agent is architecturally unable to do.

The vocabulary

• Injection — instructions in content that redirect the system.
• Indirect injection — the same, arriving through content the agent fetched rather than from the user.
• Least privilege — granting only the access a task requires.
• Scoped credential — one limited in what it can reach and how long it lives.
• Allowlist — an explicit list of what is permitted, with everything else refused.
• Sandbox — an isolated environment for running code or acting.
• Egress control — restricting what the system may connect out to.
• Audit log — a durable record of what the agent did.

The mental model

The central problem, stated precisely. Everything reaching the model arrives as one sequence of tokens with no privileged channel. Your instructions, the user's request, and the contents of every document and tool result share that channel, and the model has no dependable way to tell which of them you authorised. An agent multiplies this because it reads far more — every search result, every fetched page, every file — and because it can act, so a successful redirection does something rather than merely saying something.

Indirect injection is the shape that matters. The attacker never contacts you. They put instructions in a page your agent browses, a document somebody uploads, a ticket, a review, a code comment, an email — and your agent reads it and follows it. The person using the system did nothing wrong and cannot see what happened. Any system that reads external content must treat that content as untrusted, and that is a design constraint, not a prompting problem.

Prompt-level defences reduce the rate and do not solve it. Instructions to ignore embedded commands, delimiters around untrusted content, a separate model asked to screen inputs — each raises the bar and none creates a boundary, because the model is following text either way. Use them; do not rely on them. Real safety lives in what the agent cannot do.

Least privilege is the first structural control, and the question is what a hijacked agent could accomplish.

Grant the minimum. If the task reads, give read access only — a hijacked reader leaks at worst what it could already see, which is bounded and knowable. Scope credentials to the task, not to the system: this customer's records, this repository, this bucket, this table, rather than an account key that reaches everything. Make them short-lived, minutes rather than months, so a leaked credential expires before it is useful. Never give an agent a person's own session, because it then inherits everything that person can do, including things nobody intended and nobody logged separately.

Allowlists rather than blocklists, everywhere the set is knowable: which hosts may be reached, which tools may be called for this task, which file paths may be touched, which recipients may be written to. A blocklist enumerates the attacks you thought of; an allowlist enumerates the operations you intended, and refuses the rest by default.

Sandboxing and egress control are the second, and they are what contain a compromise.

Any agent that runs generated code runs it in isolation — a container or virtual machine, disposable, with a filesystem containing only what the task needs, resource limits so a loop cannot exhaust the machine, and no ambient credentials sitting in the environment where a script can read them. Treat generated code exactly as you would code from a stranger, because functionally that is what it is.

Egress control is the one people skip and the one that decides how bad an incident gets. A compromised agent with unrestricted outbound access can send anything it has reached to anywhere; the same agent restricted to an allowlist of hosts can do very little with what it stole. Outbound network control is what stops a compromise becoming a data loss, and it applies to the whole system rather than only to executed code — including which addresses the agent may send to and which endpoints its tools may call.

Audit logs and irreversible-action gates are the third, and together they make an incident investigable and survivable.

The audit log records what the agent did, not what it said: each tool call, its arguments, its result, the identity it acted as, when, and which run it belonged to. Write it somewhere the agent cannot modify, keep it long enough to investigate a problem discovered weeks later, and make it queryable by user and by tool, so the question "what did this agent touch on Tuesday" has an answer. This is separate from the tracing in the observability topic — that exists to improve the agent, this exists to answer for it.

The irreversible-action gate is the tools topic's control, restated as a security boundary: anything that cannot be undone, or that reaches outside your system, requires a person to approve it, with the real arguments shown, enforced in code that no instruction can reach. This is the control that turns a hijacked agent from an incident into an attempt, because an injected instruction can persuade the model and cannot persuade the gate.

A practical way to hold all of it: assume the agent is hostile and ask what it could do. Take each tool, each credential, each network path, and ask what an attacker who fully controlled the model's decisions could accomplish with it. Where the answer is unacceptable, the capability is wrong — not the prompt, the capability. That exercise takes an afternoon, it can be done before any code exists, and it produces a list of architectural decisions rather than a list of hopes.

And design for it from the start, because these controls shape the architecture: which tools exist, what retrieval may return, where credentials live, what the network permits, what is logged, what needs approval. Retrofitting them onto a working agent means rebuilding it. Design for the agent to fail safely, then make it work.

What you should now be able to explain or do

State the central assumption and say why an agent makes it worse than a chat interface. Describe indirect injection and why it needs no contact with you. Say what prompt-level defences buy and why they are not a boundary. Apply least privilege with scoped, short-lived credentials and no borrowed sessions. Prefer allowlists and say why. Sandbox generated code and control egress, and say what egress control specifically prevents. Keep an audit log distinct from tracing, and gate irreversible actions in code. Run the hostile-agent exercise before writing code.

Check yourself