Insecure output handling downstream

Read these first

• Data exfiltration channels

What you will know

• Show model output causing XSS in a naive renderer

  A browser executes attacker-controlled markup when text is inserted into an HTML context without the correct escaping or sanitization policy.

• Trace model output from generation to a downstream sink and mark each trust-boundary crossing

  The dangerous assumption is usually introduced by a serializer, template, query builder, or tool wrapper rather than by the model call itself.

• Patch HTML, SQL, shell, path, and redirect sinks with context-appropriate controls, then verify with malicious fixtures

  Escaping is context-specific: parameterize queries, avoid shell interpretation, constrain canonical paths, and allowlist redirect destinations instead of applying one generic sanitizer.

• Reproduce a path traversal using a model-chosen filename, then block it with canonicalization and an allowed-root check

  Checking the raw string misses encoded separators, dot segments, symlinks, and alternate path representations.

• Run the downstream API with adversarial model outputs and verify schema validation, authorization, and least privilege reject unsafe requests

  A valid-looking tool call can still target another tenant, request excessive scope, or exploit a parser unless policy is enforced outside the model.

Chapters

1. 1

   Insecure output handling

   Insecure output handling is what you have when application code hands a model's words to something that acts on them. Teams that scrub every field at the edge conclude they are covered, and the model's answer is a new string created after that scrubbing ran, arriving at sinks the original input never touched. Rendering it with innerHTML in a support console means an injected ticket can execute script inside a staff session, with an agent's authority over every account the console reaches.

   Read chapter
2. 2

   Tracing output to its sinks

   Tracing a model response from generation to every place it takes effect, and marking each trust-boundary crossing on the way. the boundary crossing is rarely at the API client. It is a template that inserts the string into markup, a query builder that concatenates a fragment, a tool wrapper that formats an argument into a command — call sites that read as formatting and behave as interpretation.

   Read chapter
3. 3

   Escaping by destination grammar

   One sanitiser at the boundary cannot serve an HTML view, a SQL query, a shell command, a filename and a redirect, because each destination parses differently: contextual encoding, bound parameters, an argument array, canonicalisation, an allowlist. The case worth dwelling on is a model-composed WHERE clause, which is structure rather than values and therefore cannot be parameterised at all.

   Read chapter
4. 4

   Path traversal and canonicalisation

   Rejecting names that contain '..' or start with a slash validates a string while the operating system decides the outcome, and percent-encoded separators, alternate representations and a symlink sitting in the workspace all resolve outside the root afterwards. The safety property is established by resolving the path first and then checking that the result is inside an allowed root.

   Read chapter
5. 5

   Schema validation and authorisation

   Firing adversarial model outputs at a downstream API, most malformed payloads bounce off schema validation, which then feels like the control. It is not: the request an injection actually produces is well-formed, correctly typed and aimed at somebody else's resource, or asks for a scope wider than the task needs.

   Read chapter
6. 6

   Ranking sinks by reachability

   Inheriting a service where one model response builds a SQL query, picks a redirect target, names an S3 key and fills an HTML template, severity ordering puts SQL first and misses the deciding question. A SQL sink fed from a fixed enum is a smaller problem than an HTML sink fed with text from a customer email.

   Read chapter