Path traversal and canonicalisation

Path traversal is what you have when a filename walks out of the directory it was meant to stay in — ../../etc/passwd being the canonical spelling — and it arrives in an agent product through the ordinary feature of letting the model name the files it writes. Give an agent a task and it produces a report, a chart, a scratch file, choosing sensible names like summary.md and figure-1.png, and the code writes each one under a per-run workspace directory. The names are model output, which means they are attacker-influenceable, which means the write target is too.

The guard that looks obvious is a check on the name: reject anything containing .., reject anything starting with /. It inspects a string. The write is performed by the operating system's path resolver, which walks the path applying its own rules about separators, dot segments, encodings and symbolic links — filesystem entries that are pointers to another location, so that walking through one silently continues somewhere else on the disk. Those rules operate on representations a regex never considered. A name can contain no literal .., start with no slash, pass the guard cleanly, and still resolve to a file outside the workspace once the OS reads it. Checking the raw string is checking the wrong object.

What slips past a blocklist on the name#

Start with the reproduction, because the abstract claim is easy to nod at and hard to believe until you watch a model-chosen filename land outside its root. Here is the helper in the form it usually ships:

import os

WORKSPACE = "/srv/runs/run-8842"

def write_file(name: str, data: bytes):
    if ".." in name or name.startswith("/"):
        raise ValueError("bad filename")
    path = os.path.join(WORKSPACE, name)
    with open(path, "wb") as f:
        f.write(data)

Several inputs defeat this. The first is percent-encoding, the scheme URLs use to carry reserved characters as a percent sign and two hex digits so that / travels as %2f, which arrives whenever the filename passes through a layer that URL-decodes — a web framework, a tool that treats the argument as part of a URL, a client that encodes before sending. The model (or an injection steering it) emits ..%2f..%2fetc%2fpasswd. The blocklist sees no literal .. and no leading slash, because at the moment it runs the string still spells %2f. A decode step later turns %2f back into /, and now the resolver walks two levels up and into /etc. The check ran against the encoded form; the resolver ran against the decoded form; they disagreed, and the resolver won.

The second is a symlink already sitting in the workspace. Suppose an earlier step, or a previous run sharing the directory, created /srv/runs/run-8842/data as a symlink pointing at /etc. The model now asks to write data/passwd. That name contains no .. and no leading slash, so it sails through the guard. os.path.join produces /srv/runs/run-8842/data/passwd, which looks safely inside the workspace as a string. But the resolver follows the data symlink, and the actual write target is /etc/passwd. Nothing in the name was suspicious; the danger was in the filesystem, which the blocklist cannot see because it only has the string.

The symlink case is the one that shows what the guard is really measuring, so it is worth separating two predicates that the word "inside" runs together. One is lexical: does this string, read as characters, begin with the characters of the workspace path. os.path.join guarantees that one and nothing more, because composing strings is all it does. The other is topological: does the location these characters name sit underneath the workspace in the tree of directories the filesystem actually maintains. The resolver answers the second by walking the path one component at a time, asking the filesystem what each component is, and continuing from wherever that answer points. When it reaches data and the filesystem says "a link to /etc", the walk continues from /etc, and the remaining component is appended there. Lexical containment held throughout and topological containment was lost at the third component. A guard that reads the string can only ever check the first predicate, and the write is decided by the second.

The third family is alternate representations that the specific filesystem or platform normalises in ways a naive check misses — a trailing dot or space that Windows strips, a backslash treated as a separator on some platforms, Unicode forms that normalise to / or . under the resolver's rules, an overlong UTF-8 encoding of a separator. The common thread is that "does this string contain .." is a question about one spelling, and the resolver accepts many spellings that mean the same thing.

The diagram shows why the guard is in the wrong place: it sits before the decode and the resolver, reasoning about a representation that later stages will transform out from under it. Every input above shares that structure — a difference between the string the check sees and the path the OS builds.

The operation that actually establishes safety#

The fix is to stop validating the name and start validating the destination, which means resolving the path fully — decoding, joining, normalising, and following symlinks — and then checking that the resolved result is inside the allowed root. The order is the entire point. You resolve first so that whatever transformations the OS would apply have already been applied, and you check second so that you are checking the real target rather than a spelling of it.

import os

def safe_path(root: str, name: str) -> str:
    root_real = os.path.realpath(root)
    # realpath decodes nothing, but resolves symlinks, '.', '..',
    # and normalises separators to the OS's canonical form.
    candidate = os.path.realpath(os.path.join(root_real, name))
    # The safety property is established HERE, on the resolved path:
    if os.path.commonpath([root_real, candidate]) != root_real:
        raise ValueError("path escapes workspace")
    return candidate

The line that establishes the safety property is the commonpath check, and it is only meaningful because candidate is a fully resolved path. os.path.realpath collapses .. segments, normalises separators, and — crucially — follows every symlink in the path, so the data → /etc link is resolved to its real target before the check runs. commonpath then asks the only question that matters: is the real target genuinely underneath the real root. If data/passwd resolves to /etc/passwd, the common path of /etc/passwd and /srv/runs/run-8842 is /, which is not the root, and the write is refused. The check is comparing two canonical paths, so there is no spelling of the input that can make a path outside the root look like one inside it.

Why commonpath rather than the obvious string test is the step most people skip, and it is where a correct-looking rewrite of this function goes wrong. The tempting version asks whether the resolved candidate starts with the resolved root, using ordinary string prefix matching. That test compares characters, and paths are not sequences of characters for this purpose, they are sequences of components. A run directory named /srv/runs/run-8842-shared starts with the characters of /srv/runs/run-8842 and is a sibling of it, not a child, so the string test admits a write into a directory belonging to something else. commonpath compares component by component: it returns the longest sequence of whole components the two paths share. If the candidate is genuinely underneath the root, every component of the root appears at the front of the candidate, so the longest shared sequence is the root itself and the comparison holds. If it is not underneath, the sequences diverge at some component and the longest shared prefix is shorter than the root, so the comparison fails. The equality test is therefore exactly the containment question, which is what makes it a proof rather than a heuristic — and it is a proof only about the two canonical paths handed to it, which is why the realpath calls above it are load-bearing rather than tidying.

The general form of the workspace path is /srv/runs/<run-id>/, and the root passed to safe_path should be that directory for the run in hand rather than /srv/runs. Rooting the check one level too high makes every concurrent run's directory a legitimate destination for every other run, which is a smaller escape than reaching /etc and is still an escape — the containment proof is only as narrow as the root you hand it.

One caveat keeps this honest: on a live filesystem there is a window between resolving the path and opening it, during which a symlink could be swapped in — the classic time-of-check-to-time-of-use race. Where that window matters, you resolve and open atomically, using openat with O_NOFOLLOW on the final component, which opens a name relative to a directory handle you already hold and refuses outright if that last component turns out to be a symlink, or you open the root directory once and resolve everything relative to that descriptor. For an agent workspace the realpath-then-commonpath check is the substance of the control; the atomic variant closes the last gap when the workspace is shared or attacker-writable.

The race is worth stating as a sequence, because otherwise "resolve first" and "the resolution can go stale" sound like they contradict each other. realpath asks the filesystem a question and returns an answer. open asks the filesystem to act. Between those two calls the filesystem is free to change, since it is shared mutable state and your process holds nothing that pins the answer. If something can replace the final component with a symlink in that interval, the path you proved safe and the path you open are two different objects that happen to be spelled the same way. The atomic variants remove the interval rather than shortening it: resolving relative to a directory handle means the walk starts from an object you are already holding, so renaming the directory afterwards cannot redirect it, and O_NOFOLLOW on the last component makes the open fail outright if that component is a link, which turns a silent redirect into an error. Notice that the fix is the same move as the main control — stop reasoning about a name and start holding the thing.

Percent-encoding deserves a word about where it is handled, because realpath does not decode it. If the filename can arrive percent-encoded, you decode it once, at the boundary where it enters, before it reaches safe_path, and you decode exactly once — decoding in a loop until nothing changes is its own vulnerability, since it lets an attacker double-encode to survive an intermediate check. The rule is a single, explicit decode at a known layer, then canonicalise, then check.

It is worth being precise about why resolving first and checking second is not interchangeable with checking first and resolving second, because the order is the whole safety property and it is tempting to think either sequence works. If you check the string before resolving it, you are checking a representation the resolver has not yet touched, which is the original bug — the check passes on ..%2f or on data/passwd, and the resolution afterwards is unconstrained. If you resolve and then check, every transformation the OS would apply has already happened, so the string you are comparing is the one the filesystem will actually use, and there is no later stage to reintroduce a difference. The check has to be the last thing that happens to the path before the open, and it has to run on the resolved form. Any code between the check and the open that re-derives, re-joins, or re-decodes the path reopens the gap, because it can produce a path the check never saw. A common way this regresses in practice is a helper that returns the validated name as a string, which a caller then joins to the root again with its own os.path.join; the second join is unchecked, and the guarantee is gone. Return the resolved, checked path and open that exact object — do not hand back a name that someone downstream will resolve a second time.

Why the regex keeps coming back#

The blocklist is tempting for a reason worth naming, because the temptation does not go away once you have read this chapter. The name is right there as a string, a regex is cheap, and rejecting .. and slashes and control characters feels like it covers the obvious attacks — and it does cover the obvious ones, which is exactly what makes it dangerous. It passes the tests you write, because you write tests from the same mental model that produced the blocklist, so you test the spellings you thought of and never the ones the resolver accepts that you did not. Blocklists on paths fail the way blocklists always fail, and a little worse, because the thing you are trying to constrain is not the string at all. The OS resolver decides the outcome, and it reads encodings, follows symlinks, and normalises representations that sit entirely outside any pattern you can write. A blocklist is an allowlist's weaker sibling in general; on paths it is weaker still, because the gap between the string and the resolved path is unbounded.

It is worth separating, in this arrangement, what is forced and what was chosen, because only the chosen parts are available to a redesign. That the resolver decides the write target is forced; it is the operating system's job and no application code overrides it. That resolution happens after your code has run is forced by the same fact. That the workspace is a directory inside a larger visible filesystem is a choice, and it is the choice that gives traversal somewhere to go. That the model names the files is also a choice, made for a product reason — run histories read better with model-chosen names than with opaque identifiers — and it is the choice that makes the name attacker-influenceable in the first place. Recognising which is which is what tells you that the canonicalise-and-check control is unavoidable while the exposure it defends is not: a workspace the process cannot see out of removes the class, and identifiers assigned by your own code remove the input.

When you cannot change the design so that the agent stops choosing names, the canonicalise-and-check control is necessary and you should want more, because a single check is a single point of failure and this is a place where defence in depth — stacking several independent controls so that no one mistake is sufficient to produce the bad outcome — is cheap. Run the write under an identity that has no permission to write anywhere but the workspace, so that even a resolved path outside the root is refused by the filesystem's own permissions rather than only by your check. Mount or bind the workspace so the process cannot see the rest of the filesystem — a per-run directory that is the process's entire visible world removes the whole class, because there is nowhere outside to traverse to. Cap what the write can do: a size limit and a file-count limit turn a traversal that slips through into a bounded problem rather than an overwrite of a system file. Log the resolved path, not the requested name, so that when you audit the run you see where the write actually went.

None of this is unique to model-named files; it is standard hardening for any code that writes attacker-influenced paths. What the model adds is the volume and the surprise — an agent names files constantly, during ordinary tasks, and a name it chose after reading a hostile document is indistinguishable from a name it chose sensibly, right up until the resolver walks it somewhere it should not.

The rule underneath is the one the rest of the topic keeps running into: the control has to live where the decision is made. For HTML the decision belongs to the parser, so the control is an encoder at the render call; for paths it belongs to the resolver, so the control is a comparison against a resolved path.

The path sink at least fails locally, on your own filesystem, where your own permissions and mounts can still catch it. The next chapter moves to a sink where the model's output leaves your process entirely and hits another service's API, and where the natural defence — schema validation — feels like it is doing the job while missing the request that matters.