Permission filters belong in the retrieval query, not after the answer

A retrieval-augmented assistant answers from documents rather than from what a model absorbed during training. A search picks passages out of a prepared store, an application pastes those passages into a prompt, and a model writes an answer from them. Point that arrangement at an organisation's own documents and a problem arrives with it: some of those documents are readable by some people and not others, and a similarity search has no idea who is asking. An access control list, ACL for short, is the set of identities permitted to read something. A principal is the identity making the request, resolved from the session into a user identifier and a set of group memberships. The only question this chapter settles is where along the chain from ingest to generation the ACL is consulted.

There are two candidate places, and they are not two implementations of one design. Consulting the ACL before the search changes which passages exist as far as everything downstream is concerned. Consulting it after generation changes what is displayed, and nothing else. An assistant can produce an answer whose every citation names a document the reader is entitled to see, while its sentences carry a fact from a document they are not, and that is possible because a fact can travel out of a restricted passage and into an answer along a path that carries no document identifier with it.

Why a citation list is not a record of what the model read#

Start from what a model receives. A model receives a prompt and nothing else. It has no handle on the vector store, no session, no view of the retrieval call that assembled its input. Everything it can say about a company's internal affairs came in through that one channel, so the answer is a function of the prompt in the strict sense: a sentence stating that a complaint was substantiated in March can only have come from a passage in the prompt that carried the substantiated complaint. That is the first step, and it is worth stating as a one-way street. A fact can move from the prompt into the answer. Nothing done to the answer afterwards can move that fact back out of the prompt, out of whatever archived the prompt, or out of the provider that received it.

The second step is what a citation list actually is. The citation list at the bottom of an answer is not a record of what the model read. It is a record of the passages the model chose to attribute, which for most prompt templates means the passages it quoted or leaned on closely enough to name. Paraphrase produces no quotation. A model that reads a paragraph, extracts one fact from it, and writes that fact in its own words has used the passage and attributed nothing, because there is no quoted string for a citation to hang on. So the citation list is a subset of the passages used, and the passages it omits are precisely the paraphrased ones.

Put a permission check on top of that list and the third step follows on its own. The check's inputs are the document identifiers in the citation list. Its output is a permission decision on each of those identifiers. Now vary the thing the check exists to catch: hold the question, the reader and the template fixed, and change only whether a restricted passage was among the retrieved chunks. In the world where no restricted passage was retrieved, the check examines identifiers the reader may read and passes. In the world where a restricted passage was retrieved, paraphrased and left uncited, the check examines the same identifiers and passes again. Its output did not move when the variable it exists to discriminate did.

That is the shape of the mistake, and it is tempting for a good reason. A citation check looks like enforcement — it reads permissions, it can drop sources, it produces a decision — and every part of that appearance is real. What it lacks is any dependence on the quantity it is supposed to control. A defence whose output is constant while the thing it must distinguish varies is not a weak defence; it is not a defence at all, and its passing carries no information. Making it stricter is the same mistake at a higher setting, since strictness only changes how the check treats identifiers it can see, and the leak is in the sentences it cannot.

The output-filtering idea fails on its own terms as well, before you even get to the boundary argument. Post-generation redaction works by matching output against restricted source text, and a paraphrase shares almost no n-grams — the short runs of consecutive words that any such matcher compares — with its source. "A complaint was substantiated in March" and "the investigating panel upheld the allegation on 14 March" have one content word in common. Facts leak without wording: a date, a number, the existence of a document, the fact that an investigation happened at all. There is no string to match because the model's job was to not produce one.

Those two sentences are worth reading as an exact accounting rather than as an illustration, because the overlap between them is not merely small — it is a single token, and naming which token it is names the residual. Everything a matcher could catch has been replaced except the month. What survives paraphrase into matchable form is the part of a fact that has no synonyms: the date, the identifier, the figure. A leak stated without one of those leaves a matcher with nothing to compare at all, which is why the class of leak that matters most — the existence of an investigation — is the class a matcher is structurally blind to.

Where the bytes go before any check runs#

The mechanism is easier to see as a sequence than as an argument, because the ordering is the whole point: every meaningful boundary the restricted text crossed was crossed before the check existed.

By the time the check runs, the restricted paragraph has been written verbatim into the prompt archive, where it is now readable by anyone with access to observability tooling and retained for whatever the log retention period is. It has been transmitted to the model provider, where it may sit in request logs under a separate retention policy and may have been used to populate a prompt cache keyed on the prefix. If any of those systems is queryable by the same reader who asked the question — and prompt archives frequently are, because they were built for debugging — the answer on screen is the least of it. A redaction pass applied to the output would be hiding the visible part of a disclosure that already completed three network hops earlier.

That accounting is what turns the argument from a preference into arithmetic. Whether a restricted passage is disclosed is not one probability; it is a product of several independent ones. Write the factors out in the order the sequence above visits them: the chance the store returns the chunk to the application at all, the chance the prompt archive is reachable by someone without rights to the source, the chance the provider's request log or prompt cache is reachable, and the chance the answer repeats the fact where the reader can see it. Multiply those together and you have the exposure, in the same way that any risk expressed as a number of opportunities times a per-opportunity probability is a product rather than a sum.

Reading it that way settles which controls can work. A control placed at the end of the chain acts on the last factor only, and the product still contains every earlier one at whatever value it already had. A predicate inside the search sets the first factor to zero, and a product with a zero factor is zero however large the remaining factors are. Nothing about the quality of the downstream checks enters that conclusion, which is the useful part: the argument does not require anyone to agree about how good a redactor could be made.

The first factor has a second reading that names what has gone wrong more precisely. What should reach the model is capability intersected with authority — the capability the query was granted, which for an unfiltered search is every chunk in the collection, against the authority the asker actually holds, which is some much smaller set. An unfiltered search hands the full capability set forward and asks a later stage to perform the intersection on an object, the answer, from which the excess can no longer be subtracted.

As an analogy, and one that understates the problem, a citation check is a bag search at the exit of a reading room. It breaks in two places. A bag search happens after the reader has read and memorised the page, and memory is what paraphrase is; and a bag search at least inspects what is being carried out, whereas this check inspects a list the reader wrote themselves. The understatement is the copies: in a reading room the document stays in the building, and here the same paragraph was written into a log and posted to a third party before the reader reached the door.

The filter is a predicate on the search, not a step after it#

The decision has to happen where the candidate set is chosen, which means access labels travel in the chunk payload — the acl field from the contract table in the first chapter of this topic, which sets out what each stage boundary must hand the next — and the retrieval call carries a predicate over them. The query becomes a filtered nearest-neighbour search rather than a nearest-neighbour search followed by anything.

def retrieve(question: str, principal: Principal, k: int = 5) -> list[Hit]:
    qvec = embedder.embed_query(question)
    # principal.grants: {"user:a.rivera", "group:support", "group:everyone"}
    return client.query_points(
        collection_name="kb",
        query=qvec,
        limit=k,
        query_filter=Filter(
            must=[
                FieldCondition(key="acl_allow", match=MatchAny(any=sorted(principal.grants))),
                FieldCondition(key="tenant_id", match=MatchValue(value=principal.tenant)),
            ],
            must_not=[
                FieldCondition(key="acl_deny", match=MatchAny(any=sorted(principal.grants))),
            ],
        ),
        with_payload=["doc_id", "text", "source_url", "version"],
    ).points

Three properties of that call matter more than its syntax. The predicate is evaluated inside the vector store, so an unauthorised chunk is never returned to application memory, never rendered, never archived, never transmitted. The principal's grants are resolved before the search from the identity system rather than inferred from anything the user typed. And acl_deny exists separately from acl_allow, because most enterprise sources model restrictions as both grants and explicit denials, and collapsing them into one list gets the precedence wrong on exactly the documents that are restricted for a reason.

Getting the labels into the payload is the part that takes the work. The ingest stage has to read effective permissions from the source at crawl time and flatten them into a set of principal identifiers, which for Confluence means page restrictions plus space permissions plus inherited defaults, and for SharePoint means resolving broken inheritance per item. The flattening is a snapshot, and snapshots go stale: a group membership change or a page restriction added at 10:00 is not reflected in the index until the next crawl touches that document. That gap is the price of pre-filtering on stored labels, and it is why the labels stored should be groups rather than expanded user lists — group membership then resolves at query time from the identity provider, and only the document-to-group mapping is a snapshot. Revoking a person's membership takes effect on their next query; restricting a document takes effect on its next crawl.

The two revocation paths in that paragraph are the whole value of the group decision, and they are asymmetric. A label naming a group is evaluated against a membership set fetched at query time, so a membership change propagates immediately; a label naming users is evaluated against a list written at crawl time, so the same change waits for a crawl. The decision converts one of the two staleness windows to zero and leaves the other exactly as it was. A reviewer should ask for both to be demonstrated separately, since a system that handles one correctly says nothing about the other.

It is worth separating the requirement from the decisions here, because they get argued about as though they were the same thing. The requirement is that no reader receives content they have no right to read, and it admits several designs. Storing flattened labels in the payload and filtering inside the search is one. Holding a separate collection per tenant is another. Dropping chunks in application code after retrieval and before the prompt is built is a third. Only the requirement is fixed; each design meets some part of it at a different price, and the table below is what the prices look like side by side.

The second column is the one to read first, and it closes on itself: every row whose decision happens after the search reads Yes, every row whose decision happens before it reads No, and there is no row that breaks the split. That is not a coincidence of which five designs happened to be listed. The column is a restatement of the row's position in the sequence diagram, so it can be predicted for any design anybody proposes, without building it, by asking one question about where the decision sits relative to the retrieval call. A team can therefore rule out a category of proposal on the whiteboard.

One row does break the pattern, and the break is the informative part. The post-retrieval chunk drop reads "Yes, if drop is after prompt build" — a conditional where the others are flat. It is conditional because that design straddles the boundary rather than sitting on one side of it, and reading why sharpens where the boundary actually is. It is not the line between retrieval and generation. It is the line between the bytes existing only in a local variable and the bytes being rendered, logged and posted, which is a few statements later in the same function.

Read the remaining columns as gaps between adjacent rows rather than as levels, because no row's cost means anything on its own. The gap between the citation check and the chunk drop is the entire disclosure path, bought for one list comprehension. The gap between the pre-filter and the per-tenant collection is not security at all; it is granularity, which is why the last row's failure mode is stated as a limit of what it models rather than as a way of going wrong.

Dropping chunks after retrieval but before prompt construction is the interesting middle row, because it does close the disclosure path — the restricted text never leaves the process — and it is still a poor design. Filtering k results down to whatever survives means a user with narrow access gets three chunks where a broad-access user gets five, and a user with no access to any of the top 50 gets nothing at all, with no mechanism to search deeper. The pre-filter gets k authorised results by construction. In a store that supports it, the filter is applied during graph traversal rather than as a post-pass over an unfiltered result set, which is the difference between "the top 5 the user may read" and "whatever survives from the global top 5".

What a pre-filter costs#

Filtered approximate search is not free, and the cost is a recall cost rather than a latency cost. Approximate nearest-neighbour search, usually shortened to ANN, is what makes a vector query cheap enough to run inside a web request: instead of scoring the query against all three million vectors, the store walks a shortcut structure and touches a few thousand of them. HNSW indexes, which are what most vector stores build for this, navigate a proximity graph; when a filter excludes most of the graph, the traversal can stall in a region with no admissible neighbours and return fewer or worse results than an exhaustive scan of the admissible set would. Vector stores handle this with a cardinality estimate — below some threshold of matching points, they fall back to a brute-force scan over the payload index, which is exact and, for a few thousand points, faster anyway. The number worth measuring on your own corpus is filtered recall against an exhaustive baseline for your most restrictive principals, because a user whose grants match 0.4% of a 3-million-chunk collection is the case where filtered ANN quietly degrades and nobody notices, since the answers are merely mediocre rather than wrong.

The reason the most restrictive principal is the one to measure follows from how the graph is built rather than from how the filter is written. A proximity graph connects each vector to its nearest neighbours by meaning, and access labels have nothing to do with meaning, so the admissible points for a narrow principal are scattered thinly across the whole structure instead of sitting together in one region. A traversal that must step only between admissible points therefore runs out of moves, and it runs out sooner the narrower the principal is. That is why the degradation is a function of grant breadth and why an average-case measurement will miss it entirely.

The measured overhead on a 3.1-million-chunk collection with a keyword payload index on acl_allow was 11 ms of p50 search time unfiltered against 14 ms filtered, and 28 ms against 41 ms at p95. Building the payload index costs memory — roughly 60 MB for a set-valued field at that scale — and the ingest side pays a permission read per document, which for Confluence is one extra API call per page and dominates crawl time on large spaces unless it is batched by space.

Those four latency figures are worth reading as two gaps rather than as four levels. At the median the filter costs three milliseconds; at the tail it costs thirteen. The tail gap is several times the median one, and that ratio is the measurement that matters, because it says the cost of filtering is not a constant added to every query. It is concentrated in the queries that were already slow, which are the ones where the traversal had to work hardest and where the filter therefore had the most opportunities to reject a candidate. A team that budgets for the median figure and sizes its timeouts from it will be surprised by the tail, and the tail is where user-visible timeouts live.

The other structural option is one collection per tenant, which is the right answer when the boundary being enforced is between organisations rather than between roles inside one. Physical separation removes an entire class of mistake — there is no query anyone can write that crosses the boundary, because the collection name is chosen from the authenticated principal — and it makes deletion on offboarding a single drop rather than a filtered sweep. It stops helping the moment the boundary is intra-tenant, since one customer's own employees still need different views of the same documents, so most deployments end up with both: a collection per tenant for the hard boundary, and payload labels with a pre-filter for everything inside it. The cost of many collections is a memory floor per collection and an index count that becomes an operational concern in the thousands, which is why per-user collections are rarely workable.

Caching is where the cost is easy to miss. A retrieval cache keyed on the question alone becomes a mechanism for handing one user another user's authorised results, so the cache key has to include the principal's resolved grant set, which fragments the cache and drops hit rates. On a corpus with fifteen distinct group configurations, a question-only cache that ran at 34% hit rate fell to 9% when the grant set was folded into the key. That is a real efficiency loss and it is the correct trade; caching the retrieval step across principals is the same class of error as filtering after generation.

Roughly a quarter of the original hit rate survives that change, and the fifteen distinct group configurations are why. A key that was one question becomes a key that is one question per distinct grant set, so the same traffic divides across many more keys and each sees a fraction of the hits it used to.

Five readable chunks and an inference no chunk supports#

A pre-filter enforces who may read a chunk. It says nothing about whether the assembled answer is itself sensitive — five individually readable chunks can compose into an inference about a person that no single chunk supports — and it says nothing about whether the user should be asking. Those are policy problems that live above retrieval, and pre-filtering is not a substitute for scoping which corpora an assistant is pointed at in the first place.

It also does nothing about the conversation history. A chunk correctly filtered out today can already be sitting in the transcript of a session started before the restriction existed, and a system that replays prior turns into each new prompt will keep re-sending it. Session state needs its own retention rule, and the honest one is short.

The session case is the product argument arriving from the other side. A pre-filter zeroes the first factor for chunks retrieved from now on, and anything that already persisted prompt content — session state, an archive, a provider's cache — holds a copy of an exposure computed under the old labels. That copy has to be expired on its own schedule rather than inheriting the fix.

An instruction the model has no inputs to obey#

The mitigation that gets proposed in the meantime is an instruction: add "only use passages the user is permitted to read" to the system prompt. It cannot work, and the reason is mechanical rather than a matter of model quality. The model has no permission information — the only thing it receives is the prompt, and every passage in the prompt looks equally authorised because the pipeline put it there. The instruction asks the model to apply a predicate whose inputs are absent. Even granting a model that followed it perfectly, the restricted text has already crossed every boundary in the sequence above by the time the model reads the instruction, so the best case is an answer that declines to mention content that has already been logged and transmitted. The correct interim mitigation is to stop retrieving from collections containing restricted content until the labels are in the payload, which is a scope reduction the team can ship the same afternoon.

That instruction is the same failure as the citation check, and seeing them as one thing is worth more than treating them as two lessons. Both are controls whose output cannot vary with the fact they are meant to act on: the check reads a list that omits the leak, and the instruction addresses a component that holds no permissions. Both will report success identically on the day nothing went wrong and on the day something did. The test to apply to any proposed control is that small and that mechanical — find the variable it must discriminate, ask what its inputs are, and check whether one is a function of the other.

Where a permission decision is made is not an implementation detail with two acceptable answers. Made inside the search, it removes a passage from the candidate set, and every later stage operates on a set that never contained it. Made after generation, it operates on an object from which the restricted content can no longer be subtracted. The labels in the payload and the predicate on the query are what make the first version possible, and their cost — a permission read at crawl time, recall loss for narrow principals, a fragmented cache — is the honest price of a control that acts on the quantity it is supposed to control.

With versioning, deletion and access labels holding across the eight boundaries, the pipeline's contracts are sound. None of that says whether the thing answers questions well, and the fastest way to find out is a path so small that every answer it produces can be traced end to end — which is what the next chapter builds, and deliberately does not optimise.