A classification string in the text is not a boundary

Once a chunk carries a tenant, a classification level and an audience, there is an obvious-looking place to put them: into the text itself, prepended to the chunk before it is embedded, in the same position as a heading path (chapter 1, where a chunk is prefixed with the chain of headings the splitter cut it away from). Such a header — Tenant: Acme Corp | Classification: Confidential | Visible to: finance-team — sits directly above the content it describes, travels wherever the content travels, and is legible to a human who reads the snippet. Nothing about it can drift away from the chunk it labels, which is the property that makes it feel safer than a field stored off to one side.

That intuition is worth taking apart carefully, because the marker's placement is the thing that disqualifies it. A payload field is a stored value that travels with a vector and can be named in a query filter; a filter is a predicate over those fields, evaluated by the vector store to decide which points are eligible to be scored at all. A marker inside the chunk text is neither. Where a fact about access is written turns out to decide whether anything can act on it, and the phrase "the header is a control" only means something if some component acts on it.

What each component actually reads#

Six components, and the classification string is legible to exactly one of them in a way that could change an outcome. Every other row either does not see it as a distinguishable thing or has no capacity to act on it.

The gap that matters in that table is not between one row and its neighbour; it is between two timelines the table lays side by side. The first two rows happen at ingestion, once, long before anybody asks a question. The last four happen at query time, on every request. Only one object crosses from the first timeline to the second, and that is the stored point: a vector, plus a payload. A fact written into the chunk text at ingestion crosses as part of the vector, having been turned into floats. A fact written into a payload field crosses as a payload field. Those are different roads, and the table says where each of them arrives.

Follow the marker down its road and the outcome is forced. The filter stage — the one component that can remove a chunk from consideration before its text goes anywhere — reads payload fields named in the query filter, and reads nothing else. The marker is not a payload field, so the filter stage cannot see it. By the time any component can see it again, the chunk has been scored, selected and handed over as text. An access decision is a decision about which chunks are in the returned set; the set is chosen before it is returned; and a string inside a chunk is readable only after that chunk has been chosen. So the marker becomes legible at the moment the decision it was meant to inform has already been taken the other way. Anything a retriever can read, a retriever has already returned.

Read the same table forwards rather than backwards and it says something more useful than "the string fails". Row four names the only enforcement point, and it names what that point reads. Any control that is going to work has to arrive as a payload field referenced by a query filter, because that is the sole input to the sole component positioned to act in time. The table is not merely diagnosing the marker; it is specifying the shape of the replacement.

The embedding model turns "Confidential" into a contribution to a vector, which has one concrete and slightly perverse effect worth noting: because every confidential chunk in the tenant now shares that phrase, those chunks cluster with each other, and a query that lands anywhere near the cluster pulls several of them at once. It is the sibling-collapse effect from chapter 2 — where chunks sharing a long identical prefix are pulled toward one another in the vector space — applied to precisely the population you least want retrieved together.

So the header is a control only if the generator is a control. A model told to refuse is following an instruction, and an instruction is a thing that can be outweighed by other instructions — by a user who says the classification was a mistake, by a document in the same context that says the restriction was lifted, by a phrasing the refusal rule did not anticipate. Even a model that refuses perfectly has already read the confidential text into its context, so the content crossed the boundary before the refusal happened; whatever the model says next is influenced by material the user was not entitled to. A refusal is a behaviour with a high success rate and no floor, and a boundary is a thing with a floor. Describing the first as a layer of defence is how a missing check gets signed off, because a review sees a list of protections and the reviewer's attention goes to whether they are correct rather than to whether the second and third exist.

The counting is worth doing explicitly, since it also settles a question that comes up later. One row of six can act on the string; five cannot; the two counts account for the table with nothing left over, and the partition is exhaustive because every component either receives the string as a distinguishable value or does not. What that licenses is a conclusion about every possible version of the marker, rather than about this one. The count is a property of what each component reads, and no component's reading changes when the words inside the string change. A marker that is perfectly worded, perfectly current and perfectly formatted is read by the same one row.

There is a smaller problem alongside the large one. The header is returned with the chunk, so anything that surfaces retrieved snippets in the UI shows the string to the user, and the string names a group and a classification level. Even when the refusal works, it discloses that a confidential finance-team document exists and roughly what it is about.

The day after a user leaves the group#

Yesterday someone was removed from finance-team. Today they ask the assistant a question that is semantically close to a confidential finance chunk.

The tenant filter passes, because they are still an Acme Corp user and tenant_id is unchanged. The ANN search — the approximate nearest-neighbour structure that finds close vectors quickly by walking a graph of them rather than comparing the query against all 2.4 million — returns the chunk, because the search computes distance to the query and nothing in it knows about group membership. The chunk enters the prompt, header and all. The model reads Visible to: finance-team, and then has to decide whether this user is in finance-team — which it cannot know, because group membership is not in its context; the system prompt told it to check, and there is nothing to check against. In practice it either refuses on the presence of the marker alone, refusing for everybody including the people who should see it, or it proceeds. The index in this state returns to the departed user exactly what it returned the day before, because nothing in the pipeline observed the change.

That failure has a shape worth naming, because it is the shape of most defences that turn out not to be defences. The marker was written once, at ingestion, and is thereafter a constant. The thing it has to discriminate — whether the person asking, right now, belongs to the named group — varies by user and by day. A constant cannot discriminate a variable, whatever the constant says. What makes the mistake tempting is the marker's one genuine virtue, its inseparability from the content: a field stored elsewhere can be forgotten by a query, and a header never can. But inseparability is the wrong property to want. Enforcement has to happen before the content moves, and a marker that travels with the content arrives exactly where the content arrives, which is to say everywhere, and too late in each place.

What it should have returned is nothing from that document. Removal from a group has to change the candidate set, before scoring, by way of a filter derived from that user's live permissions:

def search(user, query: str, k: int = 10):
    groups = acl.groups_for(user.id)          # live, from the source system
    if user.tenant_id is None or not groups:
        raise PermissionError("no scope resolved for user")

    return store.query_points(
        collection_name="kb",
        query=embed(query),
        query_filter=models.Filter(
            must=[
                models.FieldCondition(
                    key="tenant_id", match=models.MatchValue(value=user.tenant_id)
                ),
                models.FieldCondition(
                    key="acl_group", match=models.MatchAny(any=list(groups))
                ),
            ]
        ),
        limit=k,
    ).points

The filter references acl_group, a payload field, against a group list resolved at query time. When the user leaves finance-team, the next request resolves a shorter list, the chunk stops matching the filter, and it is never scored. Latency of the change is however long the ACL service's cache lives, which is a number you can set and state.

It is worth separating what that code is forced to do from what it chooses to do, because the two get defended in the same breath and only one of them is negotiable. Resolving the group list at query time rather than at ingestion is forced: the fact changes between the two moments, so any earlier resolution is answering a question that has since moved. Referencing a payload field rather than the text is forced, by the table above. What is chosen is the rest. Two things make this robust rather than merely present. The single search entry point takes a user and builds the filter itself, so no caller can construct an unfiltered query; a bare store.query_points call anywhere else in the codebase is the vulnerability, and a lint rule or a wrapper package is a reasonable way to keep it from being written. And the function fails closed — an unresolvable tenant or an empty group list raises rather than searching without a constraint, because the natural implementation of "no groups" is an empty any list, which some stores treat as matching nothing and others treat as an absent condition.

That last one deserves its distinction. Failing closed is a decision in general and a requirement here, and what converts it is the ambiguity in the stores themselves: when one plausible implementation of an empty list means "match nothing" and another means "no condition at all", the difference between the safe reading and the catastrophic one is a library version. Raising before the query removes the dependence on which reading you got.

Where the filter is applied relative to the search also matters, and the two options are not interchangeable. Passing the filter into the store, as above, lets the engine constrain the graph traversal so that ineligible points are never candidates; fetching the top 50 and discarding the ones that fail the check afterwards is a different operation with two defects. The first is arithmetical: a request for 10 returns however many of the global top 50 happen to survive the filter, which for a user with narrow permissions can be two or zero, and the correct tenth-nearest permitted chunk was never fetched. The second is that the results you discard were still read out of the index and passed through your process, which makes the discard step a piece of application code standing between restricted content and a log line.

The arithmetical defect is worth walking, because its consequence is the opposite of what an operator expects. Post-filtering scores the whole collection, keeps the global top 50, and then intersects that fixed set with the user's eligible points. How many survive depends on what share of the collection that user may read: a user with broad access keeps most of the 50, and a user with narrow access keeps a handful or none. The request asked for 10, and the returned count is not 10 but the size of an intersection, which shrinks as permissions shrink. Meanwhile the tenth-nearest chunk that user was entitled to may sit far below rank 50 in the global ordering, and post-filtering never fetched it. Pre-filtering computes the top k over the eligible set alone, so it returns k whenever the eligible set has at least k members, and rank is measured in the only ordering that was ever relevant.

The same conclusion arrives by a second route that shares no step with that one. Some stores switch to post-filtering internally when the filter is highly selective, so it is worth knowing which strategy yours uses and what it does when the filter excludes most of the collection; the answer is usually a tunable, and the wrong setting shows up as mysteriously short result lists for exactly the users with the least access. That symptom is reported from production by people who have not looked at any ranking arithmetic, and it matches the arithmetic exactly — narrow permissions, short lists. Two independent routes landing on the same set is the strongest evidence available that the mechanism has been correctly described.

If you want a genuine second layer, add one made of code: after retrieval and before prompt assembly, re-check each returned chunk's acl_ref against the live ACL service and drop any that fail. It costs a batch call and catches the case where the filter was built from a stale group cache. It is a layer because it does not depend on persuading anything.

Re-embedding on membership change inverts the design#

The proposal that follows naturally from all this is to keep the header accurate by re-embedding affected chunks whenever group membership changes. It is the wrong direction on every axis, and it is worth being specific about why, because it sounds like diligence.

Group membership is the fastest-changing fact in the system. On a corpus of 2.4 million chunks with a few thousand employees, joiners, leavers and project reassignments produce membership changes daily; a single reorganisation moving forty people between two groups can touch every chunk restricted to either. The vector is the most expensive artefact to change: re-embedding 40,000 chunks means 40,000 embedding calls, a rewrite of 40,000 points, and — with typical batch throughput of a few hundred chunks per second — several minutes during which some points carry the old header and some the new, which is a window where the index disagrees with itself about who can see what.

Two figures in that paragraph share a leading digit and are unrelated, which is worth saying before anybody divides one by the other. Forty is a count of people; 40,000 is a count of chunks restricted to the groups those people moved between, and it is set by how much material those groups own rather than by how many of them there are. The number that produces the outage window is the second one: chunks divided by throughput, with throughput in the low hundreds a second, lands in minutes rather than seconds, which is exactly long enough for the index to be internally inconsistent while somebody is querying it.

Compare that with the filter, where the same reorganisation costs one write to the ACL service and takes effect on the next query. Storing a fact in a vector means paying an embedding to change it; storing it in a payload field means paying an update; resolving it at query time means paying nothing. The right home for a fact is set by how often it changes, and re-embedding puts the most volatile fact in the most expensive place.

The decisive objection is that even after all that expense, the marker is still advisory. A perfectly current Visible to: finance-team in the embedded text is read by the same six components as before, and the same five of them still cannot act on it. That follows from the count established above rather than from a further judgement: since no component's reading changes when the string's contents change, no programme of keeping the string current can add an enforcement point. Correctness has been bought for the weakest link at the highest available price, which leaves the system exactly as strong as it was.

A refusal is a default, not a layer#

None of this says the model should never refuse. A refusal rule in the system prompt is a fine thing to have as a behavioural default — it improves the median outcome and costs nothing. What it cannot be is the mechanism you point at in a review when asked what stops an unauthorised chunk from reaching a user, and it cannot be counted as one of the layers.

The marker itself keeps a narrower use, and it is worth stating so the advice is not read as "delete the string". A classification label written into text is a label for a human reader on the far side of the boundary: somebody who has already been correctly granted access, reading a snippet in a UI or pasting it into a document, and who needs to know how to handle what they are looking at. It is good at that job precisely because of the property that disqualifies it as a control, since it travels with the content into places no payload field reaches. The conditions attached to that use are strict. The access decision must already have been made correctly somewhere else; the audience for the string must be authorised readers rather than the retrieval machinery; and both costs stay on the bill — the string is visible to whoever receives the snippet, and it still pulls the chunks that share it toward one another in the vector space.

Where filtering itself gets hard#

Per-chunk ACL fields are not always the right granularity. Where the security model is coarse, a collection per tenant gives isolation that no query construction error can cross, at the cost of managing thousands of collections. Where a user belongs to hundreds of groups, a MatchAny over that list gets expensive and some stores cap its size, which pushes you toward a smaller derived set of access labels or an external pre-filter that produces candidate document ids. These are real engineering tradeoffs with real limits, and they are all tradeoffs within filtering, since none of them is available at all to a design whose boundary is a string.

Nothing above says metadata is unimportant to security; the reverse. The fields from chapter 5 — the document id, the revision identifier and the ACL reference stored beside each vector — are what make the filter expressible and what make an audit possible after the fact. Storing them is necessary; it is the step that has to be followed by a query that references them. A payload full of correct access fields that no filter names is in the same position as the header, arriving with the content instead of before it.

Where a fact about access is written decides whether anything can enforce it, and the six-row table is the whole argument compressed. A marker in the text takes the vector road out of ingestion and arrives at the generator, which can comply or not; a value in a payload takes the payload road and arrives at the filter stage, which can exclude before anything is scored. The choice between those two roads is made at ingestion, months before the request that depends on it, and it cannot be repaired afterwards by improving the marker, since every version of the marker travels the same road.

One consequence of all this lands on the next chapter. The filter changes the candidate set before anything is scored, which means recall is always measured under a filter, and a query set replayed with a different tenant or a different group list is measuring a different index. Any number about whether contextual headers were worth it has to hold that constant along with everything else — which is one of several ways a clean-looking evaluation turns out to have been comparing two things at once.