A bad write conditions every later session; a missed one costs a re-ask

A write gate can be wrong in two ways. It can decline a fact that was worth keeping, which this chapter calls a miss, and it can admit a fact that should not have been stored, which it calls a bad write. Both are ordinarily reported as one number, because a gate is evaluated the way a classifier is evaluated and a classifier's mistakes are counted together. Reporting them together is the mistake this chapter is about.

The two errors differ on how they end. A miss is corrected by the person who noticed it, in the turn after they notice it. A bad write is not corrected by anybody, because nothing about the answers it produces looks wrong. That difference is not a matter of degree, and once it is measured rather than felt, it decides where the default of an admission policy sits and which side of the line has to be instrumented before the policy is allowed to loosen.

The costs do not come out anywhere near even#

Put the two errors on the same footing and take them apart into the quantities that actually differ. A missed write costs one clarifying question, paid once, at a moment when the user is present and can supply the fact immediately. The error is self-announcing: the user notices, corrects it in a few words, and the conversation continues. If the gate misses the same fact repeatedly the cost accumulates linearly, and a user who has to restate their warehouse four times will complain, which is a detection mechanism that works.

A bad write costs a distortion applied to every future session in which the record is retrieved, for as long as the record exists. The error is silent, since a wrong memory produces answers that look exactly like right ones — the assistant does not hedge, does not ask, and has no way to know that the premise it was handed is false. Worse, the wrong record is self-confirming from the user's side: an assistant that consistently talks about us-east-1 reads as an assistant that has understood the setup, so the user stops mentioning regions at all, which removes the evidence that would have corrected it.

Those two descriptions can be turned into a comparable quantity, and the arithmetic is worth doing once with your own numbers, because the ratio is larger than intuition suggests. The count of answers a bad record distorts is the number of sessions it survives, multiplied by the number of turns per session on which it is retrieved. Take a user with two sessions a week and a mean time to detection of six weeks, which gives a dozen sessions in the window. Take a record that is retrieved on roughly a third of turns in the sessions it is relevant to. The product is a dozen sessions times a third of a session's turns, so for the answer to land where Arbor's did — on the order of fifty to eighty responses — a session has to run to something between a dozen and two dozen turns, which is an ordinary length for a support conversation. At Arbor's volumes a single row really did reach 61 responses over six and a half weeks.

Now set the cost of a re-ask at one turn. To do comparable damage by missing, you would have to miss the same fact fifty times over, and you cannot miss it fifty times, because the user would have told you off after the fourth. The bounded, visible, self-limiting error and the unbounded, invisible, self-reinforcing one are simply not in the same category.

That identity — distorted answers equals sessions survived times retrievals per session — licenses one thing precisely: it converts a single wrong row into a count of answers, which is a unit you can put beside the one-turn cost of a re-ask and compare. What it does not license is a claim about how much each of those answers cost, and that residual matters. Of those sixty-one responses, most were of no consequence and one was pasted into an incident runbook. The identity counts them the same way, so it establishes that the two error directions are orders apart and it cannot tell you which of the distorted answers to worry about, and averaging over them is the same error one level down.

The multiplier that makes this decisive is detection. Arbor's audit of 200 sampled records found 38% of them wrong or unjustified. In the same period, users corrected or deleted 0.4% of their records through the settings page. Read those as a gap rather than as two levels. If users were the correction mechanism, the rate at which they corrected records would be bounded above by the rate at which records were wrong, and the two would be in the same neighbourhood; instead the correction rate sits nearly two orders of magnitude below the error rate. Those two numbers do not describe a system in which bad writes get cleaned up at some rate; they describe a system in which essentially none of them do. A missed write has an inherent detector attached — a present, motivated human — and a bad write has none unless you build one and pay for it.

There is a compounding term as well, and it is the one that turns a polluted store into a store that cannot be reasoned about. Later extraction runs — the passes in which a model reads a conversation and proposes facts for the gate to judge — read conversations that were themselves conditioned by earlier records. When the assistant has been discussing us-east-1 for six weeks, a summariser asked to pull durable facts from those transcripts will happily produce a second record asserting the same thing, this time sourced to a turn in which the assistant said it and the user did not object. The provenance chain now points at the model. Two records agreeing with each other also survive naive contradiction checks, because contradiction detection looks for disagreement and finds none.

There is a defence people reach for at this point, and it is worth taking apart because it sounds like it closes the gap. The defence is that a wrong record will eventually be contradicted — the user will say something incompatible with it, a conflict check will fire, and the record will be retired. Follow the conditions that has to meet. A contradiction requires a later utterance about the same slot, and the self-confirming dynamic above is precisely a mechanism that stops those utterances being produced: an assistant that consistently gets the region right, as far as the user can tell, gives the user no reason to mention regions again. So the coverage of the defence is inversely related to how convincingly the record is doing damage. The records most likely to be contradicted are the ones the user notices, and those are the ones that were going to be corrected anyway.

That is the whole case for conservative admission, and it is worth stating in the form that answers an interviewer directly: a missed write costs one re-ask, and a bad write conditions every future session until someone finds it, which in practice means until the account is deleted.

Evidence about the output is not evidence about the user#

The rule that follows is about what an utterance is evidence of. Consider a turn where the assistant produced eleven paragraphs and a table in response to a yes-or-no question, and the user replied "shorter please". A gate that writes "User prefers concise answers" has made a specific mistake: it treated a correction of one output as a statement about a standing trait.

The mistake is tempting for reasons worth naming, because the same reasons will make it tempting in your own system. The utterance is imperative, it is unambiguous, it is about the assistant's behaviour rather than about the domain, and it arrives with an implicit "you got that wrong" that feels like exactly the signal a learning system should capture. Every surface feature says "preference".

What the utterance lacks is any indication that the preference generalises past the output that provoked it. Work through the alternative explanation and the reason it survives: the assistant produced eleven paragraphs to a yes-or-no question, eleven paragraphs is too long for a yes-or-no question by any standard, and a person confronted with that would ask for something shorter whatever their standing taste in answer length happened to be. The eleven paragraphs are a sufficient explanation for the complaint without any trait being involved. Since the observation is fully explained without the trait, it is no evidence for the trait. The same user, three turns later, asked for "the full comparison, all of it" when the question warranted it.

The general form: a correction issued in response to a specific output is evidence about that output. It becomes evidence about the user when the same correction recurs across outputs that have little in common — different topics, different lengths, different sessions — or when the user states the generalisation themselves. The recurrence condition works for the same reason the single instance fails: if a complaint appears after a short output as well as a long one, output length is no longer a sufficient explanation, and something about the user has to be doing the work. "Always keep it under five sentences", "I keep having to ask you this", and "from now on, no preamble" all assert generality, and can be admitted on a single occurrence because the user did the generalising. "Shorter please" does not, and needs repetition to earn the same status.

Candidates go to a log, and the log promotes#

Implementing that rule means candidate observations cannot go straight into the store. The requirement is only that a weakly evidenced claim must not become a record. The design decision that satisfies it is to give the claim somewhere else to go: an append-only observation log, which is cheap, unretrieved and disposable, with a separate promotion step deciding when accumulated observations amount to a durable claim. Nothing forces that arrangement — a gate could simply drop weak candidates — and what the log buys is described further down. Here is the log after five weeks for one user:

[
  {"obs": "wants_concise", "session": "ses_9d2a", "turn": 6,
   "quote": "shorter please", "prior_output_tokens": 1840, "at": "2026-02-04"},
  {"obs": "wants_concise", "session": "ses_9d2a", "turn": 14,
   "quote": "still too long", "prior_output_tokens": 1120, "at": "2026-02-04"},
  {"obs": "wants_verbose", "session": "ses_a17b", "turn": 3,
   "quote": "the full comparison, all of it", "prior_output_tokens": 210, "at": "2026-02-09"},
  {"obs": "wants_concise", "session": "ses_c02f", "turn": 2,
   "quote": "just the answer, no setup", "prior_output_tokens": 970, "at": "2026-03-02"}
]

Three of the four observations point the same way, which a naive counter would read as a preference confirmed by majority. The promotion rule has to be stricter than that, and the prior_output_tokens field is why. Two of the three concise observations came from the same session minutes apart and were both reactions to outputs over a thousand tokens, so both are explained by the outputs rather than by the user. What makes the pattern durable is a complaint about a short output, and the third row is the one that carries information — an output of a couple of hundred tokens drew a complaint that it was too short, which no theory about the user's taste for brevity predicts.

from collections import defaultdict
from datetime import timedelta

MIN_OBSERVATIONS = 3
MIN_SESSIONS = 2
WINDOW = timedelta(days=90)
GENERALISING = ("always", "from now on", "every time", "in future", "i keep having to")


def promote(log, now):
    recent = [o for o in log if now - o.at <= WINDOW]
    by_claim = defaultdict(list)
    for o in recent:
        by_claim[o.obs].append(o)

    for claim, obs in by_claim.items():
        if any(g in o.quote.lower() for g in GENERALISING for o in obs):
            yield claim, "explicit_generalisation", obs[-1]
            continue
        if len(obs) < MIN_OBSERVATIONS:
            continue
        if len({o.session for o in obs}) < MIN_SESSIONS:
            continue
        if contradicted_by(recent, claim):
            continue
        yield claim, "repeated_across_sessions", obs[-1]

Each guard licenses a different conclusion, and running the four log entries through them shows which one is doing the work. The window filter keeps only observations recent enough that the user is plausibly still the same person about this; all four survive it. Grouping by claim gives three wants_concise entries and one wants_verbose. The generalisation check looks for a phrase in which the user did the generalising, and none of the four quotes contains one, so nothing takes the short path. The observation count stops wants_verbose immediately, since one is fewer than three — a single utterance never promotes on its own. wants_concise passes that guard with three, and passes the session guard too, because the three come from two distinct sessions rather than one. So it reaches the last check with every count in its favour, and only contradicted_by stops it.

contradicted_by is therefore the clause that saves this particular user from a wrong record. wants_verbose and wants_concise are declared opposites, and a claim with any live contradiction in the window is held back rather than decided by majority. Held back means no record and no re-ask either, since the assistant still has the current conversation to read; the user simply gets a system that has not concluded anything about them yet. The alternative arrangement, where the majority wins, would have written a concision preference on a two-to-one count and then produced a curt answer the next time this user asked for a full comparison.

One property of the log is what makes it cheap, and it is not the storage. The log is never retrieved into a prompt, so nothing in it can condition an answer, which means a wrong observation in the log costs nothing on the axis that the whole of this chapter has been about. That is the entire difference between the log and the store, and it is a difference in what the rows are wired to rather than in what they contain. What the log does not escape is disclosure: a row recording what a user said about themselves is data held about that user, and it belongs in the same deletion path as everything else.

The log itself is worth having even for observations that never promote. It costs a row per candidate, it is never retrieved into a prompt, and it is the only place you can go when you want to know why the store looks the way it does or what a stricter threshold would have changed. Raising MIN_OBSERVATIONS from three to four is a query against the log rather than a redeploy-and-wait-a-month experiment, and the same log supplies the labelled examples you need when someone asks how the gate performs. A store with no staging area behind it can only be tuned forwards.

Run the promotion policy retroactively over Arbor's February and March logs and it turned 41 concise-style observations spread across 3,400 users into 6 promoted preferences. A manual read of those six found all six defensible, each backed by complaints about outputs of visibly different lengths in at least two sessions. The 35 that did not promote include the genuinely concise-preferring users who had only said so once, and those users will keep getting medium-length answers until they say it again — which is the cost the asymmetry says is worth paying.

Where conservative admission stops being the right default#

Conservative admission is a default rather than a target, and none of this argues that recall is unimportant or that the ideal store is empty. A gate that writes nothing has a real cost, paid in re-asks, in users who conclude the product does not learn, and in the durable facts that were expensive to elicit in the first place — a customer's entire data model, established over an hour of conversation, is not something anyone wants to reconstruct weekly.

What the asymmetry argument actually establishes is narrower than "write less". It establishes where the default sits and which side of the line needs a detection mechanism before you are allowed to loosen it. The whole argument rests on bad writes being permanent, and permanence is a property of the system around the store rather than of the store itself. If you build the detector — sampled audits, user-visible records, contradiction checks that actually fire — bad writes stop being permanent, the asymmetry shrinks, and you can afford to write more. The conclusion is conditional on a fact about your own infrastructure, and a team that treats it as unconditional will hold a tight gate long after it has earned the right to open it.

The asymmetry is also specific to the write path, and it does not transfer to retrieval, which is why the intuitions from reranking mislead here. A mediocre candidate that survives ranking wastes context on one request and then ceases to exist; the next request re-runs the whole selection from scratch. A mediocre record that survives the gate is in the selection pool for every subsequent request, and there is no downstream stage that will remove it, because ranking chooses among what is stored and cannot question whether it should have been.

The gap between one re-ask and sixty-one distorted answers is what every rule in this chapter is paying for. It is why a single "shorter please" goes to a log instead of the store, why a claim with a live contradiction is held back rather than settled by majority, and why a gate is allowed to loosen only in proportion to the detection you have built around it. The alternative is not a worse store; it is a store nobody can reason about, in which the wrongness is invisible, self-reinforcing, and bounded only by how long the account lasts.

What the argument has not settled is who applies these rules. The evidence tests above were written as code, which is one answer; a model asked to judge whether an utterance generalises is another; and a user pressing a button labelled "remember this" is a third. Those three gates behave very differently on the same conversations, in ways that turn out to be predictable, and comparing them is the next chapter.