A net-negative memory system shows flat success and higher cost

A net-negative memory feature is one whose costs exceed its benefits while every individual measurement stays inside its normal range. It has no crash, no error-rate spike and no alert, because nothing in it is broken: retrieval retrieves, the store stores, the model answers. What has gone wrong is a trade, and a trade going the wrong way is invisible to any instrument that watches one side of it.

Reading such a system requires four numbers rather than one, and the reason is that each of them individually has an innocent explanation. Latency up by a fifth of a second is a retrieval budget. A larger prompt is an injection. A flat eval result is what the second chapter's arithmetic says an underpowered comparison usually produces. Only the combination is diagnostic, and only if somebody is looking at all four at once — which is unusual, because they live on four dashboards owned by three teams.

Four numbers that only mean something together#

Six months after launch, nothing about the memory feature has ever paged anyone. Error rates are flat, the store is healthy, retrieval p99 — the time the slowest one request in a hundred takes — is well inside its budget, and the quarterly eval reports task success of 74.1% against a pre-memory baseline of 74.3%. The numbers that did move are on other dashboards owned by other people: conversations now contain 0.44 correction turns each on average against 0.31 before, median task latency, the p50 in the table below, is up 410 milliseconds, and the blended inference cost per task, input and output tokens averaged across all tasks, is up 12%.

Two of those descriptions use different statistics on purpose. Latency is reported as a median because a task's latency is a number with a long tail, and the middle of the distribution is what a typical user waits. Corrections are reported as a mean because a correction count per conversation is mostly zero — the median conversation contains none, before and after — so a median would read the same on both sides of a change that doubled the rate. Reaching for the median out of habit here would hide exactly the signal the row exists to carry.

The healthy-looking figure at the top has the same problem in reverse. Retrieval p99 being inside its budget is a statement about one component's slowest requests, and the component was never the thing at risk: the whole retrieval step is inside its allowance at every percentile, and it still adds its full allowance to every task that uses it. A budget that a component meets is not evidence that the component is worth its budget.

Read that table by its gaps rather than by its levels. The absolute figures are all unremarkable — plenty of healthy assistants run at 6,700 input tokens and two and a half seconds. What the table reports is that the first row's gap is smaller than the instrument can resolve while every other row's gap is comfortably larger than its own noise, and a feature that costs a fifth of a second and 12% of the bill and returns nothing measurable is being paid for out of somewhere.

The two cost rows decompose in ways worth knowing before anyone argues about them, and the decompositions are not the same shape as each other. The 12% is not one thing: about 10.7 points of it come from the 1,500 injected tokens on the 62% of tasks that retrieve, and the remaining 1.3 points come from the extraction call that runs at the end of every session whether or not anything is written. Those two are billed on different populations. The injection is paid only when retrieval fires, so its contribution to the average is the injection cost scaled down by the retrieving fraction — turning off retrieval on the other tasks would remove it. The extraction call is paid on every session regardless of outcome, so it is a floor that survives any change to the read policy at all, and a team that fixes retrieval and expects the whole 12% to come back will find that part of it does not.

The 410 milliseconds decompose too — 120 ms to embed the query, 90 ms for the approximate-nearest-neighbour search, 180 ms for the cross-encoder rerank, which is the reranker of the previous chapter reading query and candidate together in a single pass, 20 ms of store round trips. Adding those gives back the 410, and the reranker alone is more than 40% of it. That is a component whose contribution to task success has never been measured separately from the rest, which makes it the largest single line of unjustified cost in the system rather than merely the largest line.

Correction turns are the sharpest of the four#

Task success is a coarse instrument: a report can be right after the user spends two turns dragging the assistant back to what they asked for, and the task scores as a success. Correction turns catch the dragging. A correction turn is a user turn that restates, negates or overrides a constraint the assistant has already applied — "no, gross, I said gross", "use calendar quarters", "stop assuming EMEA". Detecting them needs a classifier over turn pairs rather than a keyword list, because the phrasings are unbounded, and the classifier needs validating like any other grader; this one was checked against 300 hand-labelled turns and agreed on 88% of them, with most disagreements on turns that both corrected and extended the request.

def correction_rate(conversations: list[Conversation]) -> float:
    corrections = sum(
        1
        for c in conversations
        for prev, turn in zip(c.turns, c.turns[1:])
        if turn.role == "user" and classify_correction(prev, turn) == "override"
    )
    return corrections / len(conversations)

The rise from 0.31 to 0.44 per conversation is a 42% increase — an extra thirteen hundredths of a correction on every conversation, which against a base of thirty-one hundredths is a little over two-fifths more. Both readings are worth keeping, because the absolute one is what the product experiences and the relative one is what makes it sound alarming, and a review will hear whichever is quoted.

The rise survives the noise band easily for a reason that has nothing to do with the classifier being good. It is measured over every production conversation rather than over a 1,000-task eval, and at roughly 400,000 conversations a month the sampling error on a rate is small enough to ignore. The only real uncertainty left is the classifier's 88% agreement, and that affects the level rather than the change: a classifier that systematically over-counts corrects the same way before and after, so the difference between the two periods survives an error that the absolute rate does not. That is a useful property of behavioural signals generally — they are noisy per instance and enormous in sample size, whereas eval task success is clean per instance and small in sample — and it is why the two instruments belong side by side rather than in competition.

The asymmetry between gains and regressions is what makes this signal move while task success does not. A task that used to fail and now succeeds produces no user reaction worth recording, because the user simply gets what they asked for and says nothing. A task that used to succeed and now fails because the assistant applied a preference from three weeks ago produces a correction turn, a moment of confusion about why the assistant did that, and occasionally a support ticket. So the two directions contribute equally to task success and unequally to the correction rate, and a change with equal numbers of gains and regressions will read as neutral on one instrument and negative on the other. The same accounting applies in ordinary software releases, where a regression in a working feature costs more trust than a fix in a broken one, and it is the reason release gates count regressions separately rather than netting them against improvements.

Averages net out exactly what you need to see#

The +2.0-point aggregate from the enlarged eval set is a difference of two totals, and differences of totals discard the per-task structure underneath them. The paired harness already produces that structure, because each task has an outcome in both arms, so the flips can be counted directly.

from collections import Counter

def deltas(results: list[tuple[str, str, bool, bool]]) -> dict[str, Counter]:
    """results: (task_id, slice, success_on, success_off)"""
    by_slice: dict[str, Counter] = {}
    for _, sl, on, off in results:
        cell = ("gain" if on and not off else
                "regress" if off and not on else
                "both_pass" if on else "both_fail")
        by_slice.setdefault(sl, Counter())[cell] += 1
    return by_slice

Across 1,000 paired tasks, 71 flipped from fail to pass and 51 flipped from pass to fail. The headline says memory won 20 tasks, and 20 is exactly 71 minus 51 — which is the arithmetic identity that hides the second number. Any reported difference is the gain count minus the regression count over the number of tasks, so an unlimited number of regressions is compatible with any headline as long as the gains rise to match. What the product experienced is 71 tasks getting better and 51 tasks that used to work breaking, and those 51 are where the correction turns come from. Reporting a memory change as a single net figure hides the second number entirely, which is why the release gate for this system was eventually rewritten to require both a positive net difference and no more than 2% of tasks regressing — a bar the current configuration fails, with 51 regressions in a thousand.

Slicing those flips by the labels attached during set construction turns the flat aggregate into something that can be acted on.

Two columns of that table are the same quantity computed twice, which is the check that it is being read correctly. The explicit-preference row's 41 gains against 17 regressions is a net of 24 tasks, and 24 tasks out of 180 is 13.3 points — the number in the difference column. The inferred-facts row nets to 8 tasks against memory out of 240, which is the 3.3 points. Every row works this way, and the last row is not an independent measurement: it is the three rows above it added up, which is why its gains and regressions are the column sums.

That makes the aggregate a weighted average, weighted by slice size, and the weights are what produce the flat result. A gain of 13.3 points carried by 180 tasks and a loss of 3.3 points carried by 240 tasks nearly cancel before the 580 controls dilute what is left, and a difference of nearly seventeen points between two real effects arrives at the top as a difference of two, sitting inside the noise band. Reading the aggregate as "memory does approximately nothing" is reading a weighted average of a large positive and a moderate negative as though it were a small effect on everything.

Explicit preferences — things the user said in an imperative and would recognise as instructions — carry a gain far outside the band, with a discordance rate high enough that 180 tasks are sufficient to see it. Inferred facts, which the extractor derived from conversational context without the user ever asking for them to be remembered, are negative. The controls do approximately nothing, which is the reassuring part: irrelevant retrievals on tasks that do not need memory are mostly ignored rather than actively harmful. The word doing the work there is "mostly" — a difference of 0.7 points on 580 tasks contains 12 gains and 8 regressions inside it, so irrelevant injection is harmless on average and occasionally decisive, and the average is the wrong statistic for deciding whether to allow it.

Reading the regressions back to their causes#

The 51 regressed tasks all have the structured trace from the previous chapter, so the injected records can be classified by why they should not have been there.

Irrelevant retrieval is the largest group and the one with the least satisfying fix, and the reason is structural. Retrieval is asked for the k nearest records and it returns k records, because "nearest" is defined relative to whatever is in the store rather than to any absolute standard of relevance. On a task where nothing relevant exists, the k nearest are simply the k least irrelevant, and they are injected with the same confidence as a perfect match would be. Nothing in a nearest-neighbour search can decline to answer.

A similarity floor is what adds the absolute standard, by refusing to inject anything below a fixed score, and it trades directly against the ranking misses from the previous chapter. Raising the floor removes injections whose scores were low, and the relevant record on a ranking-miss task also has a low score — that is what a ranking miss is. The two populations overlap, so any floor removes some of each.

Stale records are a write-policy problem presenting as a read-policy problem. The March correction was written as a new record rather than superseding the January one, so the store held both. From the ranker's point of view these are two records with no ordering between them, since recency is not part of a similarity score, and it preferred the older phrasing because it happened to be worded more like the query. No amount of ranking work fixes a store that contains two contradictory records with no ordering between them; the repair is an update path in the write policy, which is the knowledge-update competency the previous chapter borrowed from LongMemEval, and it is the reason that category exists in the benchmark at all.

Conflicting injections are the least common and the most damaging per instance, since the user has stated a constraint in the current request and watched the assistant do the opposite. They are also the cheapest to mitigate without touching retrieval, because the defect is one of precedence rather than of content: two instructions are in the prompt and nothing tells the model which wins. Placing injected memories before the user's request and stating explicitly that the current turn overrides stored preferences supplies the missing rule.

The same shape on different traffic#

A net-negative pattern is not proof that the memory system is badly built. Retrieval quality on this system was reasonable, and the eval could have produced the same flat aggregate with a perfect retriever, because most of the product's tasks do not depend on anything a previous session established. A perfect retriever on a task where nothing relevant is stored still returns the k least irrelevant records.

Memory is workload-dependent in a way that comparisons across memory architectures keep confirming; the design is being judged against the traffic it serves, and a store that would be transformative in a long-running personal assistant can be dead weight in a product where most sessions are self-contained. The quantity that decides this is the share of requests whose answer depends on something established earlier, and it is a property of the product rather than of the code — which means the same system, ported to different traffic, produces a different verdict with nothing changed.

The pattern is also not, by itself, an argument for removal. The slice table shows a 13-point gain on 18% of the eval set, and switching the whole feature off gives that up along with the harm. What the shape does establish is that the current configuration cannot be defended on aggregate task success, and that any defence has to be built from the slice that carries the effect and the specific costs on the other side.

Those costs are so far only the ones a dashboard can see: tokens, milliseconds and correction turns. A store of user facts also carries obligations that never appear on a latency graph and a maintenance bill that shows up in engineering time rather than in the invoice, and both of them belong in the ledger before anyone weighs 2 points against anything.