The agent state document

A summariser asked to summarise a conversation will produce an account of a conversation, which is what it was asked for and usually the wrong artefact. The output of a compaction is read by exactly one consumer, the next model call, and that consumer is not catching up on what happened. It is deciding what to do next, which is a question about where things stand. The two descriptions can cover the same three hours of work and share almost no content.

The clearest symptom is an agent that re-issues tool calls it already made in the same session, in the first two or three turns after every compaction. That behaviour is not a model failure and it is not a structural one; it is the correct response to a context that records that searches happened without recording what any of them returned. The agent has no way to proceed except to re-derive the state, and re-deriving state means running the searches again.

Stepping through that link by link is what turns the symptom into a design fault rather than a quirk. The model holds nothing between calls, so what it does next is a function of the request in front of it and of nothing else. That request now says eleven companies were searched; it does not say what any search returned. To choose what to do next the agent needs the results. The only operation available to it that produces results is the search tool. Repeating the search is therefore not an error the agent is making — it is the shortest correct path from the information it has to the information it needs. Each of those steps is checkable, and only one of them can be changed: nothing obliged the compaction to record that searches happened rather than what they found.

What a fluent summary records#

Here is a compaction output from a research agent's session, trimmed but otherwise as generated.

The agent has been researching DORA compliance statements from European
competitors. It began by identifying the eleven companies on the target
list, then searched for each one in turn, reading company websites,
regulatory filings and press releases. Several searches returned limited
results, and the agent adjusted its queries to include the German and
Dutch terms for the regulation. It has been recording findings in the
report file as it goes and has covered most of the target list. Some
companies proved harder to research than others, particularly the
smaller ones without English-language investor relations pages.

Read that as a person and it is a competent status update. Read it as the input to a decision about what to do next and it answers none of the questions the next turn has: which of the eleven are done, what was found for each, which searches have already been tried and with what result, whether "most of the target list" means eight or ten, and what the remaining ones are.

The distinction is between a record of activity and a record of position. The summary above is organised around what the agent did, in the order it did it, which is how any general-purpose summariser organises a transcript because that is what a transcript is.

The state document is the larger of the two, and that is worth stating plainly rather than glossing, because the argument for compaction is usually framed in tokens saved.

The stumble that hides here is a reasonable one: compaction exists to make the request smaller, so between two compaction outputs the smaller must be the better one. That holds if the only thing a compaction does is shrink the request. It does not hold once the output has a job on the next turn, because a compaction that is 370 tokens cheaper and causes two searches to be re-run has not saved anything — it has moved the cost from the compaction into the turns after it, where nobody is measuring. The right unit is the size of the request over the turns that follow, not the size of the artefact. Under that unit the larger document is the smaller cost, and the reason the mistake is common is that only one of those two quantities appears in a compaction log.

The four sections that carry position#

The same work, written for the machine that has to act next, looks like this. The format is not important — JSON, a fixed set of markdown sections, or a typed object all work — but the four sections are.

{
  "task": "For each of 11 EU competitors: does a public DORA compliance statement exist, and what does it commit to? Output: report.md, one section per company.",
  "findings": [
    { "company": "Sparkassen Digital", "status": "statement found",
      "url": "https://…/regulatorik/dora", "fetched": "2026-03-11T09:41:00Z",
      "commits_to": "ICT third-party register published annually; 4h RTO for critical services",
      "written_to_report": true },
    { "company": "Nordlys Betaling", "status": "no statement found",
      "evidence": "site search + 3 web queries (EN/NO), investor page checked 2026-03-11",
      "written_to_report": true }
  ],
  "open": [
    { "company": "Adriatica Pay", "blocked_on": "statement is a PDF behind a cookie wall; fetch returned 403 twice" },
    { "company": "Vlaamse Clearing", "blocked_on": "not started" }
  ],
  "queries_run": [
    "Sparkassen Digital DORA compliance statement",
    "Sparkassen Digital DORA Konformitätserklärung",
    "Nordlys Betaling DORA samsvar"
  ],
  "commitments": [
    "report.md keeps one H2 per company in the order given by the user",
    "companies with no statement still get a section saying so"
  ],
  "artifacts": [
    { "path": "report.md", "rev": 14, "contains": "sections for 9 of 11 companies" }
  ]
}

An agent given that does not search for Sparkassen Digital again, because the finding is present with its source; it does not search for Nordlys Betaling again, because the negative result is recorded as a finding with the evidence behind it, which is the item narrative summaries drop most reliably. Negative results are expensive to obtain and produce nothing to narrate — "the agent searched and found nothing" is not a sentence a summariser feels obliged to keep — so an agent that only records positives will re-check every absence after every compaction.

Getting a model to produce this rather than prose is mostly a matter of asking for the schema instead of asking for a summary, and then refusing output that does not satisfy it. Requiring findings and open to partition the task — every company on the list appears in exactly one of them — turns a soft editorial judgement into a check the harness can run, and a compaction whose two lists cover nine of eleven companies is rejected and retried rather than silently accepted. The same applies to monotonicity: the number of resolved findings can go up between compactions and should never go down, so a generation that drops from nine findings to seven is a bug you can catch without reading anything. Schema validation of this kind catches perhaps a third of bad compactions in practice, which is not a solution but is far better than the zero percent that prose affords.

Both checks work by the same move, and extracting it is worth more than either check. Each one converts a property of the content into a property of the shape. The task names eleven companies; the document above puts two under open and the rest under findings, so counting the entries in the two lists and comparing the total against eleven is arithmetic where "did the summary lose a company" was judgement. Monotonicity does the same across time, since the count of resolved findings is a number the harness already holds from the previous compaction, so comparing two integers replaces asking whether anything went missing. Neither check knows what a good summary is and neither needs to. The reason prose affords nothing here is not that prose is worse writing; it is that a paragraph has no property a program can compare against anything, and that is what asking for the schema rather than for the summary actually buys.

The queries_run list does a job that looks redundant next to findings and is not. Its purpose is to stop the agent from re-issuing a query that produced nothing, which is different from knowing that a company has no statement, and it also lets the next turn vary its approach rather than repeat one. Keeping it to the query strings rather than the results costs about 15 tokens per entry against the 3,800 tokens a set of search results occupies.

That ratio is the whole design in one line and it is worth reading as a ratio. A query string is a few words; the results it returned are a page of snippets, and the second is more than two hundred times the size of the first. What the agent needs in order not to repeat itself is the query, not the results — knowing that a particular phrasing has been tried is enough to stop it being tried again. So the entry that prevents the expensive mistake is the cheap half of the pair, and discarding the payload while retaining the key costs almost nothing and removes almost all of the duplication. Any part of a state document with that shape, where a small durable key stands in for a large regenerable payload, is nearly free to keep.

References survive; payloads do not#

The reason a state document can be smaller than the material it replaces while remaining actionable is that it stores keys rather than contents. A fetched web page that arrived as 6,200 tokens of HTML-derived text leaves behind a URL, a timestamp, and one sentence of what it established. If the next turn genuinely needs the original wording, it fetches the URL again — one tool call, deterministic, and it returns the same document.

That property is what makes re-fetching acceptable and re-searching not. A URL fetch is addressed by a stable key and returns the same content; a search is a query against a ranked index that changes, so re-running it three hours later can return a different set, in a different order, and the agent can reach a different conclusion from the same intent.

The distinction is worth stating as a test rather than as two examples, because it is what decides every eviction in the document. Ask whether the operation that would recover the payload is a function of its argument alone. A fetch of a URL is: same address, same document, and if it is not, the document genuinely changed and the agent should see the new one. A search is not, because its output depends on the argument and on the state of an index the agent does not control. Storing a key is safe exactly when replaying the key is deterministic, and that condition, rather than the size of the payload, is what separates a Reference from a Claim in the eviction function below. A payload with no deterministic way back has to be turned into a claim while the payload is still in front of you, because that is the last moment the claim can be checked against anything. The general rule is to evict payloads and retain the smallest key that reproduces them: a URL, a file path plus revision, a memory-store key, a database row id, a tool_use_id if your harness can replay results by id. Anything that has no such key has to be summarised into a claim at eviction time, because there is no way to get it back.

def evict_tool_result(block: ToolResultBlock) -> Reference | Claim:
    """Payloads leave; the cheapest thing that reproduces them stays."""
    if key := addressable_key(block):        # url, path+rev, memory key, row id
        return Reference(key=key, gist=one_line_gist(block), size=block.tokens)
    return Claim(text=extract_finding(block))  # irreversible: get it right now

Where the retained material is large enough that even a state document cannot hold it — a 40-page extraction, a table of 900 rows — the destination is a store outside the window rather than the summary. Writing it to a file the agent can read back, which is what the API's memory tool is for, keeps the context small while leaving the content addressable. The state document then holds the path and a description of what is in it, and the eviction has cost nothing at all.

Where a narrative is the better artefact#

Two cases invert the argument, and both come down to who or what consumes the compaction. The first is a conversational product where the agent's job includes continuity of relationship — a coaching assistant, a long-running support thread, a tutor. There the tone of the earlier conversation, the fact that the user was frustrated at turn 12 and reassured at turn 20, is genuinely part of what the next turn needs, and it is exactly what a fielded state document has no slot for. The workable form there is a state document with a short prose field for relational context, kept explicitly separate from the fields that carry commitments.

The second is any system where a human reads the compaction, which includes almost every system during its first month. Operators debugging an agent want to know what it did, in order, and a JSON blob of current position is a poor substitute. That is an argument for emitting both — the state document into the context and a narrative into the log — rather than for putting the narrative into the request.

The honest limitation is that this whole approach depends on the tools being addressable, and many are not. A search tool that returns a formatted block of text with no result ids, an internal API that returns a report with no way to request the same report again, a shell command whose output depends on the state of a machine that has since changed — none of those can be referenced, so every one of them forces an irreversible claim at eviction time. Making tool results addressable is a change to the tools rather than to the compactor, and it is usually the higher-leverage fix; a search tool that returns stable result ids alongside its snippets converts the most common eviction from a lossy summary into a pointer.

A compaction is a handoff to the next turn, and the artefact that survives it should be the position the work has reached: what has been established and on what evidence, what is still open and what has already been tried on it, which files or reports exist and at what revision, and which commitments the agent has already made. Written that way it is bigger than the prose it replaces and it stops the agent paying for the same information twice.

There is one property of the arrangement that the next chapter has to deal with. A state document is generated by a model, from material that may itself include a previously generated state document, so by the eleventh compaction of a long session each recorded claim may have passed through several generations of paraphrase with the original page nowhere in the input. That is what turns a hedge into a decision the user never made.