Choosing a boundary strategy

A boundary strategy is how a tool decides which part of a large result the model receives. The four available are offset paging, cursor paging, bounded truncation, and pre-summary in either of its two forms — a server-side rollup or a model summariser. Each of the preceding chapters argued one of them into shape in the context of the failure it fixes, which leaves the comparison itself unmade: given a real tool, which one is right.

The comparison matters because the wrong choice here does not announce itself. An offset over a hot table produces a plausible digest with rows missing; a cursor walk over a work queue produces a description of a state that never existed; a rollup answers the question nobody asked. None of those raises an error, fails a test, or generates a support ticket, so a team that picks by habit — or by a blanket rule adopted after the last incident — finds out from a user months later or not at all.

The four strategies do not fail in the same way#

Everything the previous six chapters established about these options fits in one comparison, which is worth having in front of you at the moment the decision is made rather than reconstructed from memory.

Read across the bottom two rows and the shape of the decision appears: coverage of unanticipated questions runs opposite to cost and to how well the result can describe its own limits. Raw rows answer anything and cost everything; a rollup costs almost nothing and answers only what you thought of; a summariser buys back some of that coverage by giving up the ability to say what it dropped.

The decision that leads into that table is not a preference. It is a short sequence of questions about the payload and the question being asked of it.

The questions in that diagram are not arbitrary, and knowing where each comes from is what lets it be applied to a tool the diagram does not cover. The first — does the answer need individual rows — decides whether any row has to cross the boundary at all. When the answer is a total, a distribution or a handful of outliers, the rows are an intermediate result, and computing an intermediate result on the side of the boundary that has cheap exact arithmetic beats shipping its inputs to the side that has expensive approximate arithmetic. The second question arises only once rows must cross, and it asks whether the traversal has to be exact: must every matching row be seen once and once only. The third is not a separate concern but the answer to whether that guarantee can be given, because seeing every row exactly once is a promise about a sequence of calls, and a sequence of calls is only as stable as the ordering it resumes against. A mutable sort key means the ordering is being rewritten between calls, so the promise cannot be kept and the honest design accepts an approximate scan rather than pretending otherwise.

Offset paging has no branch in that diagram leading to it, which is deliberate. It survives in one narrow case — a single-page read where the offset is always zero — and that case is bounded truncation wearing different clothes.

Picking a strategy from that sequence rather than from the last incident is the whole of the method, and offset paging is what shows the sequence is more than a tidier way of stating preferences. No answer to any of the three questions selects it. Where rows are not needed, a rollup beats it on every axis. Where they are needed and the traversal must be exact, a cursor beats it, because the cursor's resume point survives writes that the offset's does not. Where the traversal need not be exact, truncation beats it and costs one call rather than many. A technique with no cell of its own survives on familiarity, and familiarity is exactly what LIMIT ... OFFSET has: it is the paging every SQL tutorial teaches, so it is what gets written whenever nobody is choosing.

Three payloads, three answers#

A support queue holds about 900 open items and is written to constantly: agents claim items, priorities get bumped, statuses change. The question the agent asks is always some version of what should be worked on next, and it is asked against an ordering that carries meaning, priority DESC, updated_at DESC. Nothing about that question requires seeing all 900 items, and a cursor walk over a queue this hot would take thirty seconds to complete and describe a state that never existed. Bounded truncation is the fit: return the top 20, say that 900 matched and 20 were returned in that ordering, and let the agent re-query when it has worked through them. A fresh query is more correct than resuming a cursor here, because a resumed cursor is deliberately looking away from the items that changed while the agent was busy — which in a work queue are exactly the items that now matter most.

{
    "matched": 903,
    "returned": 20,
    "order_by": "priority DESC, updated_at DESC",
    "narrow_by": ["queue", "priority", "assigned_to"],
    "items": [...],  # no cursor: re-query rather than resume
}

Why a fresh query beats a resumed cursor there runs opposite to the advice of the preceding chapters, so it is worth taking slowly. A cursor is a promise to continue past what has already been seen. On a work queue, the items that changed while the agent was busy are exactly the ones whose place in priority DESC, updated_at DESC has moved towards the front, since a bumped priority lifts an item and a fresh update lifts it again. A resumed cursor is defined to look past the front. So it is defined to look away from precisely the items that changed, which on a queue are the items that now matter most. The walk also takes as long as the number of pages requires, and the ordering is being rewritten underneath it throughout, so its last page and its first page describe different queues. Neither of those is a defect in the cursor. They are a mismatch between what a cursor guarantees, which is continuity, and what this question wants, which is currency.

A transaction ledger splits, and it splits on the question rather than on the table. Asked how much was refunded in March, the agent is asking for one number derived from about 4,000 rows, and the rollup answers it: the computed total, the row count it covers, and the largest few refunds with their customers and reasons. Nothing about that answer improves if the rows arrive. Asked to reconcile every March refund against the processor's settlement file, the same table needs a different tool entirely, because the answer genuinely depends on each row being seen exactly once — and here the ledger's redeeming property is that its sort key is immutable. Rows are appended and never reordered, so a cursor on (created_at, id) gives a stable enumeration with none of the staleness caveats that updated_at carries, and 4,000 rows across 80 pages is a real but acceptable cost for a reconciliation that runs once a month.

The eighty is the same division as before — 4,000 rows at a fifty-row page size is eighty pages — and what makes it affordable here is the frequency rather than the size. A monthly reconciliation pays eighty round trips once a month, against an answer that genuinely depends on each row. An agent asked for a total pays them every time somebody asks, and what it produces at the end is a language model's sum of four thousand decimal amounts rather than the database's. Same table, same rows, same page size: what differs is how often the traversal runs and whether anything waiting at the end of it needs the rows one at a time.

Two tools over one table is the right outcome, not a design smell to be resolved. Trying to serve both questions from one paginated list_transactions gives the aggregate case an eighty-page traversal and the reconciliation case nothing it did not already have.

An hour of logs lands on pre-summary, and the choice between the two pre-summary paths follows from how well the question is known. Explaining a latency spike is a question with known shape, so a rollup answers it in 1,150 tokens with a basis block that says what it covers and a drill-down that reaches the raw slice. The open-ended incident question — what is unusual in this window — has no known shape, and that is the narrow case where the summariser's non-determinism buys something a query cannot. Running both is reasonable if the rollup is the default and the summariser is a separate tool the agent reaches for deliberately; running the summariser on every log call spends $4 and fourteen seconds to answer questions the rollup already answered exactly.

Putting a rule where that sequence of questions belongs is the failure worth naming, because it is a failure that looks like maturity from the outside. A standard settles an argument without anyone reasoning at the point of use, it is checkable in review, and it usually comes out of a real incident — three genuine virtues. What it generalises, though, is the remedy rather than the mechanism. The incident behind the cursor standard had a specific shape, an offset counted into an ordering that concurrent writes kept changing, and the rule says nothing about that shape, so it lands with equal force on a tool whose rows nobody needs and a tool whose sort key never moves. A rule derived from the mechanism reads differently: where a traversal must be exact and the sort key is mutable, use a cursor. That sentence carries its conditions, and the conditions are what keep it from being applied where it does harm.

Granularity decides more than the boundary does#

Choosing a boundary is downstream of a question this topic has mostly deferred, which is whether the tool should exist in this shape at all. A tool that returns 40,000 rows to a model is answering the question "what does the database contain" when the agent asked "what should I do", and no pagination scheme fixes a tool whose granularity is wrong — the subject of topic 5.10, on how finely to carve a tool surface. Before comparing strategies, it is worth asking what the agent will do with the rows, because the answer is frequently a count, a filter or a join that the server could have performed, and that observation collapses the choice rather than informing it.

The frontier here moves the boundary somewhere else entirely. Anthropic's "Code execution with MCP: building more efficient AI agents" and OpenAI's "Programmatic Tool Calling" guide both describe letting the model write code that calls tools, so that a 40,000-row result lands in a sandbox — an isolated execution environment where the generated code runs with no path into the conversation — and the code filters, joins and counts it there, and only the answer crosses into the context window. That reframes the problem: instead of choosing which lossy view of the payload to send, the model specifies the reduction itself, per query, and the raw data never enters the window at all. It does not repeal anything in this topic — the code still needs pagination to read a large table, the reduction is still lossy, and a result that is a sample still needs to say so — but it does mean the aggregation no longer has to be anticipated by the tool author, which is the limitation the rollup ran into and the summariser paid for with non-determinism.

The mechanism under that is worth naming, because "lands in a sandbox" is doing the same kind of work any unexplained assertion does. A result becomes expensive at the moment it is appended to the conversation, since that is what makes it input on this turn and on every turn afterwards. Code the model writes does not run in the conversation. It runs in a process where the rows are ordinary values in ordinary memory, and the only thing appended to the conversation is what that code returns. So a forty-thousand-row result can be held, filtered, joined and counted at the cost of a program doing those things, and the boundary this topic has been drawing moves — it now sits between the sandbox and the model rather than between the database and the model. What crosses is whatever the code chose to emit, which is why the reduction stops having to be anticipated. The model writes it after seeing the question.

The choice is also perishable, in the same way a measured per-row token count is. A queue that grows from 900 items to 90,000, a ledger that gains a partial-refunds table, a log format that adds a field — each moves the arithmetic without anyone thinking of it as a context decision. The strategy chosen today is correct for today's row size, today's write pattern and today's window.

Working this through on a real tool is five steps, and the first two are the ones teams skip. Measure the p99 payload with the production tokenizer inside a real tool-result block, rather than estimating from bytes, and decide what share of the window this tool is entitled to — those two numbers turn page size from a preference into arithmetic. Then walk the two questions to a strategy, and write the contract that strategy owes: matched, returned and order_by on a capped result, a signed cursor carrying its ordering on a paged one, a basis block on a rollup, a raw handle on a summary. Add the call that reaches whatever the boundary excluded, because a result that admits it is partial and offers no route onward leaves the agent with nothing but an apology. Finally, assert the token count of the p99 fixture in CI, so that the schema change six months from now fails a build instead of an agent.

What that produces is a tool result designed as evidence rather than serialized as a side effect — legible to the model, honest about what it leaves out, and priced. Topic 5.11, on writing tool descriptions, makes the description tell the model how to call the tool; these seven chapters make the result tell it what it is holding. Both halves of that interface are now yours to write, and the next thing to do with them is to open the tool you shipped last quarter and count what it actually sends.