A framework is wrong when its execution model fights yours

A design review about whether to adopt an agent framework usually fails for a structural reason rather than a social one. The positions offered are "operations" on one side and "control" on the other, and neither can be checked, so neither can lose. An argument that cannot lose cannot resolve anything, and the meeting ends by whatever mechanism the room uses when reasoning has run out.

What replaces that is a short list of situations stated sharply enough to be wrong. Each of the five below names an observable — a share of code, a count of duplicated guarantees, a bundle size, a hash, an hours-per-month figure — and each of them can be answered by somebody producing the mechanism or the measurement that contradicts it. Three of them are facts about your environment and two are facts about your agent, and none of them is a verdict on any product.

When the control flow does not fit#

Graph and state-machine runtimes schedule work in units: a node runs to completion, its output is merged into state through a reducer, and the next node is selected. That is the entire vocabulary, and it is a good one, because a unit that has completed is a unit that can be checkpointed, retried and resumed — the four capabilities of chapter one all rest on there being a boundary at which a step is finished.

The consequence is that anything an agent needs to do inside a step has no expression at all. Agents whose control flow is not made of completed units fight this at every turn. A voice agent that must abort a model call the instant the user starts speaking, a research agent that spawns eleven parallel branches and kills the nine that stop looking promising after their first result, an agent that rewrites its own earlier messages to compact context mid-run, replacing a long stretch of history with a summary so that the conversation still fits inside the model's input limit — each of these needs to act inside a step, and the runtime's vocabulary begins at the step boundary.

The tell is what the workaround looks like. Compacting context on an append-only message reducer produces something like this, which is a two-line intention wearing forty lines of ceremony.

def compact(state: AgentState) -> dict:
    kept = summarise(state["messages"][:-6]) + state["messages"][-6:]
    # the reducer appends, so the only way to shrink history is to delete by id
    removals = [RemoveMessage(id=m.id) for m in state["messages"][:-6]]
    return {"messages": removals + kept}   # order matters; reversing it drops the summary

Read that against the reducer's rule and the ceremony becomes predictable rather than arbitrary. The reducer's job is to merge a step's output into state, and for a message list it appends, which is what makes the state recoverable from a sequence of appends. A function that wants a shorter list therefore cannot return a shorter list; it can only return more entries, some of which are instructions to remove. The comment about ordering is the same constraint biting twice: the removals and the summary are both appended, in the order given, so putting the summary first has it deleted by the removals that follow it. None of that is a defect in the runtime. It is what an append-only merge implies, encountered by code that wanted a different operation.

Parallel exploration produces the same friction from the other direction. A research agent that fans out to eleven candidate queries and wants to cancel nine of them the moment two return usable results needs cancellation of sibling branches, and most graph runtimes will happily run the branches concurrently while offering no way to stop one from inside another — the merge point waits for all of them, so the run costs eleven model calls and the latency of the slowest. The workaround is a shared cancellation flag in state that each branch checks between steps, which works and turns a scheduling property into application code that every future contributor has to remember to write.

One such workaround is a curiosity. The signal to act on is the ratio: when more than roughly a quarter of the agent code exists to express control flow the runtime does not have a construct for, the abstraction is costing more than it provides, and every new branch will cost the same again. That ratio is countable in a pull request, which is what makes it an argument rather than a preference — someone can disagree with the count, and a disagreement about a count terminates.

When you already own durability#

A team that runs Temporal, Step Functions or a comparable durable execution engine — a system that runs a workflow as ordinary code while persisting the outcome of every step, so that a crashed process resumes at the step it had reached rather than at the beginning — already has checkpointing, retries, timers, cancellation and replay, implemented by people whose entire product that is. Putting an agent framework's version of the same guarantees inside a workflow activity gives you two of everything, and the two do not compose in the direction you would hope.

Nested retry policies multiply, and the multiplication is worth writing out because it is the shape of every layered-retry surprise. The inner policy counts attempts of the node, so a failure that persists uses all of them. The outer policy counts attempts of the activity, and one activity attempt contains a full inner sequence. Attempts of the tool underneath are therefore the product rather than the sum: a workflow activity configured with three attempts wrapping a node configured with three attempts is nine executions of the tool underneath, and the outer engine's view of the run has one activity that took a long time. Neither policy is wrong on its own terms, and neither author knew about the other, which is why the number nine appears nowhere in either configuration.

Two checkpointers means state in two stores with no transaction between them, so a crash can leave the workflow believing step 5 completed while the agent's checkpoint says step 4. The measurable version of this situation is the count of duplicated guarantees. If the framework's list of operational features is a subset of what the existing engine already provides, the framework is contributing tool plumbing and prompt ergonomics, which is a library-sized contribution rather than a runtime-sized one. The sensible arrangement is to keep the durable engine as the only scheduler and to run a small loop inside each activity, with retries set to one attempt at the inner layer and the outer engine owning recovery — which collapses the product above back to the outer count.

When the deployment target cannot host it#

Some deployment targets are simply not compatible, and this is the least arguable of the five because it is a build error rather than a judgement. An edge runtime with no filesystem, no threads and a hard limit of a few megabytes of bundled code cannot host a dependency closure that unpacks to 180 MB. A serverless function with a 250 MB unpacked size limit can host it, but the import graph adds around 2.1 seconds to a cold start, which on a workload with a 6% cold-start rate and a 900 ms p95 target is disqualifying.

That last judgement is worth deriving rather than accepting, because the same reasoning decides other cases. A p95 target is a promise about the slowest one in twenty requests. A cold start happening on six percent of requests is therefore happening more often than one request in twenty, which puts cold starts inside the p95 window by arithmetic alone. And a cold start that costs about two seconds is more than twice the entire budget, so the p95 is missed by the import graph before any work is done. Had the cold-start rate been well under one in twenty, the same two seconds would have sat in the tail beyond p95 and the target would have survived — which is why the rate and the target have to be read together and neither is disqualifying on its own.

An embedded worker on a customer's hardware, a desktop application shipping a Python runtime, an agent running inside a database extension — all of these have constraints the framework was never designed against. The number to produce here is small and decisive: unpacked size of the dependency closure, cold-start delta measured with the import in place and removed, and resident memory at idle. Three measurements, half a day, and the argument is over in whichever direction the numbers point.

When the bytes matter#

Prompt caching — the provider keeping its processed form of a prefix you send on every request, so that a later request beginning with exactly the same bytes is billed at a fraction of the input price and starts generating sooner — pays for byte-identical prefixes, and an abstraction that assembles the prompt on your behalf is under no obligation to assemble it the same way twice. A framework that serialises tool definitions from a dictionary whose order depends on registration, or that injects a rendering timestamp into a system prompt template, or that reorders message metadata after an upgrade, will break the cache prefix without changing anything you can see in your own source.

# the prefix is 6,400 tokens: system prompt + 6 tool schemas, cached with a 5-minute TTL
# a stable build produces the same first 6,400 tokens on every request in a run
prefix_hash = hashlib.sha256(json.dumps(payload["tools"], sort_keys=False).encode()).hexdigest()
# observed across 500 consecutive requests: 14 distinct hashes

Note what that measurement does and does not require. It does not require knowing why the bytes differ, or which library produced them; hashing the serialised tool block and counting distinct values over a run of requests answers "is the prefix stable" directly, and a stable build produces one hash. Fourteen is not a small failure of stability. It means the cache is being written far more often than it is read, since each new prefix has to be stored before anything can hit it.

Fourteen distinct prefixes where there should be one took the cache hit rate from 88% to 12%. The bill follows from four figures and one price. On 4,000 runs a day at 11 steps each with a 6,400-token cached prefix, the prefix alone is forty-odd thousand requests carrying a bit over a quarter of a billion input tokens a day, before anything else in the prompt is counted. At a list price of $3 per million input tokens, with cached reads at a tenth of that, a hit rate near nine in ten and a hit rate near one in ten price identical behaviour very differently: the difference is roughly $180 a day against roughly $750 a day. That gap is what one unstable serialisation costs.

Token-level streaming has the same character: if you need to detect a tool call from the first tokens of a stream and start a speculative fetch before the model has finished emitting arguments, you need the raw event stream, and every layer that normalises it into tidy objects removes the thing you were reaching for. The pattern across both is that a framework's value is deciding things for you, and these are the cases where the decision is the product.

When the team is smaller than the release cadence#

An actively developed framework in this space ships something like 40 releases in six months, of which three or four change behaviour you depend on. Absorbing that is roughly four to six hours a month of reading changelogs, running the upgrade, and fixing whatever moved — negligible on a team of eight and a real fraction of the available capacity for one engineer maintaining an agent alongside four other services.

The alternative, staying pinned, is the tempting escape, and it is worth seeing why that defence works right up until it does not. Pinning removes the recurring cost completely, which is real. What it does not remove is the reason upgrades happen, since a security advisory or a provider API change is not something your pin has any authority over. When one arrives, the work is not the upgrade you skipped but every upgrade you skipped, compounded: eleven versions at once, with their interactions, which is a week rather than an afternoon. The hours per month did not go away; they were accumulated into a lump that arrives on somebody else's schedule.

The middle column is what makes the table usable. Every entry in it is a measurement somebody on your team can take this week, without a benchmark harness and without the second build, which is what turns each row into a claim that can be defeated in the review rather than after it. The right-hand column matters for the same reason: a situation that names no alternative is a complaint, and each of these names the arrangement that replaces the one being rejected.

The version of the argument that loses#

"When you want more control" fails as a position because it describes a preference everybody holds and commits to nothing. It cannot be checked, so it cannot be refuted, and it also cannot be acted on. Each of the five above names an observable, and each of them can be shown to be false: someone can demonstrate that the runtime has a first-class construct for interruption, that the two engines' retries compose because the inner one is configured to a single attempt, that the closure fits in the bundle, that the prefix is stable across 500 requests, or that the upgrade burden is an hour a month. Losing an argument that way is a good outcome, because the team has learned something either way.

Three other objections come up in these reviews and none of them belongs on the list, which is worth saying because they consume time. "Our agent is simple" is a statement about today and predicts nothing about the version with human approval steps that ships in November; simplicity is a reason to defer the decision cheaply rather than a reason to decide. "We are worried about lock-in" is real but already priced elsewhere, in the state format, the tracing dependency and the spread, and an argument that names lock-in without naming those three is asking to be reassured rather than to be answered. "The abstraction is leaky" describes every abstraction and only becomes an argument when the leak has a location and a cost, which is what the byte-level and control-flow situations above supply.

The five are also not a general verdict on frameworks. Three of them are properties of your environment rather than of the framework — the durable engine you already run, the target you deploy to, the size of your team — and the same framework could be the right answer for a neighbouring team with none of those constraints. Two of them, the control-flow mismatch and the byte-level control, are specific to a class of agent, and the same product may be fine for the batch pipeline next door. The mistake that mirrors "we want more control" is treating any of these as a permanent conclusion rather than as a fact about a system at a moment: a team that acquires a durable execution engine next quarter acquires the second situation with it, and a team that drops strict schemas may lose the fourth.

What none of the five settle is the ordinary case, where the control flow fits, you have no durable engine, the deployment target is a normal container, the prefix is stable and the team is five people. That case is common, and in it every measurement in the middle column comes back saying nothing is wrong, which is a genuine answer and not a failure of the method.

In that case the choice is genuinely open, opinions are cheap, and the only thing that resolves it is running the same agent both ways under traffic you have described in advance and looking at what comes out. Describing the traffic first is the part that does the work, and it is also the part that gets skipped.