Agent frameworks

Read these first

• The agent loop implemented from scratch

What you will know

• Instrument a framework-backed agent and trace one request through durable state, retries, tracing, and tool execution.

  The useful boundary is often hidden in middleware: a retry can duplicate a non-idempotent tool call, while persisted checkpoints alter recovery semantics.

• Rebuild one of those pieces by hand to know its cost

  This exposes serialization, cancellation, replay, and observability work that a demo usually conceals.

• Choose a framework and state what you'd have to give up to leave it

  A credible exit plan names stored-state formats, tracing dependencies, prompt and tool abstractions, and migration or rollback paths.

• Run the same agent loop with and without a framework, then compare latency, failure behavior, and operational work under a stated workload.

  A small loop can win when control and portability matter more than batteries-included operations, but its pager burden remains yours.

• Read a framework's source when it misbehaves

  Framework bugs often live below the public API, where defaults, callback ordering, and version-specific adapters decide the outcome.

Chapters

1. 1

   Durable state, retries and tracing are the actual product

   The loop that calls a model, reads tool calls and appends results is an afternoon's work, which is why 'you don't have to write the loop' is the wrong reason to adopt anything. What a framework sells is operational: persisted execution state, retry policy, spans you did not instrument, and tool plumbing. What it does not sell is responsibility, since the pager, the evals and the incidents stay with you.

   Read chapter
2. 2

   One traced request shows where the framework acts for you

   Follow a single run through checkpoint writes, retry decisions, span creation and tool invocation, and the middleware stops being a diagram. It is also how you explain a customer charged twice when your handler logged one call: a step retried below the level you can see, re-executing a call that was never idempotent. Instrument first, then push idempotency keys down into the tool rather than into the handler.

   Read chapter
3. 3

   One logical call becomes two at five different layers

   'Something retried below me' is a diagnosis, not an address, and the double charge lives at exactly one layer. The candidate set is short enough to hold in your head: the provider SDK retrying a 429 or a 5xx on its own, an HTTP client or gateway retry, the framework's step or middleware retry policy, a queue or scheduler redelivering the whole step, and the loop re-emitting a tool call after a timeout it read as a failure. Each leaves different evidence in logs and spans, and which one fired decides where the idempotency key must be minted so that it stays identical across the duplicate instead of being regenerated with it.

   Read chapter
4. 4

   Rebuilding one piece by hand is how you learn its real price

   Pick the checkpointing, or the tracing, or the cancellation path, and implement it yourself for the same agent. What appears is the work a demo never shows: serialising state that contains model objects and open handles, cancelling a run mid-tool-call, replaying without re-executing effects, and spans that survive a process boundary. You will finish knowing which piece you would never willingly rewrite.

   Read chapter
5. 5

   Framework behaviour is decided below the public API

   When the documentation describes one behaviour and production shows another, the explanation is usually a default you never set, a callback that fires in an order you assumed, or a provider adapter that changed between two patch versions. Reading that code is a normal debugging step, not an escalation. This chapter shows how to find the relevant file quickly and how to pin a version once you know what moved.

   Read chapter
6. 6

   The exit cost is the state format and how far it spread

   Leaving is rarely blocked by the loop; it is blocked by half-finished runs persisted in a format only that engine reads, by traces your dashboards depend on, and by prompt and tool abstractions that have reached every module. A decision record worth having names those three, plus the migration and rollback path for runs in flight. Write it while adopting, when the answers are still cheap.

   Read chapter
7. 7

   A framework is wrong when its execution model fights yours

   'When you want more control' names no situation and can therefore neither win nor lose an argument. These are situations: the agent's control flow does not fit the framework's execution model, so every branch becomes a workaround; you already run a durable execution engine and the framework duplicates its guarantees badly; the deployment target is edge, serverless or an embedded worker that cannot host it; you need token-level control over streaming or over the exact prompt bytes, which the abstraction hides; and the team is one engineer whose capacity is smaller than the framework's upgrade cadence. Each of these can be stated before any benchmark is run, which is what the next chapter is for.

   Read chapter
8. 8

   The same agent, both ways, under one stated workload

   Run the framework build and the hand-written loop against identical traffic and compare p50 and p95 latency, behaviour under a failing tool, and the operational work each demands in a week. Three tools, tight latency targets and a team that already owns the pager is the shape where the small loop wins; long runs, human pauses and many contributors is the shape where it loses. Name the number that decided it.

   Read chapter