Choosing an architecture

Read these first

• Workflows vs agents

What you will know

• Match a task shape to a topology and defend the match

  A planner can add latency and failure surfaces without improving completion quality when the task does not require decomposition or exploration.

• Start from the simplest thing that passes evaluation

  A single loop is usually easier to operate than a coordinated topology whose extra calls and state introduce new failure modes.

• Write the question that would decide between two candidate architectures before choosing one

  Useful discriminators include whether work can run in parallel, needs shared state, or has a bounded horizon.

• Compare two candidate topologies on quality, latency, and cost using the same evaluation set

  Hold prompts, models, tools, and task mix constant so topology—not tuning—accounts for the result.

• Change your mind about a topology based on a measurement

  A small quality gain may not justify doubled calls, higher tail latency, or a larger operational blast radius.

Chapters

1. 1

   Topology is a hypothesis about the shape of the work

   Tickets vary enormously in content; that is not the same as varying in the sequence of operations, and only the second decides topology. This chapter separates three task shapes — a fixed sequence known in advance, steps chosen from the last step's result, and independent subtasks enumerable up front — and shows which pattern each one implies. You leave able to name the shape of a job and defend the match.

   Read chapter
2. 2

   The simplest design that passes evaluation is the one to build first

   Before any evaluation exists, the single loop is the only design whose failures you can attribute cleanly. Build it this week alongside the smallest evaluation set you can write, because its numbers are what later let you say whether a richer topology earns its extra calls. Coordinated designs add calls, state and failure modes that have to be paid for by something you have actually measured.

   Read chapter
3. 3

   The question that decides between two architectures, written before you choose

   Useful discriminators are narrow: can the subtasks be enumerated before any of them runs, does each one need to read what another produced, is the horizon bounded? Here you learn to write that question down before picking, and to state the concrete requirement change that would flip the answer — the invoice job whose three operations have a fixed order flips only when the order stops being fixed.

   Read chapter
4. 4

   Move one thing at a time or the comparison means nothing

   A three-stage chain beat a single prompt by six points — on a larger model, with a rewritten system prompt. Nothing about topology was learned. This chapter sets up the comparison that does teach something: same evaluation set, same model, same prompts, same tools and task mix, so shape is the only thing that moved. It also covers the cheapest honest re-run when the old design has already been deleted.

   Read chapter
5. 5

   Blast radius is partial failures, retry fan-out and lost attribution

   'Operational blast radius' stays a slogan until you can itemise it. A fan-out has to decide what the aggregation step does when one worker fails or comes back empty; retries multiply across branches rather than happening once; run state has to be recoverable when a coordinated execution dies halfway through; an iteration cap has to define what a capped run hands back to the caller; and when three calls produced one answer, no single trace tells you which call was wrong. That list is what a richer topology charges you every day it serves traffic, and it is what a one-point quality gain is being asked to pay for.

   Read chapter
6. 6

   A one-point gain that costs 2.4x the calls is a losing trade

   Ninety-three against ninety-two is not a reason to ship an orchestrator when it doubles and then some the model calls and pushes p95 from four seconds to 7.6. Learn to put quality, call count, tail latency and operational blast radius side by side and decide out loud — including the uncomfortable case where an honest re-run shrinks a five-point win to one point on a design already serving traffic.

   Read chapter
7. 7

   Traces tell you when the shape you assumed was wrong

   Six weeks in, 70% of production plans contain exactly one step and quality is unchanged: most of your traffic never had the shape that justified the split. Read the traces for that signal, route the degenerate classes back through the fused loop, and keep the elaborate path for the classes that genuinely decompose, measuring both on the same set. It also settles the case for standardising every feature on one topology, which is a real argument about operations — one trace format, one retry story, one runbook, no architecture debate per feature — bought at the price of making fan-out machinery the floor for work that never needed it. Put the uniformity in the tooling instead: topology-agnostic instrumentation and a short approved menu of shapes.

   Read chapter