Topology is a hypothesis about the shape of the work

The service under review summarises support tickets for the weekly account-health digest: it pulls the message thread from the ticketing API, extracts the entities that the digest needs — customer, product area, blocking status, the identifiers of any linked incidents — and writes three sentences of prose. It has run those three operations, in that order, on every one of the roughly 2,600 tickets it processed last quarter. A staff engineer wanted to replace it with an orchestrator that reads each ticket first, decides what work that ticket needs, and dispatches subtasks accordingly. The reason given was that tickets vary enormously, which is true. A ticket about a failed SSO migration and a ticket asking where the invoice PDF went have almost nothing in common as documents.

There is a decision to make here because a model call costs money, takes seconds, and sometimes returns something wrong, so every call after the first has to be earned by the work it does. A badly arranged system rarely looks broken, which is what makes the choice worth attention: it looks like a system that costs three times what it needs to, fails in ways nobody has written a runbook for, and scores about the same as the cheaper design on every metric anyone is currently watching. You meet the decision in a design review before any code exists, in a pull request that replaces three function calls with a planner, and in the framework documentation you read while deciding, which lays the shapes out as a catalogue of patterns without telling you which one your work needs. A catalogue cannot answer that on your behalf, since which shape fits is a property of the work being done, so the way to settle it is to describe the work precisely enough that one shape follows from the description.

What the word topology is a statement about#

Whether the difference between those two tickets is the kind that matters depends on what the proposal would actually change, which is the service's architecture and almost nothing else about it. A feature like this one is a small number of model calls with ordinary code in between, and its architecture is the arrangement of those calls: how many of them there are, what order they run in, and who decides that order — the code, which fixes it in advance, or the model, which picks the next step while the run is in flight. The usual name for that arrangement is the system's topology, borrowed from networks, where the same word describes whether machines are wired in a line, a ring or a star.

That borrowing is a picture rather than a fact, and it is worth saying where the picture stops being true. A network's topology can be surveyed: the cables exist, and a diagram of them is right or wrong about the building. The wires between model calls are not all like that. Some of them are written in your repository and are exactly as surveyable as a cable, and some of them are produced during the run by a model deciding what to do next, so a diagram of those is a claim about what most runs will look like rather than a description of what is installed. That is the sense in which a topology is a hypothesis: the parts of it drawn by the code are facts, and the parts drawn by the model are predictions about the distribution of your traffic.

Three topologies cover most of what gets built. A chain runs a fixed sequence of prompts with code holding the edges between them. A single tool-using loop hands one model a list of tools and lets it choose the next call each turn until it produces an answer or hits an iteration cap. An orchestrator with workers, also called a fan-out, has one call enumerate the pieces of work, several calls carry them out in parallel, and a final call merge what comes back. The summariser under review is a chain, and the staff engineer is proposing an orchestrator.

The step from "a model call costs something" to "the arrangement is worth arguing about" is short enough to walk in full, and walking it is what makes the rest of the chapter checkable rather than a matter of preference. A run's bill is the number of calls it makes multiplied by what each call costs, and its exposure to a wrong answer works the same way: each call is an independent opportunity to return something wrong, so a design with three calls has three such opportunities where a design with one has one. Neither factor is a statement about how hard the task is. Both are statements about how many calls the arrangement produces. So the arrangement is exactly the thing that sets the multiplier on cost and on failure surface, and a change to it that does not change what the system can do is a change that raises both multipliers for nothing.

The two tickets have everything in common as jobs. Both are pulled with the same API call, run through the same extraction, and rendered by the same summarisation prompt. The variation lives in the content that flows through the pipeline, and the pipeline's shape is decided by the sequence of operations, which does not vary at all. That distinction is the whole of this chapter, and it is worth stating in the form you would use in the review: what varies here is the input, not the step list, and the step list is what topology is a statement about.

The claim has to be falsifiable, or it is taste#

Saying "the sequence is fixed" is a hypothesis about the distribution of production traffic, and like any hypothesis it needs a stated condition under which you would abandon it. For the summariser, the condition is concrete: an evaluation slice containing a class of ticket that needs a step the pipeline does not have. If the digest starts covering enterprise accounts whose tickets reference external status pages, and getting a usable summary requires fetching that page — for some tickets and not others — then the step list is no longer fixed, and the argument for a fixed pipeline has been falsified by data rather than won or lost by seniority.

Here is the pipeline as it exists, which is the artefact the argument is actually about:

def summarise(ticket_id: str) -> Digest:
    thread = ticketing.fetch_thread(ticket_id)          # HTTP, no model
    entities = extract_entities(thread)                 # model call 1
    summary = write_summary(thread, entities)           # model call 2
    return Digest(ticket_id=ticket_id, entities=entities, text=summary)

Two model calls, one HTTP call, no branch. Across 2,600 tickets it produced a mean of 2.0 model calls per ticket with zero variance, because there is nothing in the code that could produce variance. That last clause is worth unpacking, because zero variance is a strong statement and it is being made from the source rather than from the measurement. Variance measures how far individual runs sit from the mean, so it is zero exactly when every run produces the same count. The count here is fixed by the text of the function: there is no branch, no loop and no model choosing a next step, so the number of model calls is a constant function of the input, and a constant has nothing to vary. The measurement across 2,600 tickets — five thousand two hundred model calls in total — confirms the reading of the code rather than discovering it.

What that closure licenses is narrow, and the residual matters as much as the result. Zero variance in the call count says nothing about variance in tokens, in latency or in quality: a thread ten times longer costs roughly ten times the input tokens through the same two calls, and the extraction can be excellent on one ticket and wrong on the next. What it does license is the cost side of the comparison. If the call count is a constant, the per-ticket bill has one term you never have to sample for, which is the property that makes a chain quotable in a budget before it is built.

The proposed orchestrator would produce somewhere between three and a dozen calls per ticket, depending on how the planner felt about each one, and the first question to ask of that spread is which tickets it helps. If the answer is that it does not help any of them and the plans all come back saying "extract, then summarise", you have paid a planner call and a round trip for a hard-coded list. Notice that the spread itself is the tell, and not the size of the maximum. A design whose call count varies with the input has made the count a function of a model's judgement, and any number you quote for it is an average over a traffic mix that can move without a deploy.

Three shapes, and what each one implies#

Most agentic jobs, once you strip the domain language off them, fall into one of three shapes, and the shape is determined by a single question: at the moment the run begins, what do you know about the steps?

The first shape is a fixed sequence: the steps and their order are known before the run, for nearly every input. The summariser is this. So is a document-ingestion job that chunks, embeds, and upserts. The topology this implies is a chain — a prompt pipeline where each stage's output is the next stage's input, with ordinary code holding the edges.

The second shape is step-chosen-from-result: you know the set of operations available, but which one comes next depends on what the last one returned. A debugging agent that reads a stack trace, decides on that basis whether to grep the codebase or query the deploy log, then decides on that basis what to do next, is this shape. The topology implied is a single tool-using loop: one model, a tool list, an iteration cap. The steps cannot be written down in advance because part of the list is produced during the run, which is the boundary topic 10.2 drew.

The third shape is independent enumerable subtasks: at the start of the run you can list the pieces of work, and each piece can be completed without reading what the others produced. A competitive analysis over eleven named vendors is this shape — eleven identical research jobs, no cross-talk, one merge at the end. The topology implied is a fan-out: an orchestrator that enumerates, workers that run in parallel, and an aggregation step.

The three questions behind those shapes are ordered, and the order is what makes them a test rather than a menu. Ask first whether the step list is known before the run. If it is not, no arrangement of parallel workers can help, because you cannot dispatch work you cannot name yet, which leaves the loop. If the list is known, ask whether any item needs another item's output. If one does, the dependent items must run in some order, and the fan-out shrinks to whatever is left over. Only when the list is known and nothing in it reads anything else in it does a fan-out do work that a sequence could not.

The reason to name the shape out loud is that each one has a different tell, and the tells are checkable against a sample of real inputs rather than against intuition, or against the system's traces once something is running — the trace of a run being the ordered record of every model call, tool call and result that the run produced.

Read that table down its last column rather than across its rows. The recognition column tells you which shape a sample is consistent with, which is the weaker statement, since a small sample is consistent with several. The falsification column tells you what a future sample would have to contain to overturn the match, and those conditions are not symmetrical: a chain is falsified by one class of input needing an extra step, while a loop is falsified by traces in which the model's choice never varies. The first is a claim about inputs you have not seen and the second is a claim about runs you have already recorded, which is why a loop's shape claim is cheap to check afterwards and a chain's is not.

A job can be a composite — a chain whose middle node is a loop, an orchestrator whose workers are each a chain — and that composition is normal. What is not normal is a design whose shape nobody can name, because that usually means the shape was chosen from a diagram someone admired rather than from the work.

The cap looks like a defence and bounds the wrong quantity#

The most common way to get this wrong is to read the iteration cap as the thing that makes Design B safe. The reasoning is tempting and half correct: a cap of eight means no invoice can cost more than eight calls, so the worst case is bounded, so the loop's freedom has been paid for. What the cap bounds is the call count. What varies is whether the run returns an answer at all, and the cap does nothing to that quantity — it is the mechanism that produces the undefined result, since the run that hits it stops in the middle of its work by construction. A quantity held constant while the dangerous property varies underneath it is the shape most false reassurance in this area takes, and it is worth checking for by naming the two quantities separately: what does this bound, and what am I actually worried about?

The requirement change that flips the invoice decision is equally concrete, and it is not "invoices are complicated". It is: the job must handle multi-page invoices where line items reference attachments, so that after validation you may need to fetch an attachment and re-extract — with the number of round trips determined by what validation found. At that point the operation set is open, the order depends on results, and the shape has become step-chosen-from-result. The topology should follow it, not lead it.

Shape is not difficulty either. A fixed sequence can be a very hard job — the summariser's entity extraction is the part that fails, and no amount of orchestration would help, because the failure is in one prompt against one messy thread. Teams sometimes reason from "this task is important and subtle" to "it needs a sophisticated topology", and those two properties are unrelated. The sophistication that helps a hard extraction is a better prompt, a better model, a retrieval step, or an eval set that finds the failing class.

Shape is also not the question topic 10.2 answered. That one decides whether the model holds control flow at all; this one applies afterwards, among designs that all involve a model doing something, and asks which arrangement matches the work. A fixed sequence with a model call in each stage is still a workflow by 10.2's criterion, and calling it a chain here is a statement about its shape rather than a re-litigation of whether it is an agent.

And naming a shape is not evidence that you named it correctly. Everything in this chapter is a hypothesis generated from a sample of inputs and an argument about data dependencies, which is a good way to generate hypotheses and a poor way to confirm them. The staff engineer proposing the orchestrator might turn out to be right, if the ticket mix shifts or if some class of ticket genuinely needs a step the pipeline lacks.

What settles it is a measurement on an evaluation set that contains that class, and at the moment the design review happened, no such set existed for either side to point at. That absence is the ordinary condition of a new project, and it is what the next chapter is about: what to build in the week before you have any numbers, given that whatever you build will be the thing your first numbers describe.