The boundary between two stages is the thing you are designing

A nightly pipeline had run every night for six months. Twelve thousand scanned policy documents go in; stage one extracts entities, stage two writes a short prose summary of them, and stage three runs four regular expressions over that summary to populate a record in the claims database. On a Tuesday in March the provider shipped a point release of the model, and the next morning the on-call engineer found 963 rows missing and a log full of RecordError: no match for effective_date. Eight percent of the night's runs had failed, and the change that caused it was that stage two had started writing "the policy takes effect on 1 April 2026" where it used to write "effective date: 2026-04-01".

That pipeline is one instance of a pattern with a name. Sequential chaining is splitting a task into a fixed series of prompts that run in an order settled in code before any document arrives, each stage taking as its input what the stage before it produced: the entities from stage one are what stage two summarises, and stage two's summary is what stage three parses. It exists for two reasons that are easy to verify on any real task. A single prompt told to read a scanned policy, resolve its dates, classify its risk band and write a database row does all four somewhat worse than four prompts each doing one of them, and it hands back a single block of output, so when the row comes out wrong there is nowhere in particular to look. Splitting the work gives each step a narrow instruction and gives you an intermediate result you can inspect, test and refuse. The shape turns up constantly once it has a name: extract-then-validate over submitted forms, summarise-then-translate, retrieve-then-answer-then-check in a question-answering service, and the nightly document jobs that are many companies' first serious production use of a model. Frameworks name it too, since Google's Agent Development Kit calls the construct a SequentialAgent and in LangGraph it is a graph whose edges are fixed.

Nothing regressed, and the run still broke#

Stage two produced a competent summary before the update and a competent summary after it, and if you had shown both to a claims adjuster they would have said the second one reads slightly better. The failure is not in either stage's output. It is in the fact that nobody had ever written down what stage two was required to produce, so the only thing holding the two stages together was a phrasing habit that the model happened to have, and a habit is not something that can be met or violated. The part of a chain you actually design turns out to be the joints between the prompts rather than the prompts themselves.

The route from a provider's release note to a missing database row is short, and every link in it is checkable. Your repository pins your prompts and your code, and it does not pin the model's weights, which live on the provider's side and are replaced when they ship a release. A language model's output is a sample from a distribution over strings, and retraining moves that distribution, including the parts of it that decide surface form — whether a date is rendered as a label and an ISO value or as a sentence in English. A regular expression is a claim about surface form and nothing else, since it matches characters in an order. So a change that leaves the meaning of stage two's output identical can still leave stage three matching nothing, and the deploy that caused it happened somewhere you have no visibility into. That is the whole mechanism, and it explains the shape of the incident: the two stages were connected by a property nobody had declared, and the property moved.

Two prompts in the same function are still two processes#

Backend engineers already have strong instincts about this shape, they just do not fire when both sides of the boundary live in the same file. If service A returned a paragraph of English and service B ran a regex over it to find a date, you would call that a missing API contract in review and you would not merge it. Nothing changes when A and B become two calls to a model inside one Python function. The channel between them is still a channel, the producer still has freedom the consumer has not accounted for, and the producer is now a system whose exact output distribution shifts under you without a deploy on your side.

The comparison to two services is exact in one direction and misleading in the other, and the direction it misleads is the one that matters. It is exact about the interface: a channel between two components needs a stated contract whether the components are processes, functions or prompts. It breaks on the direction of the surprise. Service B's regex breaks when someone edits service A, which is an event you can see in a diff and a deploy log. A model's phrasing shifts when the provider retrains, when a system prompt is edited three layers away, when temperature moves, or when the input document is unusual enough to pull the response into a different register. All of those are invisible from your repository, so the instinct a reviewer brings from the microservice case — that the risk is bounded by what your own team merges — is the part to discard. The 8% is a good illustration: it was not 100%, because most documents produced phrasing close enough to the old habit, and it was not a clean zero either, so the failure looked intermittent and the first hour of debugging went into infrastructure.

It is worth knowing why prose intermediates became normal, because the people who built pipelines this way were not being careless. In 2023 the reliable way to get a model to emit JSON was to ask nicely and parse defensively, and the failure rate was high enough — trailing commas, markdown fences around the object, an apologetic sentence before the brace — that a lot of teams concluded structured intermediates cost more than they returned. Prose was what the models were good at, so prose went down the pipe and a regex picked it up at the other end. What changed is that constrained decoding and tool-call schemas arrived, and a provider that guarantees the response conforms to a supplied JSON Schema removes the entire argument for prose between two pieces of code. Pipelines built before that shift are still running, and the March incident is what one of them looks like when it finally hits a model update it cannot absorb.

So the boundary between two stages needs to be something a machine can check, and the check needs to run before the consuming stage does. That gives you three things to declare at every joint.

Shape is the field set and the types: what keys exist, which are required, what a value is allowed to be. Constraints are the semantics the shape cannot carry — that a date is ISO 8601, that tier is one of four known strings, that entities is non-empty when document_type is policy. Enforcement is the code that runs the check and refuses to call the next stage when it fails. A schema you do not validate is a comment.

Declaring the joint#

Here is the same three-stage pipeline with the middle joint made explicit. Stage two stops writing prose and starts filling a declared structure, and the structure is defined once and used in two places: to instruct the model and to check what came back.

from datetime import date
from typing import Literal
from pydantic import BaseModel, Field, ValidationError

class PolicyFacts(BaseModel):
    """Contract for the output of stage two. Stage three reads only this."""
    policy_number: str = Field(pattern=r"^[A-Z]{2}-\d{8}$")
    effective_date: date            # serialised as ISO 8601
    expiry_date: date
    tier: Literal["bronze", "silver", "gold", "platinum"]
    insured_names: list[str] = Field(min_length=1, max_length=12)
    annual_premium_cents: int = Field(ge=0)

def stage_two(entities: dict) -> PolicyFacts:
    raw = client.messages.create(
        model=MODEL,
        max_tokens=1024,
        system=(
            "Return a single JSON object matching the supplied schema. "
            "Every field is required. Do not add fields. Do not explain."
        ),
        messages=[{"role": "user", "content": json.dumps(entities)}],
        tools=[{
            "name": "emit_policy_facts",
            "description": "Emit the extracted policy facts.",
            "input_schema": PolicyFacts.model_json_schema(),
        }],
        tool_choice={"type": "tool", "name": "emit_policy_facts"},
    )
    payload = next(b.input for b in raw.content if b.type == "tool_use")
    return PolicyFacts.model_validate(payload)   # raises at the boundary

Read the class as the two declarations it carries. The field names and types are the shape: six keys, each with a type the next stage can rely on, and a date is a date rather than a string that looks like one. The arguments inside Field are the constraints: a policy number matching a two-letter, eight-digit pattern, between one and twelve insured names, a premium greater than or equal to zero. The Literal doing both jobs at once is worth noticing — declaring tier as one of four exact strings is a type in the shape sense and an enumeration constraint in the same breath, which is why a downstream if tier == "gold" can never silently miss a record that said "Gold".

The model_validate call is where the design lives. Before it, stage two's failure is a claims record with a null date discovered by a support ticket three weeks later; after it, stage two's failure is a ValidationError raised on the document that caused it, holding the offending payload, at the moment it was produced.

The provider's schema-constrained tool call makes malformed output rare, and validating anyway is not redundant, because constrained decoding enforces shape while your constraints — the premium being non-negative, the expiry falling after the effective date — are yours to check. That distinction is the second place a reader is likely to hold a comfortable and false belief. Constrained decoding is a real guarantee and it is easy to over-read: the decoder is prevented from emitting tokens that would break the schema, so what comes back really does parse and really does have the declared keys. What is held constant is conformance to the shape. What varies underneath it is everything the shape does not say — a premium of minus one is an integer, an expiry date before the effective date is two valid dates, and a tier of "gold" on a document that never mentioned a tier is a member of the enumeration. A guarantee about form is not evidence about content, and the fields where the two diverge are the fields worth writing constraints for.

The gates are worth seeing as a shape, because a chain drawn without them looks like three boxes and a chain drawn with them looks like what it is.

Each gate is cheap — a few hundred microseconds of validation against a model call measured in seconds — and each one converts an entire class of downstream mystery into a local, attributable error. The ratio between those two durations is why the usual objection to adding checks does not apply here. Validation is ordinary local computation over a payload of a few hundred bytes, and the thing it sits next to is a network round trip to a model that thinks for seconds. Adding a gate therefore changes the stage's latency by a part in ten thousand or so, which is below the noise of the call it guards, so there is no performance argument for skipping it and the only real cost is the work of writing the contract down. The Anthropic cookbook's basic-workflows notebook shows the same structure at its most minimal, with a gate function between links; it is worth reading precisely because the gates are the only thing in it that is not a prompt.

Defining the contract once and using it twice is the detail that keeps this honest over a year of edits. PolicyFacts generates the schema handed to the model and performs the validation on the way back, so there is no state in which the prompt describes one structure and the parser expects another. Pipelines that write the schema out longhand in the prompt and then validate against a separately maintained class drift within a couple of sprints, usually when somebody adds a field to the prompt and forgets the model, at which point the extra field is silently discarded and nobody notices until a product manager asks where it went. The same argument extends to versioning: when a contract has to change, changing it is a commit that touches the producing stage, the consuming stage and the fixtures together, which is a good property, because a contract change that can be made in one place is a contract change that will be made without telling the other side.

Conformance is not correctness#

A validated boundary tells you the payload is well-formed. It does not tell you the payload is true. Stage two can emit effective_date: 2026-04-01 with perfect conformance when the document says 2027, and every gate in the chain will wave it through, because a schema check is a type check and the model's job at that joint is a judgment call. Teams that add schemas after an incident like the one above sometimes come away believing the boundary is now safe, and then spend a quarter puzzled about why the claims team still finds wrong dates. Distinguishing conformance from correctness is what per-stage evaluation is for, and that arrives in chapter 4.

Contracts also cost something, which is why the temptation to skip them is not stupidity. Asking for structured output adds tokens: the 180-token prose summary in the original pipeline became a 240-token JSON object, so the middle stage got about a third more expensive on output — sixty tokens more on a base of a hundred and eighty. It is worth being precise about what that multiple covers, since it is the number someone will quote back at you. It is the output side of one stage of a multi-stage pipeline, so it is not a third added to the pipeline's bill; and it leaves out the schema handed to the model on the input side, which is charged too, at the input rate.

Over-tight schemas make things worse rather than better — a required field that the document genuinely does not contain forces the model to either invent a value or fail the whole record, so Optional[date] with an explicit null is usually a better contract than a required field plus a hopeful prompt. The discipline is to constrain what the consumer actually depends on and leave the rest free.

And prose between stages is not always wrong. It is wrong when the consumer is brittle. If stage three had been another model call reading the summary and deciding a routing category, the March phrasing shift would have cost nothing, because the consumer tolerates paraphrase by construction. The question at any joint is what the consumer can absorb, so:

Notice that the left column alone never decides. Two of the five rows emit prose and are correct. Reading down the middle column instead puts the rows in order of how much paraphrase the consumer can absorb: a regex absorbs none, since it matches characters; code that reads named fields absorbs none of the field names and all of the wording around them; a model consumer absorbs a great deal, because it re-reads for meaning rather than for form; and a human absorbs the most of all. The verdict on each row follows from that tolerance meeting whatever freedom the producer has, which is why the same producer appears on both a broken row and a fine one.

Which raises the question this chapter has been carefully not answering. Raising a ValidationError at a boundary is a decision about where to fail; it is not a decision about what the run does next. Twelve thousand documents are in flight, one of them has just failed its gate, and the process has to do something — retry the stage, fall back to a degraded record, abort the whole batch, or set the document aside for a human. Each of those is defensible and each is wrong in some situations, and picking one at three in the morning is how a pipeline acquires behaviour nobody designed.