The cheapest component is the one you cannot reconstruct later

The internal tool takes a research analyst's question, sends one prompt template to one provider, and returns a draft paragraph that the analyst reads, edits, and pastes into a report. It serves about 200 requests a day across eleven analysts. The proposal on the table is six weeks of work to build it properly: a gateway, an orchestration service, a vector store for the internal document corpus, Redis for session state, the evaluation plane, and a judge scoring every response.

Six weeks is the smallest cost in that list. The vector store is $70 a month at the managed tier and an indexing job somebody has to own, plus a class of incident where the index is stale and nobody notices for a week. Redis adds a dependency that the orchestrator either treats as optional, in which case it does nothing useful, or treats as required, in which case the tool's availability becomes the product of two availabilities. The gateway adds a hop worth 8–15 ms and a second deployable. The judge at 200 requests a day is cheap in dollars, roughly $1.30 a day, and expensive in the way that matters: it produces a daily mean over 200 mixed requests, a sample size at which normal variation swamps everything, so it will generate alerts that get muted within a fortnight.

What the tool should keep is one table and about thirty lines of code.

def log_call(req_id: str, template: str, version: str, variables: dict,
             model: str, params: dict, completion: str,
             usage: Usage, latency_ms: int) -> None:
    db.execute(
        "INSERT INTO call_log (request_id, occurred_at, prompt_template, "
        "prompt_version, prompt_variables, model, params, completion, "
        "input_tokens, output_tokens, latency_ms) "
        "VALUES (%s, now(), %s, %s, %s, %s, %s, %s, %s, %s, %s)",
        (req_id, template, version, Json(variables), model, Json(params),
         completion, usage.input_tokens, usage.output_tokens, latency_ms),
    )

That is the one component whose absence cannot be repaired after the fact. If in four months an analyst says the drafts got worse in March, a log with prompt version, model snapshot and completion answers the question in an afternoon, and no log means the question is unanswerable no matter what you build in response to it. Everything else on the six-week list can be added later at roughly the same cost as adding it now.

The log needs two decisions made about it even at this size, and both take an afternoon. Retention is ninety days, which covers the interval between a quality drift starting and an analyst mentioning it, after which rows are dropped by partition. And the completion column holds internal research text, so the table lives in the same database as the rest of the tool's data with the same access controls rather than being shipped to a logging vendor where it becomes a separate compliance conversation. Neither decision is interesting, and both are much cheaper to make now than to retrofit onto a table that has been accumulating for a year.

Volume is the wrong variable#

The team defending the six weeks makes a reasonable-sounding move: they agree that 200 requests a day would not justify it, and argue that their real case — a support-triage feature at 40,000 requests a day, one prompt, one provider, no retrieval, output written into a ticket field that an agent reads before acting — obviously does.

Volume changes the rate at which a defect accrues cost, and it changes nothing about whether the defect is detectable or what it costs per occurrence. A tool at 200 requests a day whose output is sent to a regulator without human review needs more machinery than a tool at 40,000 whose output is a suggestion in an internal search box. The variables that decide are what happens if an output is wrong, how often the system's behaviour can change underneath you, and who reads the output before it acts on the world.

At 40,000 a day the triage feature scores differently on all three than the analyst tool does. The output is consumed by an agent skimming a pre-filled field, which is a materially weaker gate than an analyst who commissioned a paragraph and reads it critically — reviewers approve plausible text, and a pre-filled field arrives with the presumption of correctness. The blast radius is a customer-facing routing decision rather than a draft an author will rewrite anyway. And a feature at that volume holding a provider API key directly is a credential and budget question: one leaked key or one prompt-loop bug is a five-figure surprise, and the control for both lives in the gateway.

Read down the two middle columns and the six-week plan resolves into roughly two weeks: the log, a gateway route with a spend cap, and a stratified sampler feeding a slice report cut by queue and ticket category. What the team wanted to build was the vector store and the judge, and those are the two rows that are cut in both columns.

The answer changes when any of the deciding variables move. If the ticket field starts auto-routing without an agent reading it, the human gate disappears and the quality signal becomes load-bearing rather than diagnostic, which justifies both a stronger sample and an offline evaluation set gating prompt changes. If the feature grows a second prompt and a second provider, provider drift and prompt drift become real and the version stamping earns its place several times over. If retrieval is added, the vector store arrives with its own index-version field on the record, because a retrieval regression is the failure mode chapter 5 built a table around.

It is worth being explicit about the thresholds, because "it depends" is not a design input. Retrieval earns a vector store when the corpus is too large to fit in a prompt and changes often enough that a nightly rebuild is meaningful; a corpus of forty documents that changes monthly is better served by putting the relevant one in the prompt and skipping the store entirely. A gateway earns its hop when more than one feature holds a provider credential, or when a single feature's spend is large enough that an unbounded loop would be noticed by finance before it is noticed by engineering — in practice somewhere around a few hundred dollars a day. A quality signal earns its cost when nobody with domain judgement reads the output before it acts, or when the output is read by someone who did not ask for it. An offline evaluation set earns its cost when the prompt changes more than about once a month, because below that rate the set goes stale between uses and nobody trusts it.

The analyst tool scores below every one of those thresholds, and the six-week proposal is not a mistake of judgement so much as a mistake of anchoring: the reference architecture is a picture of a system under all four pressures at once, and no real system is under all four at the start. The correct reading of a reference architecture is as a list of decisions with a default answer of "not yet", each with a written condition that flips it.

The flag that is ready for nothing#

Having agreed to cut the vector store, the team's next proposal is to add it back behind a feature flag, so that it is ready. That is worse than either building it or not building it, and the reason is what a dormant dependency actually costs.

A provisioned store is provisioned whether or not a query reaches it. It has a bill, a version that goes end-of-life, a credential that rotates, a network path that a security review will ask about, and a maintenance window that will land on someone's on-call. The flag being off removes exactly one of those costs, the query cost, and preserves the rest. Meanwhile nothing exercises the code path, so it decays: the client library's API changes across two upgrades, the schema drifts from what the ingestion job produces, the timeout is never tuned because no request ever waits on it. When the flag is finally flipped, it is flipped by someone who has never seen the path work, usually during a push to ship retrieval quickly, and it behaves like a first deploy with none of the care a first deploy gets.

What is worth accepting instead is the part that is genuinely cheap to keep ready, which is the seam. Topic 2.18's port pattern applies directly: define a Retriever protocol with the one method the orchestrator would call, implement it as a null retriever that returns no chunks, and leave the retrieved and index_version fields on the call record nullable. Nothing is provisioned, nothing has a bill, and the day retrieval is real the change is one adapter and one wiring line. A design note in the repository saying which store was evaluated and why it was deferred costs an hour and is the actual artifact people wanted when they said "ready".

The same reasoning applies in the other direction, and it is the honest limitation of everything in this topic. Deferring a component is only cheap when its absence does not silently change the shape of the data you keep. Deferring the vector store is cheap. Deferring the per-call log is not, because the requests served in the meantime are gone, and no later decision recovers them.

Drawing yours#

The output of this topic is a diagram of your own system, and it is worth being specific about what makes one useful rather than decorative. It has the five layers from chapter 1 with the deployment boundaries marked separately from the ownership boundaries, so a module boundary inside one service is visible as a boundary. It has the synchronous line from chapter 2 drawn explicitly, with every arrow crossing it labelled by what depends on the result. It names, at each handoff, who owns retry, who owns client-visible continuity, who owns state mutation and who owns the usage record — the four operations chapter 3 and chapter 4 showed crossing boundaries unowned. It carries the version-stamped record from chapters 5 and 6 as a real box with a real table behind it. And it has a short list underneath of what was deliberately not built, with the condition that would change each decision.

That last list is the part that gets left off, and it is the part that saves the most time. Without it, every new engineer proposes the vector store again, and every incident review proposes the judge on 100% of traffic again, and the reasoning has to be reconstructed from memory by whoever happens to be in the room. With it, the architecture review that opened this topic takes an hour.