Prototype the assumption whose failure invalidates the rest

The delivery-note project had a good first month. By the end of week three the team demoed a clerk-facing review screen with a document viewer on the left, extracted quantities on the right, keyboard navigation between fields, and a confidence highlight that turned amber below 0.8. It ran against a fixture file of twelve hand-written JSON records. Everyone in the sprint review could see it working, the sponsor asked whether the amber threshold was configurable, and the team went into week four feeling ahead of schedule.

In week five somebody finally ran forty scanned proof-of-delivery notes through the extraction prompt. These are the paper slips a driver gets signed at a loading bay: pre-printed grid, quantities written in by hand in ballpoint, often annotated where a pallet was short, photographed or faxed and arriving as 200-dpi scans with the top third of the page at an angle. Exact match on the delivered-quantity column came back at 41%. Not 41% of documents — 41% of line items, on the one field the entire workflow existed to capture. Three weeks of review-screen work were now a review screen for a stream of values that a clerk would have to retype more than half the time, which is the job they already had.

The team had not made a mistake in how they built the UI. They had made a mistake about which of three questions their month was buying an answer to.

The plan had three unknowns and they were not equal#

Written out, the proposal rested on three technical assumptions: that a clerk-facing review UI could be built (it could, the team had built a dozen), that the fifteen-year-old ERP would accept a goods-receipt posting through its SOAP interface, the XML-over-HTTP call protocol that enterprise systems of that vintage expose in place of a JSON API (probably, undocumented, nobody had tried), and that a model could read hand-written quantities off 200-dpi scans accurately enough that reviewing beat keying (unknown, and the whole point of the system).

The useful question about each assumption is what survives if it turns out false. If the UI is harder than expected, the extraction still works, the integration still works, and the fallback is a spreadsheet or a plainer screen that ships two weeks later. If the SOAP integration is impossible, extraction and review still hold their value; the output lands in a CSV that someone imports, which is worse but is a business the company can run. If the handwriting extraction fails, there is nothing to review, nothing to post, and no version of the project that survives — the UI is a viewer for values that do not exist, and the integration is a pipe with nothing in it.

The diagram is not a build order; it is a map of what depends on what for its value. Both the UI and the integration take their input from extraction, so both are conditional on it, while extraction is conditional on nothing in the plan. That structure is what makes the ordering obvious once it is drawn, and it is invisible in a backlog, where all three appear as peer epics with story points.

Why the easy part attracts the budget#

Building the UI first is not laziness. It is the rational response to three real pressures, and understanding them is what stops the same choice being made again on the next project.

The first is estimability. The team could size the review screen accurately because they had built screens before, and a task you can estimate is a task you can commit to in a sprint. The extraction work had no honest estimate, because the answer to "how long to get handwriting extraction working" is either two days or never, depending on a fact nobody had checked. Planning processes systematically prefer the estimable task, which means they systematically defer the one that carries the information.

The second is visible progress. A review screen demos. A script that prints qty_exact_match: 0.41 does not demo, and reporting it in week one feels like reporting failure, whereas the same number in week five arrives as a crisis with three weeks of sunk work attached to argue on its behalf.

The third is the ordering that most delivery methods actually teach. Barry Boehm's spiral model, from 1988, put risk resolution at the centre of each iteration: identify the highest risks, build the smallest thing that resolves them, then plan the next loop. What propagated into mainstream agile practice was a different ordering, highest business value first, which is excellent advice for a system whose feasibility is settled and misleading advice for one whose feasibility is the open question. Value-ordering says build the review screen, because clerks want it. Risk-ordering says the review screen has no value at all in the branch where extraction fails, so its expected value is conditional on an unmeasured probability.

Ranking by blast radius, uncertainty and cost to test#

Three properties decide where prototype days go. Blast radius is the share of the plan that becomes worthless if the assumption is false. Uncertainty is your honest prior that it is false, meaning the probability you would put on failure today, before spending anything to check, and honest matters more than precise — "we have never tried this" and "we did this last year on cleaner scans" are different numbers. Cost to test is what it takes to move that prior meaningfully, which is often far less than building the component.

The bottom row is the important one. The UI is not on this list to be scheduled; it is on the list to be explicitly excluded from the prototype, because the team's prior that they can build a review screen is around 2% failure, and spending days to move a 2% prior buys almost no information. The scan-resolution row shows the other end: a genuine dependency, cheap to check, resolved in an hour with a query against the mail archive rather than any code.

Ordering by blast radius alone is not enough, which is why cost to test sits in the table. Two assumptions with identical blast radius but one costing half a day and the other costing three weeks to check should not be tested in the same week; you buy the cheap information first, because it may change what you do with the expensive slot.

What the cheap test looks like#

The handwriting test that should have run in week one is smaller than the UI's first component. Thirty scans pulled from last month's mailbox, one field per line item, labels typed into a CSV by whoever knows what the correct quantities were, and a script:

import csv, json
from anthropic import Anthropic

client = Anthropic()
FIELDS = ["delivery_note_no", "line_qty", "short_flag"]

def extract(pdf_bytes: bytes) -> dict:
    msg = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        messages=[{"role": "user", "content": [
            {"type": "document",
             "source": {"type": "base64", "media_type": "application/pdf",
                        "data": pdf_bytes}},
            {"type": "text", "text":
             "Return JSON with delivery_note_no, and for each line: line_qty "
             "(integer as written by hand) and short_flag (true if annotated "
             "as short-delivered). Use null for anything you cannot read."},
        ]}],
    )
    return json.loads(msg.content[0].text)

hits = {f: 0 for f in FIELDS}
rows = list(csv.DictReader(open("labels.csv")))
for row in rows:
    got = extract(open(row["path"], "rb").read())
    for f in FIELDS:
        hits[f] += int(str(got.get(f)) == row[f])

for f in FIELDS:
    print(f"{f}: {hits[f]}/{len(rows)} = {hits[f] / len(rows):.0%}")

Half a day of labelling, forty minutes of runtime, about $0.40 of model spend at current per-page pricing for scanned PDFs. The output is three percentages, and the week-one version of the project either continues with a known problem or stops before the review screen exists.

A screen this small can only detect disasters. Thirty cases can tell you the difference between 41% and 92% with some confidence; it cannot tell you the difference between 84% and 88%, which is exactly the region where the kill criterion from the previous chapter lives. That distinction — a triage instrument that separates catastrophic from plausible, versus a measurement that can resolve a threshold — is where this chapter hands over to the next one. The mistake to avoid is running thirty cases, getting 27 right, and reporting that extraction "works".

Riskiest is not hardest, and this is not a vertical slice#

The two ideas most often substituted for this one are worth separating out.

Hardest-first is a different ordering and frequently the wrong one. The riskiest assumption in a system is sometimes trivial to test: whether the vendor's API allows the concurrency you designed for, whether the data-retention terms in the contract permit sending customer documents to a third-party model, whether the depot scanner can be configured above 200 dpi. Each of those can invalidate an entire plan and each takes an afternoon or an email. Teams that translate "prototype the risky part" into "start with the difficult engineering" skip the cheap existential checks in favour of the interesting ones, which is the same error in a more flattering costume.

A vertical slice is also a different technique with a different purpose. Building one thin path end to end — one invoice, from mailbox through extraction through posting — is a delivery pattern for systems already believed feasible, meant to flush out integration surprises and get something releasable early. It touches every component, which means it spends budget on the parts you were not uncertain about. There is a good moment for it, and that moment is after the existential assumption has survived, which is roughly the situation the delivery-note team believed they were in and were not.

There is also a case where none of this applies. When every component of a proposal is something the team has shipped before, the honest ranking has no row above a few percent, and the correct amount of prototyping is none. Running a PoC anyway costs two weeks and produces a document confirming what everybody knew, and the tell is that nobody can name a result that would stop the project.

Having put the prototype budget on the assumption that can end the project, you get a number back. That number is smaller than it looks, in a specific and quantifiable sense, and treating it as a measurement is how a well-designed PoC still ends in the fifty-minute meeting.