Errors you cannot detect are the ones that end a project

The contract extraction service ran for eight months before anyone found out it was wrong. It read customer agreements as they were countersigned — about 2,000 a week — and pulled six fields from each: term start, term end, notice period, auto-renew flag, governing law, and liability cap. Twelve thousand field extractions a week, written into the CRM record, from which the renewals team worked their outreach calendar and finance built the revenue schedule.

The launch number was 97.2% field-level accuracy, measured on 200 hand-checked contracts. That sample had been drawn from the previous quarter's closed deals, and the previous quarter had been unusually clean: 94% of those deals were signed on the company's own template. Across the full corpus, 31% of contracts are on customer paper, redlined, with the notice period buried in an amendment that references a clause number in the master agreement. On those, field accuracy is 78%. The blended rate is 95.9%, which puts roughly 490 wrong fields into the CRM every week.

Nothing downstream reconciled. The renewals team saw a date and called on that date; finance saw a term and scheduled revenue against it; nobody compared either against the document. The error surfaced in month nine when a customer disputed an invoice, legal opened the PDF, and the notice period in the CRM said thirty days where the signed agreement said ninety. The audit that followed found 214 contracts with a wrong term end and 23 where the company had missed its own non-renewal window. By then about 16,700 wrong fields had accumulated, and the remediation — re-extracting and hand-checking eighteen months of agreements — cost more than the original build.

Undetectability is the strongest thing you can put in a rejection#

Model quality was never the binding constraint on that project, and improving the model would not have helped, because nothing in the system was ever going to tell anyone that 490 fields a week were wrong. That property is what makes it the most powerful rejection available: it is a statement about the environment rather than about the technology, so it does not expire when the next model ships, and it converts an argument about capability into an argument about instrumentation, which is an argument that can be settled with evidence.

The condition has two forms, and separating them is the difference between a rejection and a condition on approval. An error is structurally undetectable when no future event ever reveals what the right answer was — a lead-scoring model that ranks accounts you then never contact, or a summary that informs a conversation with no recorded outcome, produce errors that nothing in the world will ever contradict. An error is merely unmonitored when the truth is knowable and cheap to obtain and nobody is looking, which is what happened to the contract service: the signed PDF was sitting in the document store the whole time.

The second form is not a reason to kill a project, because the fix is arithmetic. Sampling twenty contracts a week and checking six fields each is about three hours of a paralegal's time. At a true error rate of 4%, the chance of a weekly sample of twenty turning up no error at all is 0.96²⁰ ≈ 0.44, so the chance of four consecutive clean weeks is about 3.8%. A sampling regime costing roughly $300 a week would have surfaced the customer-paper problem inside the first month instead of the ninth, and the difference between those two outcomes is the entire cost of the incident. When a proposal has knowable ground truth — a recorded right answer for an item, obtained from somewhere other than the system being judged, which here is the countersigned PDF itself — and no monitoring plan, the right response is to make the monitoring plan a condition of approval with a named owner and a weekly number, and to say plainly what the exposure is per week of delay.

Structural undetectability is the one that kills. When you cannot construct a sampling regime at any price, because the ground truth genuinely does not exist rather than merely being expensive to obtain, the project has no path from version one to version two, no way to know when it has degraded after a model or prompt change, and no way to bound what it is costing. That is a rejection, and it should be written as one.

Four conditions, stated so a sponsor can act#

A rejection that a sponsor can act on names the condition, cites the evidence, and states what would change the answer. Rejections that fail do so in a predictable way: they describe a feeling, or they make a claim about model capability that the next model release appears to refute, or they are a scheduling statement that invites the reply "so what if we hire two more people".

The demand row is the one engineers are least comfortable writing and the one that most often turns out to be binding, because a proposal's stated user population is usually a guess made by somebody who is not in it. The evidence is a count from a system rather than a conversation: how many times did this event occur last quarter, in which table, and who performed it. A request to automate a monthly reconciliation that turns out to run nine times a year and takes forty minutes is a four-hour annual problem, and no amount of enthusiasm changes that arithmetic. Getting the count also produces the second useful thing, which is the name of a person who will commit to using the output, since a proposal with a named user who has agreed to a start date behaves very differently from one with a category of user.

The rejection is a document rather than a conversation, because the conversation will be re-litigated in six weeks by someone who was not in it. Something on the order of fifteen lines is enough, and the structure matters more than the length:

proposal: "Extract six contract fields into the CRM at signature"
decision: conditional-no
blocking_condition: unmonitored_error
evidence:
  - "Launch accuracy 97.2% measured on 200 docs, 94% of which were own-template"
  - "Own-template share of full corpus: 69%; measured accuracy on customer paper: 78%"
  - "Blended estimate 95.9% => ~490 wrong fields/week at current volume"
  - "No downstream system reconciles a CRM field against the source PDF"
would_change_the_answer:
  - "Weekly sample of 20 contracts x 6 fields, checked against the PDF by legal ops"
  - "Named owner and a published weekly error rate, with a 3% action threshold"
cost_of_the_check: "~3 paralegal-hours/week (~$300)"
cost_of_not_checking: "Eight months undetected at 490 fields/week = ~16,700 errors"
review_date: 2026-09-15
author: platform-eng

Where it goes matters as much as what it says. The record belongs to the person who can approve the mitigation rather than to the person who proposed the project, and it should arrive before the decision meeting rather than during it, so that the sponsor has read it privately and is not being contradicted in front of an audience. Circulating it two working days ahead converts the meeting from an argument into a review of a document, which is a format where evidence wins more often.

Two fields in that record do most of the work. would_change_the_answer makes the rejection revisitable, which stops it being read as obstruction and stops it being permanent, since the sponsor now owns a concrete path rather than a refusal. cost_of_not_checking puts a number where the argument would otherwise be about appetite for risk, and a number is what survives being repeated second-hand.

What a rejection is not#

It is not a veto, and framing it as one is the fastest way to lose the ability to issue another. The engineer writing the rejection is supplying a finding — this proposal has this property, here is the evidence, here is the consequence — and the decision to proceed belongs to whoever owns the budget. That distinction sounds like a matter of politics and is actually a matter of accuracy, because sometimes the sponsor knows something you do not: a regulatory deadline, a contractual commitment, a customer who will churn. A rejection stated as a finding survives that; a rejection stated as a refusal gets routed around.

It is also not a risk register. A list of eleven things that might go wrong, each with a likelihood and an impact rating, communicates diligence and cannot be acted on, because nothing in it says which single condition is binding. One blocking condition, stated with a number, does more than eleven graded risks, and if two conditions are genuinely binding then say two.

Nor is every undetectable error fatal on its own terms. Where the output goes straight to a domain expert who would recognise a wrong answer on sight and has a reason to care — a radiologist reading a flagged region, an engineer reviewing a proposed code change — detection lives in the reader, and that is a real mechanism worth crediting. It has two failure modes to check for rather than assume: the reader must actually inspect rather than accept, and inspection has to stay economic as volume grows. A reviewer who accepts 40 items an hour after the first month is not a verifier, and the honest version of that design states the review budget and what happens when it is exceeded.

Finally, a rejection based on cost is the weakest of the four and the easiest to overturn, because inference prices have fallen by roughly an order of magnitude every eighteen months and everyone in the room knows it. Rejecting a proposal because $0.0008 per document times two million documents is too much invites the reply that it will be $0.0002 next year, and that reply is usually correct. The durable version of the cost argument is not about the unit price at all — it is about whether the model needs to see every item in the first place, which changes the volume rather than the price, and which the next chapter works through in numbers.