A checker's accept set

The accept set of a checker is the set of outputs it will pass. Call it A, and call C the set of outputs that are actually correct for the request. A useful checker is one where A is much smaller than "everything the generator might produce" and where the leftover A minus C — the outputs it wrongly accepts — is small and consists of failures you can tolerate. Two error classes fall out of that. A false accept is an output in A that is not in C; a false reject is an output in C that the checker refuses.

Those two words are the whole of this chapter, because the gap between accepted and correct is where a verifier-first design either holds or quietly stops holding. A check that runs, returns in milliseconds and removes an entire class of production errors can leave the errors that matter completely untouched, and because it now runs on every output it attaches its endorsement to them on the way past. Nothing in the checker's own behaviour reports that, which is why the gap has to be drawn deliberately for each check you install.

What the planner is actually asserting#

Start with a check a great many teams have shipped. An agent that answers analytics questions by writing SQL against a warehouse had been throwing errors on 6.8% of queries — misspelled column names, a table that exists in staging but not in production, the occasional unbalanced parenthesis in a long common table expression, one of the WITH blocks that name an intermediate result set. The fix was to run EXPLAIN on the generated query before executing it — EXPLAIN being the planner's dry run, which returns the plan it would use without reading a single row — and, if the planner refused, hand the error text back to the model and regenerate, up to three times. Error rate went from 6.8% to 0.1% within a day and stayed there.

EXPLAIN returns a plan, and a plan is a claim about executability. It says the text parses, that every referenced relation and column resolves, that the types in each comparison are compatible or coercible, and that the current role has enough privilege to read what is named. Those are real properties and it is worth having them checked before a query runs against a warehouse that bills by the byte scanned. None of them is a claim about what the query computes.

The reason it cannot be a claim about what the query computes is structural rather than an oversight in Postgres. Planning a query walks four stages, and each one rejects a class of text. The parser rejects anything that is not grammatical SQL. Name resolution consults the catalogue and rejects references to relations and columns that do not exist. Type checking rejects comparisons between things that cannot be compared or coerced. The planner then chooses join orders and access methods using stored statistics about the tables. Notice what is absent from that list: at no point does the database read a row. That is the entire selling point of EXPLAIN — it is fast and free precisely because it touches no data — and it is also the reason the check is blind to every property that depends on what the data contains. Whether two shipments exist for one order is a fact about rows. The planner never looks.

That also explains the 0.1% that survived rather than a clean zero. A plan is a claim that the query can run, and a small class of queries plans cleanly and still fails once rows arrive: a division that turns out to be by zero, a cast that fails on one malformed value, a scan that exceeds a time limit. Those are data-dependent, so they sit exactly where the planner is blind, and they are the only executability failures left after the check.

Consider a question an analyst actually asked: what was net revenue from EU customers last quarter? Here is a query the agent produced that planned cleanly on the first attempt.

SELECT SUM(oi.unit_price * oi.quantity) AS net_revenue
FROM orders o
JOIN order_items oi ON oi.order_id = o.id
JOIN shipments s    ON s.order_id = o.id
JOIN customers c    ON c.id = o.customer_id
WHERE c.region = 'EU'
  AND o.placed_at >= '2025-04-01'
  AND o.placed_at <  '2025-07-01';

Every table exists, every column exists, the types line up, and the planner produces a perfectly good hash-join plan. The query is also wrong by a factor of about 1.4, because joining orders to both order_items and shipments fans the item rows out once per shipment, and orders that shipped in two parcels contribute their revenue twice. It ignores refunds entirely, which the finance definition of net revenue does not. The number it returns, €4.42 million, is the right order of magnitude, has no anomalies in it, and looks exactly like the number that would have been right.

Fan-out is worth following row by row, because it is the most common way a correct-looking join produces a wrong total and it is invisible in the query text. The orders table has one row per order. order_items has several rows per order, and joining them is intended: each item row is a real item. But shipments also has one row per parcel, and joining that to orders as well means the database pairs every item row with every shipment row for the same order. An order with three items in one parcel produces three combined rows, which is right. The same order split across two parcels produces six, and the SUM adds each item's revenue twice. The multiplier on any given order is simply how many shipments it had.

The 1.4 then reads back as a fact about the business rather than about the code. The inflated total is each order's revenue multiplied by its shipment count and summed, so for the whole figure to come out about forty per cent high, something like two orders in five — weighted by revenue — must have shipped in more than one parcel. That is a perfectly ordinary thing for a retailer, which is why nobody spotted it. It also means the error is a multiplicative one that scales with the size of the answer: bigger quarters are wrong by more euros, and the figure never looks anomalous because everything is inflated together.

The two defects in that query are worth keeping separate, because there is a tempting thought that they might offset. Fan-out inflates. Ignoring refunds also inflates, since refunds reduce net revenue and none were subtracted. They happen to run the same way here, and even if they had not, one is a multiplication and the other is a subtraction that was never performed, so their sizes are set by completely unrelated quantities and any cancellation would be a coincidence that changes every quarter. The correct figure is more than a million euros below the one on the slide.

There is no version of EXPLAIN that catches this. Fan-out from a one-to-many join is a valid, ordinary, intentional thing to write; the planner has no way to know that this particular one was not intended. The check and the failure are orthogonal. For EXPLAIN over generated SQL, A is enormous: it contains every syntactically valid query over the warehouse's schema, which is to say every wrong answer that happens to be well-formed, while C for a given question is typically a handful of query shapes. The check eliminated the 6.8% of outputs that fell outside A and did nothing whatever to the rest.

Different checks over the same artefact carve out very different accept sets, and the differences are worth having in front of you when you pick one:

Read down the last column and the trade appears in its usual form. The cheapest check has the widest accept set. The check that would have caught the €4.42 million requires executing the query and comparing against a number somebody else has already certified, which costs a query's worth of warehouse time and only exists for metrics finance has bothered to define. The narrowest check is also the one that rejects most of the questions people want to ask, which is a real cost paid in usefulness rather than in dollars.

The same shape in a citation checker#

The pattern is not specific to SQL, and the second instance is instructive because the check looks stronger than it is. A retrieval-augmented answering service adds this rule: after the model writes an answer containing quoted sentences with citations, the system re-retrieves each cited chunk by ID and confirms that every quoted sentence appears verbatim inside it.

def verify_citations(answer: Answer, store: ChunkStore) -> Verdict:
    for quote in answer.quotes:            # (text, chunk_id) pairs
        chunk = store.get(quote.chunk_id)  # re-retrieved, not the cached copy
        if chunk is None:
            return Verdict(False, f"cited chunk {quote.chunk_id} does not exist")
        if normalise(quote.text) not in normalise(chunk.body):
            return Verdict(
                False,
                f"quoted text not found in chunk {quote.chunk_id}: {quote.text[:80]!r}",
            )
    return Verdict(True, None)

This is a good check and it costs about 8 ms per quote against a local store. It catches invented quotations, which is the failure that embarrasses you in front of a customer, and it catches citations that point at a chunk that does not contain the claim, including the common case where the model attaches the ID of the chunk next to the right one. It is deterministic, it is independent of the generator, and its rejections come with a reason precise enough to regenerate against.

What sits in its accept set is an answer that quotes real sentences accurately and then draws a conclusion those sentences do not support. "The policy states that refunds are issued within 14 days of receipt of the returned item" is verbatim in chunk 4471, and the answer that follows it — "so this customer is entitled to a refund" — is not in any chunk, is not quoted, and is not checked. The verbatim match is mechanically decidable; the inference stacked on top of it is not, and it is the part the customer acts on. Getting the quote right and the conclusion wrong is also a more convincing failure than a fabricated quote, because the citation link resolves and a reviewer who spot-checks it finds the sentence exactly where it was promised.

The false-reject side has its own texture here. Legitimate answers get rejected when the model paraphrases lightly and still marks the text as a quote, when a soft hyphen or a non-breaking space survives from a PDF, or when the quote spans a chunk boundary and appears in neither chunk in full. In one batch of 500 answers this check rejected 41, of which 12 were correct answers with cosmetic quoting problems. Those 12 each triggered a full regeneration, and about a third of the regenerations introduced a different problem.

Those figures are worth pushing one step further, because the interesting number is not in them yet. Of the 41 rejections, 12 were mistakes, which leaves 29 genuine catches — so between a quarter and a third of everything this check rejects is an answer that was already correct. Each of those 12 costs a regeneration, and a third of the regenerations come back worse, which is about four answers in the batch made worse by a check behaving exactly as designed. Set that against the batch: four in 500 is under one in a hundred, which is small, and it is small in a way that has to be compared against how many real fabrications the same check caught rather than judged on its own. That is the form a false-reject cost always takes. It is never a reason not to check; it is a quantity that has to be put next to the catches, and a check whose rejections are mostly wrong is a check that is spending money to degrade output.

Measuring a checker instead of assuming it#

None of the above is knowable from reading the checker's source. It comes from putting a representative batch of outputs through it and recording three things: how many outputs it wrongly accepted, how many it wrongly rejected, and what it cost in wall-clock per candidate. Representative matters — a batch of the questions your agent handles well will understate both error columns, so the batch should be drawn from real traffic, including the awkward tail of requests that mention two fiscal calendars or three tables nobody documents.

The most expensive version of this mistake is a test suite used as the verifier for generated code, because a test suite feels like the strongest possible check and its accept set is set by whoever wrote the tests. If the suite has three cases and the model saw all three in the prompt, passing proves the model reproduced three input-output pairs, and the accept set is every function that agrees with the reference on those points and diverges everywhere else. Benchmarks built on small hand-written suites systematically overstate how often generated code is correct for this reason; Rethinking Verification for LLM Code Generation: From Generation to Testing measures the gap by expanding the suites and watching pass rates fall. The practical consequence is that when your verifier is a test suite, its accept set is a variable you control, and widening the suite is a direct, measurable investment in the verifier rather than in the generator.

The shape of that accept set is worth picturing, because "three test cases" does not sound like a weakness until it is drawn. A test suite constrains a function at the points it exercises and says nothing at all about any other input. Three cases pin three points. Every function that agrees with the reference at those three points and does anything whatever elsewhere is inside the accept set, and that is not a slightly larger set than the correct one — it is unbounded. Adding a fourth case pins a fourth point and the set is still unbounded. What extra cases buy is not coverage of the input space, which is unreachable, but a lower probability that a plausible wrong function slips through, since a wrong function has to agree with the reference everywhere the suite looks. That is why the investment is real and why it is subject to diminishing returns in the same way as everything else measured this way.

The situation is worse when the model saw the tests. Passing then demonstrates that the model reproduced three input-output pairs it was shown, which is a claim about copying rather than about generalising, and the accept set has not merely stayed wide — the evidence that the output sits anywhere useful inside it has been removed. Tests written after generation, by somebody who did not see the candidate, are a different instrument wearing the same name.

A wide accept set is not a bug in the checker#

Two failure modes show up at opposite ends and they need different responses. A checker with a wide accept set creates false confidence, and false confidence is worse than no check, because it removes the informal review step that was previously catching things — the runbook line about sanity-checking totals disappeared for exactly this reason, and it disappeared through a reasonable act by a careful person, which is the only way controls of that kind are ever removed. A checker that over-rejects is more visible and less dangerous, but it is not free: every false reject buys a regeneration, and regenerations of already-correct outputs are pure cost with a chance of making things worse.

It is worth being clear that neither of the checks above is defective. EXPLAIN implements exactly the property it claims to, and the citation checker's string matching is correct. The mistake is at the level of design, where a check for one property was installed in a slot that the system's promises required a different property to fill. Naming the property out loud — "this verifies that the query is executable" rather than "this verifies the query" — is most of the defence, because it makes the residual visible to whoever is deciding what the feature is allowed to be used for.

The naming discipline extends to what the pipeline calls its own stages, since a stage named verify is read as a guarantee by every engineer who arrives after the person who wrote it. Whatever you write in that identifier is the claim the rest of the organisation will act on, and it will outlive the design document that qualified it.

It applies one level further down as well, to what the verdict itself is called. A field named accepted, or a constant named Accept in an enum of verdicts, says what actually happened: this checker accepted this output. A field named valid, or ok, or passed_qa asserts a property nobody established, and it does so in the type signature, where it is read by everyone and reviewed by nobody. The rule that produces the right name every time is to describe the checker's own action rather than the output's status, because the checker's action is the only thing the code has observed.

The comparison worth holding on to here is that the two failure modes are not opposites of the same quantity. Over-rejection announces itself: it appears as regenerations in the logs, as latency, as a line in the bill. Over-acceptance is recorded nowhere at all, and its cost arrives as a decision somebody made on a wrong number weeks later. So a team tuning a checker will feel pressure from one direction only, and the direction it feels pressure from is the one that is safe to get wrong.

Of the three numbers a checker measurement needs, two come for free. Runtime is on the clock and false rejects are in the logs, because a rejection is an event the system produced and can count. False accepts are different in kind: the outputs the checker wrongly approved are, from the checker's point of view, indistinguishable from the ones it rightly approved, so there is nothing in the system that can be queried to find them. Everything in this chapter about accept sets stayed qualitative for that reason.

The next chapter is about where the missing column comes from, and about what it takes to say that a generate-then-verify loop improved accuracy rather than merely improved conformance.