An edge with no source document is indistinguishable from an invention

A graph built over a corpus of contracts, scanned vendor directories and years of procurement correspondence answers questions by walking edges — each edge a relationship one document states between two things. The report comes in on a Thursday from someone in procurement: the assistant says Acme Corp supplies Northwind Traders, and that is not true, has never been true, and the two companies are competitors. The graph has been in production for six months. Searching the corpus for any sentence connecting the two returns nothing.

The edge is there, and it has a document on it, which is what makes the first hour confusing.

{"type": "supplies", "src": "acme-corp", "dst": "northwind-traders",
 "doc_id": "scans/2019-vendor-directory.pdf", "page": 14,
 "quote": "Acme Corp Northwind Traders  connectors  Tier 2  net-45"}

The line that really was in the text layer#

Page 14 of that PDF is a scanned vendor directory: a five-column table, printed in 2019, photographed slightly skewed, run at ingestion through OCR — optical character recognition, the step that reads a page image and produces the text layer everything downstream actually parses. The text layer the OCR produced interleaved the "Supplier" column with the "Customer" column, so a row that read Acme Corp | Kestrel Ltd | connectors | Tier 2 | net-45 on the page came out of OCR as a single line in which Acme Corp sits adjacent to a company name from a different row entirely. The extractor read that line, saw two company names next to the word "connectors" in a document titled vendor directory, and emitted the only relation its schema had for the shape. Every step behaved correctly given its input.

Follow that chain one link at a time, because the whole chapter turns on where it starts. A skewed photograph gives the OCR engine rows whose baselines are not horizontal. Column assignment in OCR is a geometric judgement about which horizontal band a glyph belongs to, so skew makes that judgement wrong in a way that is systematic rather than random — every row on the page tilts the same direction. A wrong column assignment produces a text layer that is fluent, well-formed and false. Extraction then reads that text layer, and it has nothing else to read: the page image is not an input to it. So the extractor's output is a correct reading of an incorrect transcription, and the resulting edge is indistinguishable, in every field it carries, from an edge whose transcription was perfect.

That is the sentence to hold on to, because it is where the belief this chapter breaks lives. A reader who reaches for the obvious defence will say that every edge in their graph quotes its source verbatim, so no edge in it was invented. The claim is true and it is the wrong claim. Quoting verbatim establishes one link — that the extractor did not depart from the text it was handed — and the corruption here happened one link upstream of that. The quantity the defence measures, whether the quote appears character for character in the source text, is exactly identical for a faithful transcription and a mangled one. It stays constant while the dangerous property, whether the text layer represents the page, varies freely. A verification chain is only as long as its links, and this one begins after the damage.

How many edges are in the same condition#

The natural response is to fix the OCR on that page, re-extract the document, delete the bad edge, and add the page to a regression set. That is all worth doing and it addresses one edge out of a graph that has been accumulating them for six months.

Count the population. This corpus is 1.1 million edges, of which roughly 12% derive from scanned material — vendor directories, signed contracts returned as photographs, decades of archived correspondence. Pull a stratified sample of 200 edges from the scanned population — stratified meaning the sample is drawn to a fixed quota from each source class rather than uniformly, so that a small population is represented well enough to estimate a rate for it — and check each against the page image by hand, and the fraction that no document actually asserts comes back somewhere around 8–9%. Extrapolated, that is on the order of 10,000 to 12,000 edges in the graph that are supported by nothing, sitting alongside a million that are fine, with no field on any of them that distinguishes the two. The scanned population is where the density is highest, and the same audit over born-digital documents still turns up 1–2%, from tables in HTML exports, from footnote markers, from a model that read "should not be treated as a supplier relationship" and emitted a supplier relationship.

That extrapolation rebuilds itself exactly, and rebuilding it is what fixes its scope. Twelve per cent of 1.1 million edges is a hundred and thirty thousand or so, and eight to nine per cent of that is ten to twelve thousand. So the range is a rate measured on a sample multiplied by the size of one stratum, and the stratum is the scanned one. Nothing about it covers the rest of the graph.

Apply the same multiplication to the other stratum and the picture changes shape. The born-digital remainder is the other eighty-eight per cent, close to a million edges, and one to two per cent of that is another ten to twenty thousand unsupported edges — as many as the scanned stratum contributes, and possibly twice as many. The stratum with the lower defect rate carries more total defects, because it is roughly seven times larger. Reading a rate as though it ranked the strata by how much damage each does is the arithmetic mistake here, and it has a direct consequence: a team that gates only on scan confidence has addressed at best half the population. It is also why the scoped verification pass proposed later in this chapter covers the scanned edges and the single-support born-digital ones rather than the scans alone.

So the defect the incident exposed is not the OCR. It is that nothing in the system can tell an edge somebody asserted from one that extraction invented, and therefore nothing can be quarantined, audited, or ranked by trustworthiness, because there is no dimension on which to sort. The graph presents every edge with identical authority, and the answer generator has no basis to do otherwise.

Making the edge carry its own evidence#

Provenance is the record an edge carries of where it came from — which document, which version of it, which passage, read by which extractor — and it belongs on the edge itself. The fields worth having are more specific than "we stored the doc ID".

{
  "type": "supplies", "src": "acme-corp", "dst": "northwind-traders",
  "provenance": {
    "doc_id": "scans/2019-vendor-directory.pdf",
    "doc_version": "sha256:9f21c4…",
    "locator": {"page": 14, "char_span": [2841, 2903]},
    "quote": "Acme Corp Northwind Traders  connectors  Tier 2  net-45",
    "source_modality": "ocr",
    "source_confidence": 0.61,
    "extractor": {"model": "claude-sonnet-4-5", "prompt_version": "extract-v7"},
    "extraction_confidence": 0.72,
    "corroborating_docs": []
  },
  "status": "quarantined",
  "status_reason": "ocr_confidence_below_0.80"
}

That is the incident's edge as it would look under this schema, not as it looked in production. In production it carried a document, a page and a quote, and nothing more, and it was used in answers for six months. Most of what is written above it did not exist to be checked.

Four of these fields do work that the plain doc_id cannot. The document version hash makes it possible to find every edge derived from a document that has since changed, which is the mechanism chapter 6 needs for re-extraction. The character span plus the verbatim quote makes a deterministic check possible at write time: does this quote appear, character for character, in the text the extractor was given? That check is free, catches a genuine class of model fabrication, and — importantly — would not have caught this incident, because the mangled line really was in the text layer. Grounding an edge in the text layer proves the extractor did not invent it; it says nothing about whether the text layer represents the page.

That is what source_modality and source_confidence are for. OCR engines report per-page and per-word confidence, and the pages that produce merged columns are usually the pages with poor confidence scores. Gating on it — anything from a scan with page confidence below 0.80 is written with status: quarantined — puts roughly 4% of this corpus's edges into a state where they are not used in answers until something promotes them. It is a coarse instrument that trades recall for the ability to say which edges are load-bearing.

How coarse is worth deriving, because that 4% is quoted against the whole graph while the edges it acts on are all in one stratum. Four per cent of 1.1 million edges is a third of the hundred and thirty thousand scanned ones. So the gate quarantines a third of the scanned stratum to reach a defect population of eight or nine per cent of it, and even granting perfect recall — every bad edge falling below the threshold — at best one quarantined edge in four is genuinely unsupported. The source_confidence of 0.61 on the edge above is below 0.80 and would have been caught, which is what the status_reason records. So would three others that were fine.

corroborating_docs is the field that gets used most in practice. An edge asserted by three independent documents is in a different category from an edge asserted by one, and single-support edges from low-confidence sources are exactly the population the incident came from. It gives you something to rank by and something to sample.

Refuted edges are kept rather than deleted, with the reason attached, because the same document will be re-extracted and would otherwise regenerate the same bad edge on the next ingestion cycle. A refutation is a durable statement about a document, and it should outlive any particular extraction run.

Three instruments cheaper than a user report#

Three mechanisms would each have found it before a user did, and they cost different amounts.

Sampled audit at ingestion is the cheapest and the one most often skipped. Fifty edges a week, stratified so that low-confidence and single-support edges are over-represented, checked against the page image by a human, is around two hours of somebody's time. At an 8% defect rate in the scanned stratum, the probability of a weekly sample of fifty containing at least one bad edge is above 98%, so this failure surfaces in week one instead of month six. What the audit produces is not just the individual corrections but a defect rate per source modality, which is the number that justifies the gating threshold.

That probability is worth deriving rather than accepting, because the shape of it explains why so small a sample works. Each drawn edge is clean with probability one minus the defect rate, and if the draws are independent then all fifty coming back clean has probability that quantity raised to the fiftieth power. At a defect rate of eight per cent, one minus the rate is a little over nine tenths, and raising that to the fiftieth power leaves a little over one and a half per cent. The complement — at least one bad edge — is the ninety-eight per cent figure. A miss is therefore expected about one week in sixty-five, and six months of weekly sampling would have to be extraordinarily unlucky to see nothing.

Two residuals sit under that number, pulling in opposite directions. The sample is deliberately enriched with low-confidence and single-support edges, so its true hit rate is higher than the calculation, which makes the ninety-eight per cent conservative. Against that, the eight per cent came out of a stratified sample of 200 and is a stratum-weighted estimate, so it is only the right rate for the fifty if the two samples are weighted the same way. And two hours for fifty edges is about two and a half minutes each, or twenty-five an hour, which puts the ceiling on the whole idea: auditing the hundred and thirty thousand scanned edges at that rate would take over fifty years. The audit measures a rate. It never becomes a census, and nothing built on it should assume it does.

Contradiction checks run against the schema's own semantics. A supplies edge marked sole_source: true alongside a second supplies edge into the same part from a different company is a contradiction the graph can detect without a model. So is a supplier relationship between two entities that a competitor_of edge already connects. These checks find a narrow class, and they find it deterministically and for free.

Degree and structure anomalies catch the systematic version. When OCR merges columns across a whole document, it does not produce one bad edge, it produces forty, all from the same document, all connecting entities that share no other neighbours. A weekly job that ranks source documents by the number of edges they alone assert, and by how unlike the rest of the graph those edges are, would have put 2019-vendor-directory.pdf at the top of a very short list.

The reason that third instrument works is the reason the first one nearly misses: skew is a property of a whole page, so its defects arrive in correlated batches from one document rather than scattered across the corpus. A random sample sees a batch of forty as forty independent chances to be unlucky; a per-document ranking sees it as a single, very loud signal. The two instruments are strong on opposite failure shapes, which is why running one is not a reason to skip the other.

What a model re-reading the text can settle#

A teammate proposes an LLM pass over the whole graph: for each edge, fetch the source document, ask whether the document supports the claim, delete the unsupported ones. The instinct is right, the scope and the verb are both wrong, and it is worth being precise about what such a pass can and cannot establish.

Read that table as one boundary drawn four times rather than as four separate facts. Every cell on the left is a question about the text, and a model with the text in front of it can answer all of them. Every cell on the right is a question about something the text cannot see, and each names a different stage of the system: the page behind the text layer, the document's currency in the world, the evidence that was never extracted at all, and the node identities the edge hangs on. The right-hand column is a map of where the rest of this topic's failure modes live, and no amount of re-reading crosses from the left column to the right.

The incident in this chapter falls entirely in the right-hand column: a model re-reading the OCR text layer would find the quote present and would judge the relation supported, because the corrupted text does assert it. Verification against the page image with a vision-capable model is a different and more expensive pass that would catch it, and is worth running over the scanned stratum specifically rather than over all 1.1 million edges.

The cost of the naive version: 1.1 million edges, each needing the claim plus enough surrounding document text to judge it, at roughly 1,500 input tokens per call, is about 1.65 billion input tokens. At $3 per million that is around $5,000 for one sweep, plus output, plus the retries, and it must be re-run whenever the extractor or the judging prompt changes. Scoped to the 130,000 edges from scanned sources and the single-support edges from born-digital ones, the same pass is under $600 and catches a much higher share of what is actually wrong per dollar.

The scoped figure carries a constraint worth extracting, because it is not obvious how much of it the scans consume. A hundred and thirty thousand edges at 1,500 tokens each is a little under two hundred million tokens, which at the same price is just under five hundred and ninety dollars. That is nearly the whole of the six-hundred-dollar budget, leaving room for only a few thousand born-digital edges. If your single-support born-digital population is larger than that — and on a graph with close to a million born-digital edges it very easily is — the scoped pass is not under six hundred dollars, and the filter that selects it needs to be tighter than "single-support" before the figure holds.

Deletion is the part to refuse. A verifier with a 5% false-positive rate applied to a million edges destroys roughly 50,000 good edges, and because the missing hops are silent — chapter 3's point, arriving from a different direction — the resulting graph answers questions with confident partial evidence and no indication of what it lost. The pass should write status transitions and a reason, leaving refuted edges in place and excluded from traversal, so that a bad verification run is reversible and its precision is measurable against the human audit sample.

Set that fifty thousand against the population it is meant to clean and the trade settles itself. The two strata between them hold something in the range of twenty to thirty thousand unsupported edges, so a verifier with a five per cent false-positive rate destroys more good edges than there are bad ones in the entire graph — and that is before asking what fraction of the bad ones it actually catches. The false-positive count barely depends on the defect rate at all, because it is a percentage of the good edges and the good edges are ninety-nine per cent of the graph. A classifier can be improved; this particular arithmetic cannot be argued with while the action stays destructive.

Using the provenance at query time#

Once edges carry status and corroboration, the traversal can use them, and this is where the fields stop being bookkeeping. Three behaviours are worth having. Quarantined and refuted edges are excluded from expansion by default, with an override for an explicit "show me everything" mode used during investigation. Paths are ranked by their weakest link rather than by length, so a three-hop chain of corroborated edges outranks a two-hop chain that passes through a single-support OCR edge. And the evidence strength travels into the answer, so that a chain containing a quarantined link produces hedged output naming the weak step rather than a flat assertion.

def path_confidence(path) -> float:
    return min(EDGE_WEIGHT[e.status] * e.provenance.extraction_confidence for e in path.edges)

EDGE_WEIGHT = {"corroborated": 1.0, "supported": 0.9, "proposed": 0.6, "quarantined": 0.0}

The min in that function is the design decision, and swapping it for a product would quietly reverse the behaviour the paragraph above asks for. Multiplying the per-edge weights together gives a score that falls with every additional term, so a long chain of perfect edges would score below a short chain containing a weak one — length would dominate evidence, which is the ordering being rejected. A minimum has no term count in it at all. It returns the worst edge on the path and ignores how many good ones surround it, which is what "ranked by their weakest link rather than by length" means arithmetically.

The zero in EDGE_WEIGHT then does a second job for free. Any path containing a quarantined edge has a zero in the set being minimised, so the whole path scores zero, and the exclusion of quarantined edges is not a separate filter at all — it falls out of the weight table. What does not fall out is refuted, which the state diagram treats as a state and which has no entry in EDGE_WEIGHT, so this function raises rather than scoring a refuted edge zero. Refuted edges must be filtered out before the traversal reaches this line, which is consistent with the diagram's "retained, excluded from traversal" and is not visible in the code.

One more consequence is easy to read past. Because status weight is multiplied by extraction confidence rather than applied after it, the status ordering is not strict. A supported edge and a proposed one carry weights of 0.9 and 0.6, a ratio of two-thirds, so a supported edge is outranked by a proposed one whenever its extraction confidence falls below two-thirds of the proposed edge's. That may be exactly what you want, since it lets a hesitant extractor's grounded edge lose to a confident one. It is not what a reader who treats EDGE_WEIGHT as a hierarchy of statuses will predict.

Lineage, citation and support are three different records#

Provenance is not pipeline lineage. Knowing that this edge came from the 14 March ingestion run under prompt version 7 is useful for debugging and does not tell you whether anything supports the claim; those are different questions and it is easy to build the first and believe you have built the second.

It is also not verification, and the citation shown to a user is not verification either. Chapter 2's traversal trace attaches a document to every hop, and this chapter's incident had a document attached to the hop and a quote from it. What a citation buys is the ability for a human to check, cheaply, whether the claim survives contact with the source. Whether anybody ever does that check is a property of the product and the review process rather than of the graph.

Provenance is a chain of custody, in the sense an evidence locker uses the term — an analogy, and one whose limit is precisely where this chapter's incident sits. A chain of custody is built to detect a person tampering with evidence, and it works by making every handover accountable. Every handover here was accountable and none of them was tampered with. The record is a faithful account of a corrupted original, and a chain that begins after the corruption certifies it rather than catching it. What extends the chain backwards is not a better record but a different instrument: something that reads the page rather than the text layer.

None of this machinery is optional once you have decided to build a graph, and all of it is fixed cost: schema design, entity resolution, provenance, audit, verification passes. The question that should come before any of it is whether the corpus and the questions asked of it need traversal at all, because a large number of corpora do not, and the next chapter is about how to tell.