A text layer guarantees characters, never their order

A PDF that has been through parsing arrives in one of two shapes, and which one you are holding decides whether extraction is a reading problem or a recognition problem. A born-digital file was produced by software directly from the text — Word, LaTeX, a reporting tool, a print-to-PDF driver — so it carries a text layer, meaning the character data the producer stored alongside the drawing instructions, and extracting it is a matter of reading that data back out. A scanned file was photographed: a feeder or a camera captured a printed sheet, and the file holds pixels, with everything a reader can see existing only as ink in an image.

The distinction runs through the rest of this topic, and the classification has to happen before any other decision does, because every later choice branches on it. There is no useful default: a parser configured for one shape produces nothing usable on the other. The classification is also the cheapest thing in the whole of ingestion — it reads a count that the extractor has already computed — so there is no argument for deferring it or for sampling rather than checking every page.

The two shapes, in one queue#

An ingestion queue holds two files. Both are named with a hash, both have the .pdf extension, both are exactly forty pages, and both were uploaded by the same connector within a minute of each other. Extraction on the first returns 96,412 characters in 178ms. Extraction on the second returns 31 characters in 164ms.

The two timings are the first thing worth noticing, because they are almost identical while the outputs differ by a factor of thousands. Extraction time tracks the work of walking the file's objects, and both files have forty pages of objects to walk. It does not track how much text comes out. Duration is therefore another quantity that stays put while the thing you care about varies, and it belongs on the list of measurements that cannot tell you anything here.

Those 31 characters are worth looking at, because they are not noise:

CONFIDENTIAL — Ward & Pryce LLP

That string is a stamp applied by the document management system after scanning, drawn as real text on top of an image. Every other mark on all forty pages — every clause, every defined term, every signature — is a 200 dpi bitmap of a printed page that somebody put through a Xerox. The PDF is a container holding forty JPEGs and one line of type.

The check is characters per page, and it runs per page#

Total character count classifies these two files correctly and will keep doing so, but the unit matters. A per-document count breaks on the file nobody plans for: a 40-page master services agreement where pages 1 through 38 came out of Word and pages 39 and 40 are a scanned, signed and re-inserted signature page. That file returns 91,000 characters, sails past any document-level threshold, and silently drops the two pages that say who signed and on what date.

The mechanism of that failure is worth stating in general terms, because it recurs in this topic in at least two more places. A document-level count is a sum over pages, and summing destroys the location of a deficit. Divide the two figures already on the table by their page counts and both files land near the same number: 96,412 over forty pages, and 91,000 over the thirty-eight pages that actually carry text. Two documents from different producers, arriving at the same characters-per-page figure by routes that share no step, is what licenses treating that figure as a property of the source material rather than of either file. Against a full page of roughly that size, the two scanned pages of the agreement are worth about a twentieth of the total. Any document-level threshold loose enough to admit healthy files admits this one, because the majority class dominates the aggregate and the minority class is the part you needed.

import fitz  # PyMuPDF

SCAN_CHARS_PER_PAGE = 100

def classify_pages(path: str) -> list[dict]:
    doc = fitz.open(path)
    out = []
    for i, page in enumerate(doc):
        text = page.get_text("text")
        chars = len(text.strip())
        page_area = abs(page.rect.width * page.rect.height) or 1.0
        img_area = sum(
            abs(fitz.Rect(b[:4]).get_area())
            for b in page.get_image_info(xrefs=True)
        )
        out.append({
            "page": i + 1,
            "chars": chars,
            "image_coverage": round(min(img_area / page_area, 1.0), 3),
            "has_fonts": bool(page.get_fonts()),
            "route": "ocr" if chars < SCAN_CHARS_PER_PAGE else "text",
        })
    return out

Run that over the second file and the shape of a scan is unmistakable:

page  chars  image_coverage  has_fonts  route
   1     31           0.998       True   ocr
   2      0           0.997      False   ocr
   3      0           0.998      False   ocr
 ...
  40      0           0.996      False   ocr

Read that output as three columns moving independently rather than as four lines of evidence for one conclusion. The chars column and the has_fonts column move together and they move exactly once, at page 1: thirty-one characters and a font on the first page, nothing and no font on every page after. The image_coverage column does not move at all. It sits just under one on all forty pages, and that refusal to vary is precisely what identifies the file, because a document whose every page is a photograph has no page that can score low. The single row where the other two columns come apart from it is the stamp, and the absent row — a page with low coverage anywhere in the file — is the finding.

Three signals agree, and using all three rather than the character count alone is what keeps the classifier from misfiring. Character count catches the ordinary scan. Image coverage above roughly 0.9 catches the page that is a picture of text regardless of what is stamped on it, which is the case where the character count is high enough to pass on its own. The font list catches the inverse case, a page with no images and no glyphs, which is a genuinely blank page and should be routed to neither OCR nor the index — it should be counted, because a source that suddenly starts emitting 30% blank pages has a producer bug worth an email. Each signal covers a page shape that would defeat the other two, which is the reason for three rather than a better one.

The 100-character threshold is not a constant of nature. A slide export legitimately runs 40 to 120 characters per page and is fully born-digital; a title page of a report may hold twelve words. What makes an arbitrary round number safe on ordinary documents is the size of the margin around it: against a page carrying the characters-per-page figure derived above, the threshold sits at roughly a twenty-fourth of a full page, so a page has to lose almost all of its text before it is misrouted. What makes it unsafe on slides is that the slide range straddles it, and no margin exists to absorb an error.

Any threshold you pick will misroute some pages, so the useful posture is to treat it as a router with a known error rate rather than a classifier with a truth value, and to log the borderline cases — say, pages between 40 and 250 characters with image coverage above 0.5 — as a queue somebody reads once a month. That band is chosen rather than discovered: it is drawn wide enough on both sides to contain the slide range that straddles the threshold, which is the known population of misroutes. On the corpus this chapter is drawn from, that band was about 1.4% of pages and roughly half of them were genuine scans with a caption typeset over the image.

Two of the four leaves in that diagram are not routing decisions at all. The borderline queue and the blank-page leaf both end in counting something and telling a person, which is the shape a decision takes when the available signals establish that a page is unusual and cannot establish what is wrong with it.

The file that fails loudly is the file that gets fixed#

Ask which of the two documents is more likely to be the source of a confidently wrong answer six weeks from now and the intuitive answer is the scan, because the scan is the broken one. That instinct is reasonable and it is wrong, and it is worth knowing which step in it fails. It assumes risk is proportional to how damaged the output is. Risk here is set by how long the damage survives, and survival is set by detectability, which runs the other way: the more obviously broken an output looks, the sooner somebody sees it. The scan is not the dangerous one, for a reason that has nothing to do with parsing quality and everything to do with feedback.

A file that yields 31 characters produces a document with no retrievable content. Every question about that contract returns chunks from other contracts or returns nothing. Somebody notices within days, because "the assistant does not know about the Ward & Pryce agreement at all" is a complaint a user makes immediately and a complaint that leads straight to the file. The failure is total, attributable and visible, which are the three properties that get a defect fixed.

The born-digital file yields 96,412 well-spelled characters in an order that chapter 1 already established is a reconstruction and may be wrong. Its chunks read as fluent English. They embed to sensible vectors, they retrieve for plausible queries, and the model summarises them without hesitation. When the reading order has interleaved two columns, or has glued a footnote reading "Superseded by Amendment 2, March 2024" into the middle of the clause it annotates, the resulting answer is grammatical, sourced to a real document, and wrong. No downstream check catches it: there is no exception, no empty result, no low similarity score, no refusal from the model. The only detector is a human who knows the correct answer, reading the output, which is a detector you have for approximately the first two weeks of a project.

The first row of that table is the whole argument compressed into four words appearing twice. "No exception" is the verdict for the file that works and the verdict for the file that does not. Exception handling is the defence most teams are relying on here, and it returns the same answer whichever file it is given, so it discriminates nothing — and neither does anything downstream of it, since the embedder embeds scrambled text without complaint, the reranker ranks it, and the model writes a paragraph from it. Only the last row varies, and it varies by having no value at all in one column: the cost of the fix is knowable for the scan and unknown for the born-digital file. That empty cell is the reason the second column is the expensive one. A defect nobody can detect is a defect nobody can budget for.

This is the shape of parsing defects in general, and it is why the rest of this topic spends more time on files that extract successfully than on files that do not. Extraction failures are a work item. Extraction that succeeds incorrectly is a liability, and it is the default outcome for the majority of a typical corpus.

What one code path would actually cost#

Running OCR over everything is the normalisation that gets proposed instead of routing, and it is the wrong one. The proposal arrives on every project, usually from whoever is maintaining the ingestion code, and it has a real argument behind it. One code path is cheaper to reason about than two. The classifier above has a threshold, and thresholds have error rates, and a pipeline with no classifier has no classification errors. There is a further point that is genuinely correct and rarely made: OCR engines perform layout analysis as a matter of course, because they have to segment the page image into text regions before recognising anything, so running OCR over a born-digital two-column report can actually produce better reading order than a naive text-layer extraction does. Tesseract's page segmentation will find the gutter that pypdf never looks for.

The proposal still loses, on three grounds.

The first is that it trades exact characters for recognised ones, and the two are not the same kind of thing. A /ToUnicode CMap gives you the producer's own statement of which character each glyph is, which is correct by construction — the producer held the character first and drew the glyph from it, so writing the mapping down is a record of something it already knew. OCR runs that arrow backwards. It starts from ink and produces a probability distribution over characters, marginalised into a best guess. Reversing a step that discarded information cannot recover the information, so the guess is right most of the time rather than right by construction. Across ordinary body prose the difference is invisible, and across the strings that carry the answer — account numbers, dates, dollar amounts, statute citations — it is not, which is the whole of the next chapter.

The second is cost and latency, which are unglamorous and decisive at corpus scale, and which the scenario below works out with figures. The shape of it is that both scale with the number of pages processed, and routing multiplies that number by the scanned share of the corpus, so the saving is whatever fraction of your pages have a working text layer.

The third is that it does not actually deliver the uniformity it promises. OCR output still needs the same downstream handling as text-layer output: reading order still has to be reconstructed, tables still lose their structure, headers and footers still need stripping. You end up with one extraction call and two sets of quality characteristics anyway, plus a new class of defect — recognition errors — spread uniformly across a corpus where it previously existed on 22% of pages. The single code path is a simplification of the code and a complication of everything the code produces.

Where characters-per-page stops being informative#

The check answers exactly one question, which is whether there is a text layer to work with. It says nothing about whether that layer is any good. A file with 96,412 characters in perfectly scrambled order and a file with 96,412 characters in perfect order are indistinguishable by every signal in the classifier above, and no refinement of the threshold will separate them, because the information the classifier reads is not the information the distinction depends on. All three signals read the presence of character data. Order is a property of the arrangement of that data, and no count of a set can report the order of the set. That is why adding a fourth count would not help either.

It also does not detect the partially useful text layer, which is more common than it sounds. Some scanning workflows run OCR at capture time and embed the result as an invisible text layer behind the page image, so the file reports 88,000 characters and looks born-digital to the classifier while actually being OCR output of unknown vintage and quality. The router does put that file somewhere sensible without understanding it: plenty of characters and near-total image coverage is the one path through the diagram above that ends in the borderline queue rather than in a route. Others embed a text layer for the first page only. Distinguishing "text layer written by the typesetter" from "text layer written by a 2011 scanner appliance" requires looking at the producer metadata in /Info and /XMP, and even then the answer is a guess.

What survives all of that is the narrower claim, and it is worth stating in its narrow form because the broad form is what gets people into trouble. Characters per page is a reliable router between two extraction strategies and a useless judge of either one's output. It earns its place by being free, by running on every page rather than every document, and by failing in a direction you can see — a misrouted page goes to a queue, not to the index.

Both branches of the router now lead somewhere with its own failure mode. The scanned branch leads to a process that reports its own accuracy as a percentage, and that percentage is about to turn out to mean something quite different from what it appears to promise.