A PDF stores glyph placements, not paragraphs or columns

Document parsing is the stage of an ingestion job that takes a file in the form somebody actually stored it — a PDF, which stands for Portable Document Format, or a Word document, a web page, a slide deck — and turns it into the plain text that a later stage chunks, embeds and retrieves. It exists because retrieval operates on text and real document collections almost never arrive as text. They arrive as formats built for printing, for editing or for display, each of which stores something other than a clean sequence of sentences, so something has to translate. You meet parsing as the first stage of every ingestion job whether or not you notice it: a connector pulling contracts off a vendor's file drop, a crawler fetching a help centre, a folder of quarterly packs left in a bucket, each hands its files to a parser before anything else happens. Naming that stage before anything else is worth the paragraph, because the whole of this topic is an argument about it.

Where the source is a folder of markdown files, as in a naive ingestion job, that stage is a call to open a file and read it, and it needs no thought. Pointed at documents produced by the rest of an organisation, the same stage is where most of the eventual answer quality is decided. The specimen worked through below is a forty-page two-column research report exported from LaTeX, which is a mild example rather than a pathological one. What makes it worth working through is that it fails without failing. No exception is raised, no character is lost, and retrieval quality collapses anyway.

What the format guarantees, and what it therefore cannot record#

PDF descends directly from PostScript, which Adobe shipped in 1984 as a language for driving laser printers, and it inherits that job description. The format was designed to print, and it was not designed to be read back. The guarantee PDF makes is that the page renders identically on any device: the same glyphs — the individual drawn shapes a font supplies for each character — at the same coordinates in the same fonts, whether the output is a 1993 imagesetter or a phone screen. It makes no guarantee at all about what the text means, because a printer does not need to know.

It is worth separating the requirement from the decision that met it, because only one of the two was forced. The requirement is identical rendering anywhere. Satisfying it takes exactly three things per mark: which glyph, from which font, at which coordinate. A device holding those three can put ink in the right place, and no fourth fact improves the page. The design decision that follows is to encode a page as a sequence of drawing operations and stop there. And a paragraph is not a mark. A column is not a mark. Neither has a coordinate, so under that encoding neither has anywhere to be written down. Reading order is therefore absent by construction rather than by oversight — and, as the tagged files at the end of this chapter show, it was a decision and not a necessity, since the same rendering guarantee holds perfectly well with a structure tree present.

What a page actually contains is a content stream, which is a small program in a stack-based operator language. The relevant operators are few. BT and ET bracket a text object, Tf selects a font and size, Tm sets a text matrix that positions the next run in page coordinates, and TJ draws an array of strings with kerning adjustments between them. A fragment of the report's page 4 looks like this:

BT
  /F14 9.96 Tf
  1 0 0 1 72.00 708.52 Tm
  [(Recall) -250 (at) -250 (10) -250 (impro) 20 (v) 15 (es) -250 (from)] TJ
  1 0 0 1 317.24 708.52 Tm
  [(W) 80 (e) -300 (thank) -300 (the) -300 (anon) 5 (ymous) -300 (re) 25 (vie) 25 (wers)] TJ
  1 0 0 1 72.00 696.57 Tm
  [(0.61) -250 (to) -250 (0.74) -250 (when) -250 (the) -250 (reranker)] TJ
ET

Three things in that fragment are worth reading closely. There is no space character anywhere; the gaps between words are the -250 values, which are kerning adjustments in thousandths of an em that the renderer applies as horizontal displacement. The Tm operators jump from x=72 to x=317 and back to x=72. And nothing in the stream says "column", "paragraph", "heading", "footnote" or "table" — those words do not exist in the file, because they do not exist in the format.

Take the missing space character first, because it is where the reconstruction starts. The renderer's job on reaching the next string is to place it, and a gap is a displacement, which is a number. A character that draws nothing would add work and change no ink, so a producer has no reason to emit one. Every extractor you will ever use therefore has to infer word boundaries from those displacements by picking a threshold — a decision made inside the extractor, against no value the format prescribes — which is why extractions occasionally run words together or split a word in the middle.

The fragment shows why the inference works at all, and it shows it completely. The numbers in a TJ array are subtracted from the current position, so a negative value pushes the next glyph further along and opens a gap, while a positive one pulls it back and tightens the fit. Every negative value in that fragment sits at a word boundary: between Recall and at, between at and 10, between thank and the. Every positive value sits inside a word: the 20 and 15 inside "improves", the 5 inside "anonymous", the 25s inside "reviewers", the 80 that closes up the "W" and "e" of "We". There is no exception in the fragment, and the two groups are separated by more than an order of magnitude — the largest intra-word adjustment is 80, the smallest inter-word gap is -250. A threshold placed anywhere in that gulf gives identical output, which is what licenses word inference on this producer at all. It also says where the method fails: on a producer whose two distributions overlap, and there is no threshold that separates overlapping distributions.

That the fragment carries two different word-gap values, -250 in the left column and -300 in the right, is the same lesson at smaller scale. Both are the same producer on the same page, setting two columns to two measures. The inter-word gap is not a constant even within one file, which is why the extractor holds a threshold rather than an equality test.

The Tm jumps are the second thing, and they are the failure in raw form. The producer sets the pen at x=72, draws a line, sets it at x=317, draws a line, then returns to x=72 for the next one. That is a line of the left column, then a line of the right column, then the next line of the left. The 9.96 in /F14 9.96 Tf is the type size for all three, so the two columns are not even distinguishable by the font selected. Nothing in the sequence marks where one column ends.

The two mappings a PDF carries, and the one it does not#

The characters themselves come out correct through a separate mechanism. A font in a PDF is usually subset and re-encoded, so the byte 0x03 in the stream might be the glyph "R" in one font and "µ" in another, and the mapping back to Unicode lives in an optional /ToUnicode CMap that the producer embeds. LaTeX, Word and InDesign all embed one, which is why the report's characters are perfect.

The asymmetry between the two mappings is not arbitrary, and seeing why makes it much easier to predict. Glyph identity is unrecoverable from the page: once a font has been subset, the byte 0x03 carries no clue about which character it stands for, so if the producer does not write the mapping down it is gone for good. Reading order has no equivalent mechanism, and it does not need one to be guessable: it can always be reconstructed from coordinates, because the coordinates are right there. A format that stores what is unrecoverable and omits what is guessable is behaving sensibly for a format whose stated job is printing. The bill for that choice lands entirely on anybody trying to read the file back.

So the format carries a reliable answer to "which character is this" and no answer at all to "which character comes next". That asymmetry is the whole of this topic's first lesson, and everything else in this chapter follows from it.

Why sorting by position makes it worse#

The obvious repair is to ignore stream order and sort by geometry, since you know where every glyph is. Sorting by y descending and then x ascending across the page produces exactly the failure below, and it produces it more reliably than stream order does, because the two columns share baselines. In a two-column layout both columns sit on the same grid, so the left column's line at y=708.52 and the right column's line at y=708.52 sort as adjacent, and the extractor emits them adjacent. Emitting adjacent lines from alternating columns is, repeated down the page, an interleave. What comes out of chunk 14 of the report reads like this:

Recall at 10 improves from We thank the anonymous reviewers 0.61 to 0.74 when
the reranker for their careful reading of an is applied to the top 50 earlier
draft of this paper, and candidates rather than the top the Sponsored Research
Office 10, at a cost of 180ms per for grant 2019-4471 which query at p50.
funded the annotation effort.

Two paragraphs about entirely unrelated subjects have been shuffled together a line at a time. The first two runs of that chunk — "Recall at 10 improves from" and "We thank the anonymous reviewers" — are the same two TJ runs printed in the content stream fragment above, the pair sharing y=708.52. The failure visible in the chunk and the pair visible in the raw stream are the same event arriving by two routes that share no processing step, which locates the defect exactly: it is the ordering decision, not the chunker, not the encoder, and not the font.

An embedding of that chunk sits somewhere between "reranking results" and "funding acknowledgements", close to neither, and the sentence a user needs — that recall at 10 improves from 0.61 to 0.74 — exists nowhere in the index as a contiguous string. No character was lost, no library failed, and no error was logged.

Stream order is not better; it is merely unpredictable. For a plain LaTeX article with no floats, pdfTeX ships each column out as a completed box, so stream order is often correct. Add a figure that spans both columns, or a page where the balancing routine splits the flow, or export the same content from Word, and the drawing order changes without anything visible on the page changing. Producers that re-emit an existing PDF — linearizers, "optimize for web" passes, redaction tools, print-to-PDF drivers — routinely rewrite the stream in an order chosen for file size.

Read the third column against the fourth and the table stops being a ranking. The first four rows all answer "where is it right" with a shape of page — one column, ragged baselines, clean gutters — and all four fail by emitting wrong text. The layout-model row breaks that pattern twice, and the break is the informative part. It is the only row whose right-hand condition names a corpus rather than a page shape, and the only one whose failure column quotes a price instead of a wrong output. The first four can therefore be compared on correctness alone. The fifth cannot be compared with them at all until you know your page budget, which is why it is a routing decision rather than a default.

The last row carries a cost that is easy to skip past. A layout model is a document-image detection network — LayoutLMv3, DocLayNet-trained YOLO variants, the models inside Docling — and running one means rasterising each page — rendering it into a bitmap image, at 150 or 200 dots per inch — and pushing that image through a convolutional or transformer stack. On a laptop CPU that is most of a second per page, so a 40,000-page corpus is roughly nine hours of compute for the layout pass alone, before any embedding happens. On a mid-range GPU the same pass falls to 30–60ms per page.

Those two ranges look like a contradiction and are not; they measure two machines, not two opinions, and the ratio between them is about an order of magnitude. The nine-hour figure is worth reconstructing rather than trusting, because it does not come from the middle of the CPU band. Multiply forty thousand pages by the top end of that band and you get eleven hours or so; multiply by the middle of it and you get under seven. Nine hours corresponds to "most of a second", near the upper end — a deliberately pessimistic estimate, which is the right posture for a number whose job is to decide whether a pass runs everywhere or only where it is needed. Neither figure is a reason to avoid the approach, but they are the numbers that decide whether the layout pass runs on every document or only on the sources that need it, which is a question chapter 8 comes back to with measurements attached.

The projection method deserves a note because it is the one most teams write themselves and it works well enough to be dangerous. You build a histogram of glyph density across the x axis, look for a sustained trough, split the page there, and order the regions left to right. On the body pages of the report this is close to perfect. It fails on the first page, where the title and abstract span the full width and there is no gutter above y=560, and it fails on page 22, which holds a wide table whose column of figures happens to leave a vertical white band in exactly the place a gutter would be, so the table is torn in half and re-stitched with its right side after its left.

The checks that come back green either way#

Four diagnostics are usually run on a new source, and on the report every one of them passes. The extraction returns 96,412 characters. No exception is raised. Every word is spelled correctly. Spot-checking the first two hundred characters shows clean English. Read as four independent pieces of evidence that the extraction is sound, they are worth nothing at all, and the reason is worth stating for each one separately.

The character count cannot move. Reordering is a permutation of the extracted characters, and a permutation preserves the count exactly — not approximately, exactly. A correct extraction of this report and a shredded one return the same figure to the last digit. The exception check cannot move either: nothing failed, because every operator in the stream was understood and executed. The spell-check cannot move, because permuting whole lines never misspells a word; every token in chunk 14 is a real English word sitting in the wrong sentence. And the spot-check of the opening is the sharpest case, because it is sampling from the one region where the defect is structurally unable to appear. The report's first page is a title and an abstract spanning the full page width. There is no second column above the gutter, so there is nothing to interleave with, so the first two hundred characters read cleanly in a document whose body is shredded.

The general form is the thing to carry away, because it recurs throughout this topic. A check that returns the same verdict whether the system is right or wrong carries no information about whether the system is right. It does not carry a little information, or weak information. Four such checks are not four pieces of evidence weakly supporting the extraction; they are zero pieces of evidence, run four times. The temptation is real and it is arithmetical: people add green checks together. What has to be asked of each one first is whether its output could have differed had the thing gone wrong.

Reading what the parser produced, next to the page#

The habit worth forming now costs about ten minutes per new document source and pre-empts most of the rest of this topic. Extract one page, dump the words with their coordinates, and read that dump beside the rendered page.

import pdfplumber

with pdfplumber.open("report.pdf") as pdf:
    page = pdf.pages[3]
    words = page.extract_words(use_text_flow=False)

for w in words[:6]:
    print(f"x0={w['x0']:7.2f} top={w['top']:7.2f} {w['text']}")

x0=  72.00 top= 133.48 Recall
x0= 100.31 top= 133.48 at
x0= 111.84 top= 133.48 10
x0= 317.24 top= 133.48 We
x0= 331.10 top= 133.48 thank
x0= 356.02 top= 133.48 the

Two words at x=72 and two at x=317 share top=133.48 to the hundredth of a point. That single line of output is the whole failure, stated in the file's own terms, and it takes longer to describe than to observe. It also tells you the gutter is somewhere between x=290 and x=317, which is the number a column split would need.

Two details in that dump are worth checking rather than skimming, because both look like discrepancies and neither is. The x coordinates 72.00 and 317.24 are the same to the hundredth as the two Tm operands in the raw content stream shown earlier — one recovered by reading the file's own instructions, the other by a library that groups glyphs and reports word boxes. Two routes sharing no processing step landing on identical figures is what licenses using the friendlier tool: whatever pdfplumber is doing to assemble words, it is not moving anything. The vertical figures disagree, and that is the second detail. The stream says the line sits at y=708.52; pdfplumber reports top=133.48. Both are correct and they measure different things — PDF page coordinates run upward from the bottom edge, and pdfplumber reports distance downward from the top. Add the two and you get the height of the page in points, which is how you confirm they describe one baseline rather than two.

Tagged PDF, and the repairs that do nothing#

Reaching for encoding fixes is the most common wrong turn, and it is wrong because every character in the interleaved chunk is correct. Re-extracting with a different Unicode normalisation, or forcing a codepage, changes nothing; there is no mis-decoded byte to fix. It is also not an OCR problem, OCR being optical character recognition, the process of reading characters back out of a picture of a page. Running OCR over a file that already has a good text layer, meaning real character data stored in the file instead of an image of the marks, throws away perfect characters in exchange for recognised ones, and that trade is the subject of the next chapter.

It is not solved by switching libraries either, though libraries differ. pypdf's default extract_text() walks the stream; its extraction_mode="layout" reconstructs a fixed-width visual rendering, which for this report gives you both columns side by side on each line, so the interleave becomes visible rather than fixed. pdfminer.six with layout analysis enabled groups glyphs into text boxes and often gets the columns right. PyMuPDF's get_text("blocks") returns blocks with coordinates and leaves the ordering to you. Each is making a different guess, and none of them is reading a column marker, because there is no column marker to read. A library choice changes which guess you get; it cannot change the fact that a guess is being made.

Tagged PDF is the exception that proves the point. Since PDF 1.4 the format has supported an optional structure tree — /StructTreeRoot — that marks content as /P, /H1, /Table, /TD, in logical reading order, and PDF/UA makes it mandatory for accessibility compliance. When it is present, extraction is a tree walk and everything in this chapter becomes moot. It is present in a small minority of documents in most corpora, concentrated in government and regulated publishing, and it is worth checking for at ingestion because the check is three lines and the payoff is total.

import pypdf

reader = pypdf.PdfReader("report.pdf")
root = reader.trailer["/Root"]
has_struct = "/StructTreeRoot" in root
print(f"tagged: {has_struct}")   # tagged: False, for this report

The check is worth running even though it almost always returns False, and the asymmetry of its payoff is why. A True saves you the entire chapter. A False costs three lines and tells you which branch you are on, which is information you need before choosing between the five rows of the ordering table. It is also the one check in this chapter whose answer does vary with the thing it is testing.

Doing this on a handful of pages from each source teaches the thing that no amount of reading about parsers teaches, which is that reading order is a reconstruction performed by your pipeline, on evidence that is geometric rather than semantic, and therefore a place where your pipeline can be wrong. Every later decision in this topic follows from accepting that: whether to trust a text layer, how to keep a table's rows intact, which parser to buy, what to measure when retrieval misses. None of them makes sense while you still believe the PDF contains paragraphs.

The report at least gave you characters to misorder. The next file in the queue is also a .pdf, also forty pages, and yields thirty-one characters in total, which is a different kind of problem wearing the same extension.