Flattening a table deletes the binding between header and cell

A number in a table means nothing on its own. 38.6 is a quantity; what makes it a fact is the row it sits in and the column standing above it. A table is a device for storing those two labels once each rather than repeating them on every cell, and that economy is the entire reason tables exist. Flattening is what an ordinary text extractor does to the device: it visits the cells, emits their contents in reading order separated by whitespace, and discards the grid. Every character survives the trip. The two labels do not, because they were never inside the cells — they were held in the geometry, and a run of text has no geometry.

The reason this belongs in a topic about parsing rather than in one about prompting is that a chunk of text is the whole of the evidence a language model receives. Whatever binding between value and label is missing from that text is missing from the model's world, and the model will supply a replacement from the only signal it has left, which is adjacency. The damage is worst where the grid was doing the most work: under a multi-level header, meaning a header built from two or more stacked rows where an upper cell spans several columns and the row beneath it subdivides that span. Financial packs, clinical tables and regulatory returns are built almost entirely out of those.

What a merged header does to a run of text#

Here is a table of the kind a quarterly pack is full of, printed on page 7 of one:

And this is what an ordinary text extractor emitted for it:

Region FY2022 FY2023 Q3 Q4 Q1 Q2 Q3 AMER 61.3 64.8 66.0 69.4 74.1 EMEA 22.4
24.9 27.1 31.8 38.6 APAC 26.0 29.3 33.5 37.7 41.2 LATAM 8.1 8.4 9.0 9.6 10.2

Every one of the twenty figures survived, spelled correctly. Twenty is not a coincidence and it is worth doing on paper: four region rows times five quarter columns is twenty body cells, and the run contains twenty numbers. The count closes exactly, with nothing left over. That exactness is the trap. A check that counts figures, a check that compares character totals before and after, a grep for 38.6 — each of them returns the identical verdict whether the binding survived or not, because none of them looks at anything the flattening touched. A defence that gives the same answer in both of the cases it is meant to tell apart is not a weak defence; it is not a defence at all.

The grid did not survive, and the arithmetic of why is short. The top header row holds three labels: Region covering one column, FY2022 covering two, FY2023 covering three. Those widths sum to six, which is exactly the number of columns in the table, with no column uncovered and none covered twice. The second header row holds six entries — a blank under Region, then five quarter labels — each covering one column, which also sums to six. Two rows, each tiling the same six columns exactly, and the only thing that distinguishes them is the widths. Flattening writes out the labels and throws away the widths.

That is why the header emits eight tokens where the body rows emit six. Three labels from the top row plus five from the second row is eight; a body row has one region name and five figures. The surplus of two is not noise, and it can be named: it is one extra token for each header cell that spanned more than one column, which is to say one for FY2022 and one for FY2023. There is no line structure left in the run to realign the eight against the six, and the string Q3 appears twice with nothing to distinguish the two occurrences, because the thing that distinguished them was the year label two positions up and one row back.

The dropped widths also explain a smaller detail that surprises people. The run above is 43 tokens, which is more than the number of whitespace-separated items in it — counting those gives thirty-two, being the eight header items plus six for each of four body rows. Both figures are correct and they measure different things. Whitespace items are what a person counts by eye; 43 is what a language model's tokeniser counts, and it is larger because a decimal figure such as 61.3 is broken into several subword pieces before the model ever sees it. Any budget stated in tokens has to be compared against other token counts, never against a word count.

A model handed that run reconstructs the table by proximity, which is the only evidence available to it, and proximity puts 41.2 four tokens after a Q3 and inside a run that began with a region name. Nothing in the text contradicts that reading, so nothing in the model's behaviour will flag it.

The information was destroyed upstream of the model#

The instinct on seeing this is that the table was chopped across chunk boundaries, and the fix is to detect table regions and guarantee each one lands whole in a single chunk. That is a reasonable thing to do for other reasons and it does not touch this defect, because the table was whole in a single chunk. The run of text above is 43 tokens; it fits in any chunk size anyone uses. Enlarging the window around information that is already gone adds nothing, and the same argument disposes of raising k, widening the context, or feeding the model the entire page.

Each of those moves is a knob that increases the amount of text reaching the model. The step that is easy to skip is asking what such a knob can and cannot change. It changes which characters are present; it cannot change what those characters encode. Since the row and column identity was not encoded anywhere in the run, every setting of every one of those knobs produces the same wrong answer, and an experiment that turns them and sees no improvement has learned nothing about whether they were the right knobs.

The second instinct is to fix it at generation time — instruct the model to read tables carefully, and require it to cite the row and column label it used. The instruction has no purchase, since the model is not being careless; it is inferring from the only signal present. The citation requirement is actively harmful. Asked to name the row and column for 41.2, the model produces "EMEA, FY2023 Q3", because that is what the question presupposed and nothing in the context contradicts it. An unsupported number has been converted into a number that looks sourced, and the reviewer who was going to sanity-check a bare figure now sees a citation and stops.

That is the same shape as the counting checks above, one layer higher up. A citation requirement is meant to discriminate between a figure the model read off a label and a figure it guessed from position, and against a flattened table it returns a confident, well-formed citation in both cases. The requirement is constant while the thing it is supposed to detect varies, so satisfying it carries no information — and it costs something, because it moves a human reviewer from suspicious to satisfied.

Both fixes share a shape: they act on the consumer of a representation that has already lost the information, and no consumer can recover what the representation does not carry. The defect is that row and column identity was destroyed at parse time. That is the sentence to reach for when someone describes this as a hallucination, because it locates the work in the right place.

What the retriever needs to end up holding#

The repair runs in three steps, all of them before chunking. Extract the table as cells with geometry and spans — how many rows and how many columns each cell covers — rather than as text. Resolve the multi-level header into one fully-qualified label per column. Emit each row as a self-contained record that carries its labels inline.

A layout-aware extractor gives you the first step, layout-aware meaning that it works from where marks sit on the page, labelling regions as tables, headings or body before pulling any text out of them, instead of concatenating draw operations in the order it meets them. pdfplumber will do it on ruled tables, camelot on ruled and some unruled ones, and a layout parser such as Docling returns a cell model with spans for both. What matters is that the output is a list of cells with row index, column index, row span and column span, rather than a string.

import pdfplumber

with pdfplumber.open("q3-pack.pdf") as pdf:
    tbl = pdf.pages[6].find_tables()[0]
    grid = tbl.extract()          # list of rows, each a list of cell strings

for row in grid[:2]:
    print(row)

['Region', 'FY2022', None, 'FY2023', None, None]
['', 'Q3', 'Q4', 'Q1', 'Q2', 'Q3']

The Nones are the continuation columns of a merged cell, and they are the whole reason the flattened version misaligned: dropping them collapses eight header positions into five. Forward-filling them restores the span, and joining the two header rows gives a label per column that is unique inside the table.

Forward-filling deserves a moment, because it looks like a heuristic and is not. A heuristic would be a rule that is usually right. This one is exact, and the reason is the sum that closed above: because the span widths tile the six columns with no gap and no overlap, every None has exactly one non-None cell to its left that owns it, and there is never a second candidate. Carrying the last seen label rightwards is therefore the precise inverse of the merging that produced the Nones. Where that guarantee fails — a grid extractor that returns None for a genuinely empty cell as well as for a continuation — the inverse stops being exact, and the symptom is a blank cell inheriting its neighbour's label.

def resolve_headers(header_rows: list[list[str | None]]) -> list[str]:
    filled = []
    for row in header_rows:
        out, last = [], ""
        for cell in row:
            last = cell.strip() if cell and cell.strip() else last
            out.append(last)
        filled.append(out)
    return [
        " ".join(dict.fromkeys(p for p in parts if p)).strip()
        for parts in zip(*filled)
    ]

columns = resolve_headers(grid[:2])
print(columns)

['Region', 'FY2022 Q3', 'FY2022 Q4', 'FY2023 Q1', 'FY2023 Q2', 'FY2023 Q3']

Two details in that function are doing work that is invisible from its output. zip(*filled) transposes the filled header rows, so each iteration of the final comprehension holds one column's stack of labels top to bottom — which is the arrangement the label has to be built from, and the arrangement the flattened run destroyed. dict.fromkeys removes duplicates while keeping order, and it is there for the first column: Region forward-fills into the top row and the second row's blank forward-fills to nothing, so without the de-duplication a column whose two levels carry the same text would come back with that text twice.

FY2022 Q3 and FY2023 Q3 are now distinct strings, which is the property the original representation lacked. From here each data row becomes a set of records, each one a sentence that would still be unambiguous if it were the only thing a reader ever saw.

def emit_records(grid, columns, caption, doc_id, page):
    stub, value_cols = columns[0], columns[1:]
    for row in grid[2:]:
        label = (row[0] or "").strip()
        if not label:
            continue
        cells = ", ".join(
            f"{col} = {(val or '').strip()}"
            for col, val in zip(value_cols, row[1:])
            if val and val.strip()
        )
        yield {
            "doc_id": doc_id,
            "page": page,
            "table_id": "t7-1",
            "row_label": label,
            "text": f"{caption}. {stub}: {label}. {cells}.",
        }

Revenue by region, $M, unaudited. Region: EMEA. FY2022 Q3 = 22.4,
FY2022 Q4 = 24.9, FY2023 Q1 = 27.1, FY2023 Q2 = 31.8, FY2023 Q3 = 38.6.

That record embeds as a statement about EMEA, retrieves for a query naming EMEA and Q3, and puts the string FY2023 Q3 = 38.6 in front of the model with the binding intact. The caption is carried inline because units and scope live there and are lost the moment the row is separated from the page — a bare 38.6 is meaningless without "$M" and "unaudited", and captions are exactly the kind of adjacent text that flattening strips.

One row per record is a design decision rather than something the problem forces. What the problem forces is only that every value reach the model accompanied by both of its labels; a record per cell would satisfy that too, and so would a record per column. Rows win because retrieval queries in this domain name an entity and a period — "EMEA's Q3" — and a row record is the smallest unit that contains both, while a cell record multiplies the index size by the number of value columns for no gain in what the model can be told.

That table is more useful read as a set of gaps than as a set of levels. Four of its six rows come apart between the two columns, and those four are the repair. The two that do not come apart sit at opposite ends of the argument. "Figures preserved" is locked at "All 20" on both sides, and it is the only row a cheap automated check ever looks at — which is precisely why the defect ships and survives review. "Supports 'which region grew fastest'" is locked at "Poorly" on both sides, and it names the thing the repair does not buy. A comparison whose only agreeing rows are the one everybody measures and the one nobody fixes is a comparison worth reading twice.

The last two rows are the cost, and they are real. Row records are roughly five times the tokens of the flat version, and they fragment exactly the comparisons a table exists to support: no single record can answer "which region grew fastest between Q1 and Q3", because the growth comparison spans four records that may not all be retrieved. The practical arrangement is to store both — the row records for retrieval, plus one chunk holding the whole table rendered as markdown with the resolved headers, indexed under the same table_id — and to fetch the full-table chunk whenever any row record from that table is retrieved. That expansion is a lookup by table_id, not a similarity search, so it costs nothing in recall.

For tables that are genuinely numeric and get asked numeric questions, there is a further step worth knowing about even though it belongs to a later topic: keep the resolved grid as an actual table in a database, give the model a query tool over it, and let retrieval find the table while arithmetic happens in SQL. Row records make the numbers retrievable and quotable; they do not make the model good at summing a column.

Finding out when it breaks again#

The fix ships and then the source document changes. A new template adds a third header level, or the FY2024 pack moves the region stub — the leftmost column, the one whose cells name each row — to the right-hand side, or the table gains a total row that the header resolver mistakes for data. None of these raise an exception, and all of them reintroduce the original defect in a form that looks identical from the outside.

Two checks catch it, and they operate at different levels.

def assert_bindings(records, expectations):
    """expectations: {(row_label, column_label): value}"""
    index = {r["row_label"]: r["text"] for r in records}
    for (row_label, column_label), value in expectations.items():
        text = index.get(row_label)
        assert text, f"no record emitted for row {row_label!r}"
        assert f"{column_label} = {value}" in text, (
            f"{row_label}/{column_label}: expected {value}, "
            f"record reads: {text}"
        )

The first is a named regression case, in the same eval set that holds retrieval traps: the query "what was EMEA's Q3 FY2023 revenue", pinned to table_id t7-1, asserting that the answer contains 38.6 and does not contain 41.2. Asserting the absence of the neighbouring value is what makes the case sensitive; an answer that says "38.6" is right, and an answer that says "EMEA 38.6, APAC 41.2" is a model hedging its way past a broken binding, and you want to know about that too.

The second is a structural invariant that runs on every table at ingestion, without any hand-written expectations. Take the resolved grid, sample three cells at random, and assert that the emitted records contain each (row_label, column_label, value) triple. This is a self-consistency check between two representations of the same extraction rather than a check against the printed page, so it cannot catch a cell the extractor read wrongly — it catches the header resolver drifting, the stub column moving, spans being dropped, which is the class of failure that actually recurs. On a corpus of 3,200 tables it added about 40 seconds to the ingestion run and, the first time the vendor changed template, failed on 61 tables in one batch with a message naming the row label it could not find.

Sampling three cells out of twenty sounds far too thin, and the reason it is not turns on what the recurring failures do. A header resolver that drifts, a stub column that moves, a span that is dropped — each of these mis-binds every cell in the table at once, so a single sampled cell detects it with certainty and three is generous. A single misread digit affects one cell in twenty, and three draws will usually miss it. The sample size is calibrated to the systematic class and is deliberately blind to the sporadic one, which is the same division of labour as the two checks themselves. Spread over that corpus the cost works out at a small fraction of a second per table, which is what makes it affordable to run on all of them rather than on a sample of tables.

Where a total row exists, the strongest check available is arithmetic: sum the region records for a column and compare against the printed total, with a tolerance for rounding. It catches misalignment, dropped rows and misread digits in one assertion, and it is free. The table above does not have one, and its absence is the reason the sampled-triple invariant has to exist at all — a pack that prints its totals hands you a check that needs no code and no expectations, and a pack that does not leaves you assembling a weaker substitute out of self-consistency.

None of this makes tables safe in general. A table whose meaning depends on a footnote marker, or on a column that is only labelled in a legend three pages later, or on shading that distinguishes actuals from forecasts, still loses that meaning in any text representation, and the honest response is to know which of your sources contain such tables rather than to assume the pipeline handles them. Measured on 60 hand-checked tables from this corpus, the row-record pipeline bound the correct row and column label to a value on 91% of cells; the residual 9% were concentrated in three shapes — cells spanning two rows, header cells whose text wrapped onto two lines and came back as separate cells, and tables with no ruling lines where the column boundary had to be guessed from whitespace.

That residual is worth naming precisely, because it says where the remaining work is not. All three shapes are failures to recover the geometry — a row span misread, a header cell split in two, a column boundary guessed wrong. None of them is a failure of the header resolver or of the record emitter, which means a better resolver improves nothing in that nine percent. The lever that moves it is the extraction step above, and on ruled tables that lever is usually a different extractor rather than better code.

The binding between a figure and its labels is not stored in the figure, and nothing downstream of the parser can reconstruct it once the grid is gone. That single fact disposes of most of what teams try first — larger chunks, a larger k, a longer context, a sterner prompt, a citation requirement — and it points at the one place where the repair is cheap, which is the representation written at ingestion. Rows carrying their own labels cost about five times the tokens, fragment cross-row comparisons, and are worth it, because a chunk that cannot be misread is worth more than a chunk that is merely short.

PDF has occupied four chapters because it withholds structure by design. The other formats in a typical corpus have the opposite property: HTML, DOCX and slide exports hand you headings, real table cells and explicit styles, and the problems they cause come from everything else they hand you at the same time.