Metadata design and contextual chunk headers

Read these first

• Chunking strategies

What you will know

• Prepend document and section context to a chunk before embedding

  The added context disambiguates pronouns, headings, and references whose meaning would otherwise be lost at chunk boundaries.

• Measure the recall improvement it buys

  Use a fixed, representative query set and compare retrieval with and without headers at the same top-k, while also tracking token and index-size changes.

• Trace a retrieved chunk back to its document, section, and access-control metadata

  Stable identifiers make results auditable and allow downstream filtering or permission checks without trusting text embedded in the chunk.

• Store metadata that later powers filtering

  Fields such as tenant, source, timestamps, and ACL references support filtering, but storing them does not enforce authorization by itself.

• Justify the cost of generating contextual headers

  Per-chunk generation adds ingestion latency, token usage, and vector-storage overhead, so caching, templating, or selective application may be necessary.

Chapters

1. 1

   A chunk's subject often lives in the heading chunking threw away

   One chunk reads only 'it must be requested at least 30 days in advance', and the referent for 'it' sat in a heading the splitter discarded, so the vector lands among every scheduling clause in the corpus. Writing 'Employee Handbook 2024 > Leave > Parental leave' into the text that actually gets embedded restores the subject and the chunk starts retrieving. You build that prepend step and learn which chunks it will help.

   Read chapter
2. 2

   Four hundred shared tokens collapse a document's vectors together

   Prepending a 400-token abstract to all 900 chunks of a document truncates nothing at an 8192-token limit and still makes retrieval worse. Every vector in that document now carries the same 400 tokens, so they cluster toward one point and top-k fills with near-duplicate siblings from whichever document matched first, leaving the right chunk queued behind its own family. Context has to discriminate a chunk from its neighbours, not restate what they share.

   Read chapter
3. 3

   A templated heading path fails on a definable set of chunks

   There are two ways to produce the header, and only one of them costs anything per chunk. The template composes the document title and heading path mechanically: free, deterministic, and reproducible on every re-ingest. Generation asks a model that has seen the surrounding text to write a line saying where this chunk sits and what it is about. What decides between them is the population the template cannot serve — sources with no heading structure at all, chunks whose referent sits in the preceding narrative rather than in any heading, and scanned PDFs or signed contracts where the section path was never extractable. Naming that population is what makes selective generation possible, and what later gives the comparison two arms instead of one.

   Read chapter
4. 4

   Embedded text is searched; payload fields are filtered

   A single release prepended 'Doc: Returns Policy v4 > Section: Damaged goods' to the embedded text and added doc_title, section and effective_date as payload fields, and only the first fixed the damaged-goods query. They are not duplicates of each other: only embedded text enters the similarity computation, only stored fields enter a filter or a sort. A date inside the text cannot express 'since January', and a date in the payload cannot raise similarity.

   Read chapter
5. 5

   A revision id is what makes a quoted rule resolvable

   Someone reports the assistant quoting a retention rule rescinded in March, and the trace holds the answer and the chunk text and nothing that says where either came from. Stable stored fields settle it: document id, revision identifier, section path, and the ACL reference the source system already uses, none of them parsed back out of the chunk text. That turns 'which version said this, and was she allowed to see it' into two lookups.

   Read chapter
6. 6

   A classification string in the text is not a boundary

   'Classification: Confidential | Visible to: finance-team' is read by nothing that can act on it. Vector search returns nearest neighbours to whoever asked, and a model told to honour the marker is following an instruction it can be argued out of, which is not a layer of defence. The decision belongs in a filter on trusted fields applied to the query. Re-embedding on group-membership change puts the fastest-changing fact in the costliest artefact.

   Read chapter
7. 7

   Recall with and without headers at one fixed k, or no number

   Replay a fixed, representative query set at identical top-k against both indexes, and report recall beside token spend, index growth and the context each header consumes per retrieved chunk. Then interrogate the figure: relevance labels regenerated against the new chunk text, or a query set written by the people who designed the headers, will both inflate it. Bundling a top-k raise into the same release destroys attribution for both changes at once.

   Read chapter
8. 8

   Header tokens recur on every embed, query and re-ingest

   One LLM call per chunk across 2.4M chunks is the term everyone quotes and the smallest one in the accounting. Those sixty tokens also enter every embedding call, occupy the index permanently, take context on every retrieved chunk of every query, and regenerate on each nightly re-ingest at 4% churn. Report it per query and per refresh, then run the comparison that decides the design: generated headers against a template from the heading path. Before funding either, test whether a 0.58 is header-shaped at all — read the failing queries and see whether the gold chunk lost its subject, or whether its evidence was cut in half by 800-token windows applied uniformly to product docs, tickets and contracts. No header rejoins what chunking separated.

   Read chapter