Schema nesting depth

Nesting is the one schema choice that arrives without anyone making it. Field names get argued about, enums get reviewed, and the object graph is inherited whole from whatever already exists — the domain model, the database tables, the API response the front end consumes. That inheritance carries a cost that is measurable per field and is not visible anywhere in the response, because a deeply nested schema under strict mode produces objects that are complete, well-formed and valid whatever the values inside them are.

The cost is easiest to see at the moment a deep value is generated. Here is the token stream of one invoice response, cut at the point where the buyer's country begins:

{
  "document": {
    "invoice_number": "INV-2026-04417",
    "parties": {
      "buyer": {
        "name": "Halberd Logistics Ltd",
        "address": {
          "line1": "12 Fetter Lane",
          "city": "London",
          "country": "

Four objects are open and unclosed, each of which the decoder must eventually close in the right order, and the grammar is tracking all four. The model has emitted roughly 40 tokens of structure — braces, quotes, colons, commas and the key names themselves — for one scalar it was asked to extract. invoice_number, at the top of the same object, cost it two.

That count is worth reconstructing rather than accepting, because reconstructing it is how you price your own schema. Reaching country requires five keys to have been written — document, parties, buyer, address, and country itself — and each key costs the tokens of its own name plus an opening quote, a closing quote and a colon. Four objects had to be opened, which is four braces the decoder is still holding open and must eventually close in the right order. And the earlier members of each of those objects had to be emitted first, because a constrained decoder produces properties in schema order, so the buyer's name and the first address lines were written before country became reachable at all. invoice_number needed none of that. It is the first key of the outermost object, so its cost is essentially the key itself, which is the difference between two tokens and forty.

Depth is a cost the model pays per field#

The mechanism is not mysterious and it is not a limitation of constrained decoding. Structure has to be produced as tokens like everything else, and every level of nesting adds a key, an opening brace, and a deferred obligation to close. While the model is choosing the country, the thing it is conditioning on most immediately is a run of address fields; the fact that this address belongs to the buyer rather than the seller is four keys back, and the fact that a buyer country was one of the things the task asked for is further back still.

Grouping 800 invoice responses' fields by depth makes the gradient plain, and it is a gradient rather than a cliff:

Two readings come off that table. The first is that it is a gradient and not a cliff: there is no level at which the model stops working, and therefore no threshold to stay safely under. The second is the rate of the slide. Between depth one and depth two the empty rate roughly doubles, and it roughly doubles again between two and three, after which the growth slows as the numbers get large. Correctness falls in step, from 96.5 percent at the top to 71.0 percent at the bottom, which is about 25 points spread over four levels. Nothing about the schema changed between those rows except how many keys stand between the root of the object and the value.

Separating depth from rarity#

That comparison is not yet trustworthy, and the reason matters more than the numbers. Deep fields in a schema like this are usually also the more obscure fields — a VAT identifier is genuinely absent from more invoices than an invoice number is, so depth and rarity are confounded. The way to separate them is to hand-label a sample of documents for whether each value is present at all, and compute the failure rate only over documents where a human found the value. On 200 invoices labelled that way, the country was printed on 181, and the model returned it on 129 of those 181, which is 71 percent and cannot be explained by absence.

The two divisions there are worth doing side by side, because choosing the denominator is the whole method. Of 200 invoices, 181 had a country printed on them and 19 did not. Scoring the model over all 200 mixes a failure together with a correct abstention, since returning nothing for an invoice that shows no country is the right answer and should not count against anything. Scoring over the 181 asks only about documents where there was something to find, and 129 out of 181 is the 71 percent. Without that labelling step the comparison stays unverifiable in a precise sense: the empty rate on a deep field is two effects added together, and no quantity of extra production data separates them, because every additional document contributes to both. Conditioning on presence is the only operation that removes one of the two, and it costs a person a morning.

Flattening document.parties.buyer.address.country to a top-level buyer_country on the same 200 documents took it to 96.7 percent, missing 6 of the 181. The fact being asked for did not change and the model did not change; what changed is that the answer stopped being the end of a five-step structural commitment and became a single decision with its own name sitting immediately to its left, which is chapter 1's mechanism, on names conditioning values, operating on a field that had been denied it.

The rest of the procedure is what makes that number defensible to the person who owns the writer, because "the deep fields felt worse" is not defensible. Run both schemas on the same documents with the same model, prompt and temperature, and pair the results per document rather than comparing aggregate rates: the documents vary enormously in legibility, and an unpaired comparison of two 200-document runs has enough noise to hide a five-point difference. Report recall per field rather than as one number, since flattening helps the deep fields substantially and does nothing for the top-level ones, and a single averaged figure invites the objection that the whole thing moved by two points. Then test whether the per-field differences are real: with 181 documents where the country was present and a change from 129 correct to 175 correct, a paired binomial test on the discordant pairs — the documents where exactly one of the two schemas got the field right — puts this far past any reasonable threshold. The same test applied to a field that moved from 88 to 91 percent will tell you it did not move at all, and reporting that honestly is what makes the rest of the claim credible.

How far past the threshold is worth bounding, because it can be done without running the test at all. The flattened schema missed 6 of the 181, so at most 6 documents can be ones the nested schema got right and the flattened one did not. The flattened schema got 175 right against the nested schema's 129, so at least 46 documents went the other way. A split of at least 46 against at most 6 does not need a p-value to interpret; the two counts have already settled it, and the test is confirming something the arithmetic makes visible. Run the same bounding exercise on a field that moved from 88 to 91 percent and the wins are a handful against an unknown number of losses, which is the case where the test is doing real work rather than ceremony.

Where the deep schema came from#

Nobody designs a five-level extraction schema from scratch. It arrives by being the same shape as something else — the domain model, the Prisma or SQLAlchemy classes that map the database tables into objects, the API response the front end already consumes — and the argument for keeping it that way is genuinely good as far as it goes. If the model returns the storage shape, the writer is a single insert with no mapping code, no field-name translation table, and no place for the two representations to drift apart. Product's version of this argument is that the schema must match the database shape so the writer stays trivial, and the trivial writer is a real asset.

The instinct behind the storage shape is normalisation, which exists to solve problems the model does not have. Third normal form prevents the same fact being stored in two places and going out of sync under concurrent writes, and it makes joins cheap by giving each entity one home. A model filling a schema writes the whole object once, in a single pass, and then never updates it, so there is no update anomaly to prevent and no join to make cheap. Schema-Driven Prompt Design: Letting Your Data Model Drive Your Prompt Structure makes this case at length: the normalisation instinct is not wrong about databases and is backwards for a schema whose only job is to be filled in by a language model in one shot.

Both halves of that are worth stating as conditions rather than as a slogan, since conditions are what let you tell when the instinct does apply. Normalisation earns its keep when the same fact is written from more than one place over time, because a single home for it is what stops two writers disagreeing; and when the same fact is read from several directions, because a single home is what makes a join cheaper than a scan. A response object is written once, by one writer, in one pass, and is read once by the code that maps it. Neither condition holds, so the structure that exists to satisfy them is being paid for and used for nothing — which is a different claim from saying the storage shape is wrong, because in storage both conditions hold and the same structure earns its keep.

The cost of the deep shape is also visible in the bill. The 800-document run produced an average of 412 output tokens per response under the nested schema and 297 under a flattened one, because the structural tokens — the repeated keys, the braces, the indentation the model tends to emit — are a third of the output when the object is five levels deep. Median latency went from 4.1 seconds to 3.4. Those are secondary to the accuracy difference and they are not nothing, and they point at the same thing: depth is work.

The token figure is a subtraction worth doing. 412 against 297 is a saving of a bit over a hundred tokens per response, which is a bit more than a quarter of the nested output. That is less than the third the paragraph above attributes to structure, and the gap between the two is the structure flattening does not remove: the keys themselves still have to be written, and buyer_country is barely shorter than country was. What disappears is the scaffolding around them — the intermediate key names, the four braces, and the indentation that comes with them.

Where to stop flattening#

The overcorrection is to flatten the whole 60-field schema into one level of scalars and map back in code. It is the right direction taken past the point where it helps, and it fails in three specific ways.

The first is that some fields are only interpretable in a group. A schema with buyer_name, buyer_country, seller_name and seller_country is clear; a schema with name, country, name_2 and country_2 is not, and the flat-everything instinct produces the second one as soon as the entities have more than three attributes each. Prefixing keeps the grouping in the name, which is where chapter 1 argued the model reads it anyway, and it holds up to about four or five attributes per entity before the names get unwieldy.

The second is that a repeated structure cannot be flattened at all. An invoice has line items, and there is no honest flat encoding of a variable-length list of objects. line_item_1_sku, line_item_2_sku and so on requires you to fix the maximum in advance, forces the model to emit a fixed number of slots and decide which to leave empty, and turns "how many line items are there" into a counting problem over null checks. An array of flat objects is the correct shape, and it is worth being clear that the array is not the expensive part — depth inside each element is.

The third is the mapping layer. Flattening does not remove the nested shape from your system, since the database still has it; it moves the translation from nowhere into a function somebody has to own. That function is where the real trade sits, so it is worth pricing honestly rather than pretending it is free.

Two levels is where teams usually land: scalars at the top with entity prefixes, and arrays of flat objects for the genuinely repeated parts.

{
  "invoice_number": "INV-2026-04417",
  "issue_date": "2026-04-02",
  "total_usd": 18422.50,
  "buyer_name": "Halberd Logistics Ltd",
  "buyer_country": "GB",
  "buyer_vat_id": "GB123456789",
  "seller_name": "Cordwain Components GmbH",
  "seller_country": "DE",
  "line_items": [
    { "sku": "CC-4471", "description": "Bearing housing", "quantity": 40, "unit_price_usd": 212.00 }
  ]
}

Sixty fields became forty-eight scalars and one array of five-field objects, and the mapping to the storage shape came out at 41 lines of typed assignment that a compiler checks. The 90-line version of that mapping for a fully flat schema is the same code with more of it, which is why the two-level compromise wins over either extreme.

Flattening also has a regression to check for, because grouping carried meaning that the flat names now have to carry alone. Look specifically for fields whose values swapped between entities — a seller_country holding the buyer's country is the characteristic flat-schema failure, and it does not occur in the nested version where the address is unambiguously inside buyer. On the invoice set this happened on 3 of 200 documents, all of them invoices where the seller block appeared below the buyer block in an unusual layout, and it was the one measurable thing the nested schema did better.

Depth is the cheapest of this topic's levers to reason about and the most expensive to change, because the shape of the response is the shape of everything downstream of it. The rule that survives contact with a real codebase is not "flatten" but "stop mirroring": the schema you hand a model is a request for facts, and it should be shaped like the request rather than like the table those facts eventually land in.

That leaves the topic with a set of levers — names, descriptions, structural types, enum vocabularies, split dimensions, depth — each of which has a characteristic failure. Applied in that order they are a repair procedure for a schema you already know is broken. The more useful skill is the same knowledge run forwards, on a schema you have never seen and no production data for, to say which field will be wrong before anyone has run it.