Schema validation and truncated output

Structured output feels like it should close the gap. You declare a schema, the provider constrains generation to it, a validator confirms the response conforms, and the object that comes out the other end has every field the contract promised. If the response had been cut off, surely something in that chain would notice.

Nothing in that chain is looking. This chapter is about the precise scope of what a validator asserts when it passes, why the field most likely to be truncated is also the field most likely to be optional, and why the field-level rule that everyone writes first — reject an empty notes — is wrong about the domain and decays within two quarters of being deployed.

What a validator asserts when it passes#

Run a three-field extraction result through Pydantic — the library most Python services use to declare the shape they expect and check incoming data against it — and it will confirm four things: id is present and is a string, total is present and is an integer, notes is present and is a string, and no field of a wrong type slipped in. That is the entire scope. It says nothing about whether notes is the notes, whether the extraction ran to completion, or whether anything at all was read from the source document.

from pydantic import BaseModel

class Extraction(BaseModel):
    id: str
    total: int
    notes: str = ""          # optional, defaults to empty

Extraction.model_validate({"id": "A-1", "total": 4200, "notes": ""})
# Extraction(id='A-1', total=4200, notes='')

Extraction.model_validate({"id": "A-1", "total": 4200})
# Extraction(id='A-1', total=4200, notes='')   ← the key was never emitted

The second call is the more interesting one. The model never got as far as producing a notes key at all, and the default turned that absence into an empty string. Two genuinely different situations — the invoice has no notes, and we never found out whether the invoice has notes — arrive at the same value, and by the time the object reaches the next function they are indistinguishable. The default is a convenience that erases the distinction you most need.

Nothing about this is specific to Pydantic. The same object run through a raw JSON Schema validator passes for the same reasons, and the schema that describes it is the one most people would write:

{
  "type": "object",
  "properties": {
    "id": { "type": "string" },
    "total": { "type": "integer" },
    "notes": { "type": "string" }
  },
  "required": ["id", "total"],
  "additionalProperties": false
}

notes is absent from required because an invoice may genuinely have none. That single, correct modelling decision is what makes a truncated object indistinguishable from a legitimate one, and no rewrite of this schema fixes it without asserting something false about invoices.

The right-hand column is not a list of validator shortcomings to be fixed by a better library. A JSON Schema, a Pydantic model and a protobuf definition all describe documents. No schema language can express "the generator was not interrupted", because the object holds no evidence either way.

The cleanest way to hold that is to ask what a passing validation rules out. It rules out every document that a correct run could not have produced. It does not rule out — and no strengthening of it could rule out — a document that a correct run could have produced but in this instance did not. Those two sets overlap, and truncation survives inside the overlap. {"id": "A-1", "total": 4200, "notes": ""} is exactly the document a complete extraction of a note-free invoice would have produced, so there is no property of it left over to carry the difference. The previous chapter's case is the same argument reaching the opposite conclusion. An unterminated string is a document no complete run could produce, because a complete run closes whatever it opened, and that is the entire reason a strict parser catches it. So detection from the artefact works precisely when the cut pushes the artefact outside the set of things a correct run could have produced, and fails precisely when it does not. Closing the object on the model's behalf — by a repair pass, or by constrained decoding that force-closes — is the step that moves a case from the first situation into the second.

There is a protocol most backend engineers already trust that makes the contrast concrete. An HTTP response with a Content-Length header is checkable: read fewer bytes than the header promised and you know, from the protocol rather than from the payload, that the transfer was cut short. A chunked response is checkable for the same reason, because the terminating zero-length chunk is part of the framing. That framing is what lets an HTTP client distinguish a short body from a short transfer, and it is why nobody has ever tried to infer transfer completeness by inspecting the HTML.

Model output has no framing of that kind. The model does not know how long its answer will be, so no length can be declared in advance, and the closing brace of a JSON object is not a frame terminator — it is a character the model emits, or does not. The finish reason is the framing, sitting one level up in the response envelope rather than inside the payload. Treating it as optional is treating the framing as optional, and every consequence in this topic follows from that one decision.

There is a reason the framing can only live up there, and having it closes off the wish for a better format. A frame terminator has to be written by something that knows the transfer finished. The model does not know that; it is inside the generation, producing one token at a time, and the token it produces at the end of a complete answer is chosen for the same reasons as every other token. The serving layer does know, because it is the thing that stopped the loop and therefore the thing that knows why. Whatever it writes about the ending is necessarily written after the payload is closed, which means it cannot be inside the payload. The finish reason is not sitting in the envelope by convention or by an API designer's preference. That is the only place a statement about how the generation ended can physically be.

Why the optional field is where it hides#

The concentration of these failures in optional free-text fields is not coincidence, and the mechanism predicts which of your fields will be affected next.

Models emit an object's keys in roughly the order the schema or the prompt presents them, and constrained decoding makes that ordering strict. So the last field in your schema is the last field generated, which makes it the field a token cap hits first. Meanwhile the fields that end up last in a schema tend to be the free-text ones — notes, summary, rationale, description — because that is how people write schemas: identifiers first, numbers second, prose last. Those same fields are the longest, so they consume the most of the remaining budget. And they are the fields most likely to be marked optional, because a document legitimately might not have notes.

The field most likely to be truncated, the field most likely to be optional, and the field whose absence is most likely to be tolerated are therefore the same field. That is not a coincidence you can design your way out of by reordering the schema; putting notes first only moves the truncation onto whatever is now last. It does tell you where to look first in an existing service.

Ordering the fields is also ordering the risk, and the ordering is strict. The budget is spent from the beginning of the object forwards, so a field is reached only if every field before it has already been generated in full. A cut therefore cannot damage the third field while leaving the fourth intact — the fourth was never started. Read down your schema and you are reading a list in decreasing order of how likely each field is to survive, with the first entry safest and the last entry the one that absorbs every truncation the cap produces. That is why the advice to look at the last field first is not a heuristic about where bugs tend to be found. It is the only place a truncation can be.

Permissive schema settings compound it. A default value converts absence into a plausible value. Optional[str] with a None default converts it into a null that a downstream or "" will flatten. additionalProperties left open means a response that emitted nothing but {"id": "A-1"} under a schema with no required fields validates cleanly. Each of these decisions is defensible on its own; together they build a path along which a two-field response walks into a warehouse table as a three-field record.

The same failure has a more expensive variant in arrays. A schema with line_items as a list of objects and no minItems will happily accept a truncated extraction that captured the first three of eleven line items, because three is a valid length for an unbounded array. The total will not reconcile against the sum of the items, which means the error is at least detectable by a downstream consistency check — but only where one exists, and only after the row is written. An empty notes reconciles against nothing at all.

The difference between those two cases has a name, and it is the one lever the document side of this actually offers. The line-items case is detectable because the object carries the same fact twice: once as a total, and once as a set of amounts that ought to add up to it. Redundancy of that kind shrinks the set of documents a correct run could produce — a correct run produces only the objects where the two agree — and a cut that removes items breaks the agreement, which pushes the object out of that set and makes it visible. notes carries no second copy of anything. Nothing else in the object constrains what it should have contained, so removing its content moves the object nowhere it should not be. Where a schema does happen to contain a redundant pair, a cross-field invariant is worth writing and will earn its keep. Be clear about what it buys, though: it catches truncation only in the fields the redundancy happens to cover, only after the object exists, and only for schemas lucky enough to have redundancy in them at all.

Forcing the failure in your own code takes four lines and is worth doing before arguing about it with anyone:

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=40,                       # low enough to cut inside notes
    system="Extract id, total, and notes as JSON.",
    messages=[{"role": "user", "content": invoice_text}],
    output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
)

assert response.stop_reason == "max_tokens"

record = Extraction.model_validate_json(response.content[0].text)
print(record)                            # id='A-1' total=4200 notes=''
save(record)                             # accepted, written, invisible

A response the API told you was cut off, accepted by validation, written to storage. Keep that snippet: it turns "I think we might have this problem" into "here is our code accepting a truncated response", and the second sentence gets a fix prioritised while the first gets a shrug.

Tightening the schema trades one wrong answer for another#

You can tighten the schema rather than the model class, and it fails in the same direction. Adding minLength: 1 to notes, or minItems: 1 to line_items, moves the failure earlier and makes it loud. It is the same trade the field validator makes, with the same cost: the constraint is now asserting a fact about the domain, and the domain will eventually produce a counterexample. A schema that says every invoice has notes will be wrong about 8% of invoices, and the truncated ones are not distinguishable from the legitimate ones by any property of the object.

There is a second problem with a per-field rule that is worse than the false failures. It protects exactly one field. The next time a cap is undersized, or a document is unusually long, or a model migration shifts token counts by 30%, the truncation may land in line_items or vendor_address or a field added next quarter, and this validator will not notice. Every field would need its own rule, each rule would need its own carve-out for legitimately empty values, and the whole apparatus would be an elaborate way of inferring from the document something the API already reported in a single field on the response.

Every technique in this chapter has been an attempt to detect, from the shape of the output, a fact about the process that produced it. The attempt cannot be made to work — not with stricter schemas, not with field rules, not with heuristics about empty strings — because the fact is not in the document. Validation remains worth doing for what it does assert: types, required keys, enum membership, the absence of junk. It simply answers a different question from the one truncation poses.

The answer to that question was on the response the whole time. The finish reason was max_tokens on all 1,247 of those extractions, and end_turn on the 25,000 invoices that genuinely have no notes. One field separates them cleanly, with no false positives and no per-field maintenance, and it existed on every response from the beginning.

Which leaves one question, and it is the one that actually determines whether a fix holds: not what to check, but where in a system the check belongs.