Exporting a model to JSON Schema

Hand-writing a schema as a dictionary is fine for six fields and unsustainable the moment the same shape also has to exist as a type in the application code. The usual answer is to declare the shape once, as a Pydantic model in Python or a Zod object in TypeScript, and generate the wire schema from it with model_json_schema() or zodResponseFormat(). One declaration, one source of truth, and no drift between what the provider is told and what your code expects.

That is most of the truth. The declaration is used twice, at two moments separated by a network round trip, and the two uses do not see the same information. Knowing which half of a model crosses the wire and which half stays behind is what decides whether a rule you wrote influences generation or merely rejects it afterwards, and the two look identical in a pull request. Everything that participates in the first use is a static description of a shape; everything reachable only in the second is code, and code cannot be sent.

The same object at two different moments#

The first job happens before the request goes out. Ticket.model_json_schema() walks the class and produces a JSON Schema document, which is a static description of a shape: types, properties, required lists, enum members, nullability. That document is serialised into the request body, the provider compiles it into a decoding grammar, and from that point on the class is not involved in anything. It is not called, not imported, and not consulted; the provider has a JSON document and nothing else.

The second job happens after the response arrives. Ticket.model_validate_json(raw) takes the string the provider produced and runs it through Pydantic's validation pipeline, in your process: it coerces types, checks required fields, enforces enum membership again, and — this is the part that does not travel — executes every @field_validator and @model_validator you have written, as Python, against the parsed values.

class Ticket(BaseModel):
    ticket_id: str
    urgency: Literal["low", "medium", "high"]
    assignee: str | None
    due_date: date

    @field_validator("due_date")
    @classmethod
    def not_in_the_past(cls, v: date) -> date:
        if v < date.today():
            raise ValueError("due_date must not be in the past")
        return v

The reason not_in_the_past does not travel is that JSON Schema has no way to express it. There is no keyword for "later than the moment this document is evaluated", and even if you invented one, the provider's grammar compiler would have to evaluate it against a clock at decoding time. What crosses the wire for due_date is {"type": "string", "format": "date"}, and on most providers even the format is advisory — a hint the grammar may or may not enforce down to the digit.

Underneath that is a more general reason, and it is the one that predicts which other rules will fail to travel. A JSON Schema is a document rather than a program. The provider does not run it; it compiles it once into a state machine that has to answer exactly one question at every decoding step — given what has been emitted so far, which token ids are still legal. A rule that can be turned into that answer survives the export, and a rule that cannot does not. not_in_the_past cannot, for two independent reasons: the answer it needs depends on a value the machine has no access to, namely the date at the moment of decoding, and the rule itself is a Python closure rather than a keyword the compiler has a case for. Making it travel would mean the provider executing code you supplied inside its own sampler, which no provider offers.

The same test explains the rows in the table below that are not absolute in either direction. maxLength is expressible as a keyword and is still frequently ignored, because honouring it means the compiler has to know how many characters each candidate token would add to the string built so far and mask every token that would overshoot. The decoder emits tokens, the bound is written in characters, and one token routinely covers several — so the check is real work at every step of every string field, which is why it is among the first things an implementation declines to do. A rule the compiler can answer cheaply is enforced everywhere; a rule it can answer expensively is enforced by some providers; a rule it cannot answer at all never leaves your process, whatever the class it was declared in.

What survives the export#

The useful mental model is that the export keeps everything that can be checked by looking at the shape of a value in isolation, and drops everything that requires running code or comparing two fields to each other. Concretely, for a strict structured-output request:

The last three rows are where teams get hurt, and the descriptions row is where they get hurt most subtly. A description on a field is genuinely useful — the model reads the schema, and a well-worded description improves the values it picks — but it is persuasion, not enforcement, and it sits in the same document as the keywords that are enforced. Two adjacent lines in the same model, one of which is a law and one of which is a suggestion, is a difficult thing to keep straight in review.

The distinction between the two jobs also explains a class of bug that has nothing to do with validators. Property order in the exported schema is the order in which a constrained decoder must produce the fields, because generation is left to right and the grammar admits only one key at each position. Reordering fields in the class is therefore a semantic change to the request, not a cosmetic one — a summary declared before urgency is written before the model has committed to an urgency, and a summary declared after is written knowing it. That is a lever worth remembering; the chapter on required fields and invented values uses it deliberately.

Two facts sit under that claim and both are worth having explicitly, because together they are why the lever works. The first is why the grammar admits one key at a position rather than any of the keys not yet written. A grammar accepting the keys in any order would have to remember which subset had already been emitted in order to know which remain available, and the number of distinct subsets doubles with every field added, so an eleven-field object needs thousands of states where a fixed order needs eleven. Implementations take the cheap option and pin the order to the schema's, which silently promotes a declaration order into a generation order. The second is that decoding is append-only. A token that has been emitted is part of the context and cannot be revised, so a model writing summary first is not producing a draft it will reconcile against the urgency later. It is producing a sentence that every field below it then conditions on, written without the benefit of any of them.

Two characters that quietly make a field optional#

The most common way to break an export has nothing to do with validators, and it is a default value:

class Ticket(BaseModel):
    ticket_id: str
    assignee: str | None = None      # the ` = None` is the whole problem

Python reads = None as a default. Pydantic reads it as this field need not be supplied, and the export says so:

{
  "type": "object",
  "properties": {
    "ticket_id": {"type": "string"},
    "assignee": {"anyOf": [{"type": "string"}, {"type": "null"}], "default": null}
  },
  "required": ["ticket_id"]
}

assignee has left the required array. On OpenAI's strict mode that document is refused before any tokens are generated, with a 400 whose message is unusually clear about the rule: Invalid schema for response_format 'Ticket': In context=(), 'required' is required to be supplied and to be an array including every key in properties. That is the good outcome, because the failure is at request time and names the field.

The bad outcome is a provider that accepts the loose export, or a rotation member running in JSON mode, where an unrequired field is one the model may simply decline to produce. Nothing rejects the response; model_validate_json fills the default and hands back a Ticket whose assignee is None. On a 2,000-ticket backfill run against a provider in that configuration, assignee was populated on 0 of 2,000 records, including the several hundred tickets whose body named an owner in the first line. The field had been declared, exported, requested, never generated, and defaulted, and every record validated.

The chain from two characters to that result has four links and every one of them is checkable. = None makes the field non-required in the export. A key absent from required is a key the schema says may or may not appear, so the compiler gives the grammar an extra edge: at the position where assignee would begin, both the key itself and the comma-or-brace that skips past it are legal continuations. Two legal edges mean the choice belongs to the model and is made the way every other choice is made, by likelihood — and on a ticket whose body names nobody, the continuation that omits the field is the likelier one. Then model_validate_json supplies the default for the missing key, which is what a default is for, and hands back an object that satisfies the class. No link in that chain is a defect. The declaration asked for a field the model was permitted to skip, and every component downstream honoured the request exactly.

The underlying mix-up is that optional in Python and nullable in JSON Schema are different claims. Optional means the key may be absent from the document; nullable means the key is present and its value may be null. Constrained decoding cares about the first, because a key that is not required is a key the grammar permits the model to skip. Writing assignee: str | None with no default gives you a field that is required and nullable, which is the combination that makes the model produce the key on every response and lets it say the value is empty.

The same split, in Zod#

Nothing above is specific to Python; the two jobs and the lossy edge between them look identical in a TypeScript service, which is worth seeing because the syntax hides the seam in a slightly different place:

import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const Ticket = z
  .object({
    ticket_id: z.string(),
    urgency: z.enum(["low", "medium", "high"]),
    assignee: z.string().nullable(),
    due_date: z.string().date(),
  })
  .strict()
  .refine((t) => t.due_date >= new Date().toISOString().slice(0, 10), {
    message: "due_date must not be in the past",
    path: ["due_date"],
  });

const resp = await client.chat.completions.parse({
  model: "gpt-4.1",
  messages,
  response_format: zodResponseFormat(Ticket, "ticket"),
});
const ticket = resp.choices[0].message.parsed;   // the refinement runs here

zodResponseFormat converts the object to JSON Schema and reshapes it into the strict subset, so .strict() becomes additionalProperties: false, z.enum becomes an enum array, .nullable() becomes a two-member type, and z.string().date() becomes {"type": "string", "format": "date"} with the same advisory status it has in Python. The .refine disappears in that conversion for the same reason the field validator does — it is a closure, and there is no keyword to compile it into — and it executes on .parsed, in your process, after the tokens have been paid for.

Where the rule has to live for it to hold#

Given all that, a rule like not_in_the_past holds exactly where it executes, which is inside model_validate_json in your process. That is a real enforcement point and it is worth keeping. It converts a bad value into a ValidationError at a known line rather than into a stale date in the database, and the only broken thing about it is ever expecting it to have worked earlier.

If you want the rule to influence generation rather than only reject it, you have to encode it in something the decoder can see. Three moves, in descending order of reliability.

Reshape the schema so the illegal value is not expressible. A due date that must be one of the next 90 days can be an enum of 90 date strings, generated at request time, and then a past date is unreachable in the same way an out-of-enum urgency is unreachable. This is heavy-handed and it genuinely works; it also inflates the schema, at a cost the last chapter of this topic quantifies.

Put the constraint in the prompt and in a field description, accepting that it is advisory. Field(description="ISO date, today or later; never a date in the past") measurably reduces the rate — on one ticket-extraction service it took stale dates from 4.1% of responses to 0.6% over a 2,000-call sample — without ever taking it to zero.

Keep the Python validator as the enforcement point and decide what happens when it fires: reject, repair by clamping to today, or retry with the error message fed back to the model. That last option is a repair loop, and it is a different design with its own failure modes. The three are not alternatives; a production service usually runs the first or second alongside the third.

Testing the contract instead of declaring it#

A schema that has never been run against a bad payload is a claim, and the cheap way to turn it into a fact is a table of representative invalid cases checked against the same model the provider is being sent. These tests cost nothing to run, do not call the API, and catch the case where somebody changes a field type and does not notice that the enum it feeds no longer matches the router downstream.

import pytest
from pydantic import ValidationError

VALID = {"ticket_id": "88214", "urgency": "high",
         "assignee": None, "due_date": "2026-09-01"}

INVALID = [
    ("extra key",        {**VALID, "confidence": 0.9}),
    ("missing required", {k: v for k, v in VALID.items() if k != "urgency"}),
    ("enum outside set", {**VALID, "urgency": "critical"}),
    ("wrong type",       {**VALID, "ticket_id": 88214}),
    ("null on non-null", {**VALID, "urgency": None}),
    ("past due date",    {**VALID, "due_date": "2021-03-04"}),
]

@pytest.mark.parametrize("label,payload", INVALID, ids=[c[0] for c in INVALID])
def test_rejects(label, payload):
    with pytest.raises(ValidationError):
        Ticket.model_validate(payload)

def test_schema_is_strict_compatible():
    schema = Ticket.model_json_schema()
    assert schema["additionalProperties"] is False
    assert set(schema["required"]) == set(schema["properties"])

Five of those six cases are also enforced by the decoder, which makes the test look redundant until the day a provider in your rotation is not enforcing anything. The sixth — the past due date — passes the decoder and fails here, and having it in the same parametrised list is the clearest documentation on the team of which half of the contract runs where.

Strict mode narrows JSON Schema as well#

Strict mode does not merely fail to carry your validators; it also refuses parts of JSON Schema itself, which surprises people the first time an export is rejected. On OpenAI's strict json_schema, every property must appear in required, additionalProperties: false must be set on every object, and a field that is genuinely optional has to be expressed as a union with null rather than by omission. Pydantic's default export does not satisfy those constraints, which is why client.beta.chat.completions.parse(response_format=Ticket) exists and why wrapper libraries such as Instructor are popular — they reshape the export into the provider's accepted subset and hand you back a validated instance. Using one does not change anything in the table above; it changes who writes the reshaping code.

One Zod feature deserves more suspicion than a refinement. A .transform() also fails to travel, and it additionally changes the type on the way out, so the object your code holds is not the object the schema described: a transform that parses due_date into a Date leaves the wire contract saying string while the inferred TypeScript type says Date. Keeping transforms in a separate layer, applied after validation, keeps one declaration honest about what is actually on the wire.

The general defence is to stop treating the class as the contract and start treating the export as the contract. The class is a convenience that generates one; asserting on what it generates, and listing the rules that it cannot generate, is the only way the two stay in a known relationship. Everything checkable on a value in isolation crosses the wire and is enforced before a token is sampled. Everything else is enforced in your process, on data you have already paid for, and that is a perfectly good place for it as long as nobody believes otherwise.

What none of this addresses is the response that satisfies the grammar the whole way through and still will not parse, because it never reached the end. The four truncations in the previous chapter's fixture run were exactly that, and they behave differently from everything discussed so far.