A schema rejects a bad call before dispatch; a regex does not

Two designs sit between a model's request for an action and the function that performs it. One is the hand-parsed Thought:/Action:/Observation: loop, where a regular expression pulls a tool name and an argument string out of the model's prose. The other is native tool calling, where the request carries a machine-readable description of each tool and the reply carries the model's request as a typed field. On a well-formed call the two are indistinguishable, which is why a working demo says nothing about the choice, and why the comparison has to be made on calls that are wrong.

When a call is wrong, the designs differ in one place, and every other difference follows from it: where along the path from the model's output to your running code the failure surfaces. A schema is a description of a valid call that exists as data before the call arrives, so a violation can be detected at the boundary while nothing has yet happened. A regular expression is not a description of a valid call. It is a description of a string, and the string matched, so there is nothing left to check and the extracted values go straight to the function. Boundary or interior is the whole of the difference, and it decides what a failure costs, what you can tell the model afterwards, and what you have in your hands at two in the morning.

Three abuses, run against both designs#

The comparison worth doing is not "which one is nicer to write" but what each does when the model's output is wrong in a specific way. Three kinds of wrongness cover most of what happens in production: an argument whose value cannot be interpreted, a required argument that is absent, and an argument that was never part of the tool at all. Feed all three to a hand-parsed loop and to a native tool call against the same underlying function.

One specimen of the first abuse makes the mechanism visible before the declaration arrives, and it is worth walking rather than summarising. A support agent asked an assistant to open a case titled "Charged twice, then refunded", and the model wrote exactly what the prompt had shown it:

Action: create_ticket[cus_8812, Charged twice, then refunded, high]

Follow that line through the two operations the parsed loop performs on it, one at a time. ACTION_RE matched, because the line is a tool name followed by a bracketed argument string, and that shape is the whole of what the pattern asks about. The argument string was then split on commas, because a comma is the only separator the protocol has. Splitting produced four values rather than three, since the subject a human wrote happens to contain one. create_ticket(customer, subject, priority) was therefore called with a fourth positional argument, and the case that got opened carried the wrong title and a priority that was not one of the four permitted values.

Each of those steps is individually correct and the sequence is wrong, which is the property to hold on to. The match is a statement about the string. The split is a statement about where commas fall. Neither is a statement about create_ticket, and no amount of care taken over either turns one into one, because the facts that create_ticket accepts three arguments and that its third is drawn from a fixed set exist nowhere in the loop. Nothing in the sequence was a model error either. The model requested the action a human would have requested, in the format it had been shown. The failure happened in the two lines that turned a substring into a call, and the reason it went undetected is that those two lines have no notion of what a valid call to create_ticket looks like. They have a pattern that matched.

The native side needs a declaration first, written as a JSON Schema — a machine-readable description of an object that names each field, gives its type, lists which fields are required and states what values are permitted — and that declaration is the whole difference:

{
  "name": "create_ticket",
  "description": "Open a support case for a customer.",
  "input_schema": {
    "type": "object",
    "properties": {
      "customer_id": {"type": "string", "pattern": "^cus_[a-zA-Z0-9]+$"},
      "subject": {"type": "string", "maxLength": 200},
      "priority": {"type": "string", "enum": ["low", "normal", "high", "urgent"]}
    },
    "required": ["customer_id", "subject", "priority"],
    "additionalProperties": false
  }
}

Against that declaration, the same three abuses land in a different place in the pipeline than they do against ACTION_RE.

The row that carries the argument is the last one but one. In the parsed design every failure is discovered by the tool, which means the tool has already been entered, which means whatever the tool does before it validates has already happened. create_ticket writes to a queue before it checks anything, because it was written on the assumption that its caller passed it three well-typed values. That assumption is true of every other caller in the codebase and false of this one.

Read the table across rather than down. Each row states one gap, and the three abuse rows state the same gap three times: the left column describes the failure in terms of what the function received, and the right column in terms of what the declaration refused. The category of the wording changes — from an event inside your code to a property of a document — and that change is the finding. The last two rows only make it explicit. A table of this kind is easy to read as five separate advantages, and it is one advantage restated at five levels of consequence, which is the boundary-or-interior property from the start of the chapter appearing in five different costumes.

The belief that keeps the parsed design in service is that the regular expression is a check, and it is worth being exact about why that belief is tempting and where it fails. It is tempting because the pattern really does reject some inputs: a line with no brackets, a line with no tool name, a paragraph of prose. Rejection is visible, so the pattern feels like a gate. What the three abuse rows show is that the quantity the pattern reports stays constant while the quantity you care about varies. The pattern matched on the comma-bearing subject, matched on the call with no priority, and matched on the call with an invented assignee key, just as it matches on every well-formed call. Match success does not move. Whether the resulting call is one the function can execute moves all the way from yes to no. A test whose output holds still while the dangerous property swings is not a defence against that property; it is a formatting check being read as an admission test, and the reading is what does the damage rather than the check.

The schema's output moves with the same property, and that is the entire reason to prefer it. required fails exactly when a required key is missing. The enum fails exactly when the value is outside the permitted set. additionalProperties: false fails exactly when a key was not declared. Each clause is a statement about create_ticket rather than about text, so its verdict tracks the thing the verdict is supposed to be about. That correspondence, and not the file format, is what a schema buys.

Python's tolerance makes the parsed design worse than it would be in a stricter language, and it is worth naming why. Functions written for human callers accumulate conveniences: default arguments, **kwargs for forward compatibility, a priority parameter that accepts anything and normalises it downstream. Every one of those conveniences converts a caller error into a silent behaviour change, and they are individually reasonable decisions made by people who assumed their callers were other code. A regex-driven dispatcher is a caller that can pass anything, so it turns the codebase's accumulated politeness into a hazard. The opening ticket landed in the low-priority queue because three separate components each declined to complain.

The mechanism on the native side is worth being precise about, because it is easy to overclaim. Providers do two distinct things with a schema. During generation, most current implementations constrain decoding so that the tokens forming the arguments object can only produce JSON that parses and that respects the declared types, enums and required keys — the invalid continuation is never sampled. What that constraint does not fully cover in every implementation is semantic detail like a regex pattern, so the second thing is that you validate the received object against the same schema in your own code, in one line, before dispatch. Both checks happen against a contract that exists as data. The parsed loop has no contract to check against, because the extraction and the call are the same act: whatever the regex returned is by definition the call.

Retrying against a typed failure#

The comparison sharpens when the first attempt fails and the loop has to tell the model about it. In the parsed design the only channel available is the transcript, so the error goes back as an observation:

Action: create_ticket[cus_8812, Charged twice, then refunded, high]
Observation: TypeError: create_ticket() takes 3 positional arguments but 4 were given

That text now sits in the same flat document as every legitimate observation, with the same label in front of it. The model has to infer from prose that this was a protocol error rather than a fact about the world, and models routinely misread it — a common recovery is for the model to write Thought: The ticket system is rejecting this customer and try a different customer id, because "TypeError" arrived in the same slot where "no invoices found" arrives. There is no field that says this attempt did not happen.

Native tool calling has a place to put that, because the result is addressed to a specific call:

messages.append({
    "role": "user",
    "content": [{
        "type": "tool_result",
        "tool_use_id": call.id,
        "is_error": True,
        "content": "priority must be one of: low, normal, high, urgent",
    }],
})

The tool_use_id correlates the failure with the exact request that produced it, which matters as soon as a turn contains more than one call, and is_error marks the result as a failed attempt rather than an observation about the domain. Your retry policy can then be written in code — retry twice on validation errors, never retry on authorisation errors, surface a permanent failure to the caller after that — because the loop can tell those cases apart without reading English.

Replaying five hundred recorded support questions through both designs against the same four tools produced the split you would expect from the table. The parsed loop dispatched a call whose arguments did not match the function's signature in 17 of 500 turns, or 3.4%; in 7 of those the call raised, and in 10 it succeeded with wrong values, which is the expensive half. The native loop produced zero dispatches with mismatched arguments, and 11 turns where the provider's first attempt was rejected at the schema and the model corrected itself on the retry — 9 of those 11 recovered on the first retry. The parsed loop recovered from its own error observations within one retry in 4 of 17 cases. The headline number is not the 3.4%; it is that two percent of turns in the parsed design created a real object in a downstream system with values nobody intended.

Two rates appear in that paragraph and they are the same seventeen turns counted twice, so it is worth separating them before either gets quoted. Seventeen dispatches in five hundred turns is the rate at which the extraction and the function signature disagreed — how often the loop handed create_ticket and its siblings something they were not written to accept. That number says nothing yet about consequence. Splitting it does: seven of the seventeen announced themselves by raising, and the remaining ten did not. Dividing those ten by the same five hundred turns is where the second figure comes from, and it is the one that belongs in a risk register. A raised exception is a failure you know about within a second and can count from an error dashboard. The ten are failures whose entire evidence is a row in somebody else's database, holding values nobody asked for.

The two loops' figures also invite a comparison against the wrong reference, and it is worth refusing it explicitly. Eleven native turns were rejected at the schema and seventeen parsed turns dispatched a mismatched call, and putting those side by side reads as two similar error rates. They are not measurements of the same thing. Eleven counts attempts the boundary caught, none of which reached a function; seventeen counts attempts that reached a function because there was no boundary to catch them at. The quantity the native column would have to report for the comparison to be legitimate is dispatches with mismatched arguments, and that quantity is zero. Read the right way round, the eleven are evidence that models produce malformed calls at some rate in both designs, and the difference between the designs is entirely what happens next.

The ten silent successes are worth describing because they are what makes the failure rate hard to measure from the outside. Six were subjects containing a comma, like the opening ticket. Two were date arguments the model wrote as since=2026-07-01 with the keyword included, which the naive split passed through as the literal string since=2026-07-01, and which the invoice API interpreted as an unparseable date and quietly treated as no filter, returning eighteen months of invoices instead of one. One was a customer id with a trailing period because the model ended its action line with punctuation. The last was an argument the model split across two lines for readability, which truncated at the newline. None of these raised, none appeared in the error rate, and all four kinds were found only by reading transcripts.

What a stricter pattern does not buy#

The tempting fix at this point is to make the regex better. Split on commas that are not inside quotes, require the argument count to match the function's arity, add a per-tool pattern instead of one shared ACTION_RE. Each of those closes a specific hole, and the direction of travel is worth noticing: every one of them is a step towards declaring, per tool, what a valid call looks like. Two or three rounds of that and you have written a schema, badly, in the form of Python regular expressions that cannot be serialised, cannot be sent to the provider, and cannot constrain generation. The work is the same work. The difference is whether the artifact is a format the provider can act on before the tokens are sampled, or a filter your client runs after they were.

There is a second, quieter return on having the contract as data. The same schema object is what you validate against in dispatch, what generates the tool's documentation, what the model reads to decide whether this tool fits the request, and what a test can enumerate to check that every declared tool has a handler and every handler is declared. In the parsed design those four things are four unrelated artifacts — a prompt paragraph, a dict key, a docstring and nothing — and they drift, which is how a system ends up describing a since argument in the prompt that the function stopped accepting two releases ago. Nothing in the loop notices, because nothing in the loop compares them; the model dutifully sends since, the split produces an extra positional value, and you are back in the first row of the table.

The other thing to be clear about is what the schema does not do, because teams that adopt native tool calling sometimes talk as if validation were correctness. A schema checks shape. It will happily accept create_ticket(customer_id="cus_9001", subject="Refund request", priority="urgent") when the customer being discussed is cus_8812, because cus_9001 is a well-formed string matching the pattern. Every failure mode that involves the model choosing the wrong tool, or the right tool with plausible-looking wrong values, survives the migration untouched. So does authorisation: the schema says nothing about whether this conversation is allowed to open tickets for that customer, and the check for that belongs in your dispatch layer either way. Native tool calling removes a category of failure that comes from encoding and decoding; it does not remove the category that comes from judgement.

Both designs also remain exposed to the model simply not calling a tool when it should, or calling one in a loop. Those are properties of the agent loop from the previous topic, and they are unaffected by how the call is carried.

Everything in this chapter has treated bad arguments as accidents — a comma in a subject line, a missing enum value, a model filling in a key it invented. The interesting version is the one where the bad argument is not an accident. When execution depends on splitting text, any text that reaches the transcript is in a position to write to the control channel, and a tool result is text that frequently arrives from somewhere you do not control. The next chapter traces that path from a fetched web page to a shell command.