Detecting truncation from the finish reason

The instinct when you suspect truncation is to look at the output. It is right there, it is cheap to read, and a cut-off answer sounds like the sort of thing a person would notice. This chapter is about why that instinct fails, and about the check that replaces it.

The failure has two parts. Fluency is a local property of generated text, so it holds at every position including the one where a counter stopped the loop; and truncation rates in production are low enough that a sampled review can miss them entirely while looking carefully. What is left is the provider's own account of how the loop ended, which arrives on every response and costs one comparison to read.

A cut answer is as fluent as a finished one#

Consider two summary fields, both produced by the same model from the same document:

A: Vendor disputes the shipping surcharge applied on line 4 and requests
   a revised invoice. Payment of the undisputed balance is being processed.

B: Vendor disputes the shipping surcharge applied on line 4 and requests a
   revised invoice reflecting the corrected freight terms agreed in the
   November addendum, which supersedes

Most readers pick B as the truncated one, and they are right — but only because B was cut mid-clause. That is the easy case, and it is not the case that gets through review. The response that gets through is the one where the cap happened to land after a period. There is nothing structurally distinct about a language model's output at the point where it was interrupted: the last token it produced was, at that moment, the most plausible continuation of everything before it, and so was the token before that, all the way back. Fluency holds everywhere in the sequence, including at an end chosen by a counter rather than by the model.

That has a compact statement worth carrying, because it settles the question in advance for cases this chapter does not list: every prefix of generated text is itself plausible generated text. It follows from how the text was made. At each step the model conditions on the tokens already present and chooses a likely continuation of them, so the sequence is assembled as a chain in which every link was picked to sit well after the ones before it. Cut a chain of that kind at any point and what remains is a chain of exactly the same kind, because the links that are now missing were never holding up the ones that remain — influence ran forwards only. A truncated completion is a prefix of a completion. A prefix is therefore well-formed by construction. The cut is not damage done to the text; it is the absence of text that would have come next, and an absence leaves no mark on what is there.

Compare a corrupted file, which usually announces itself. A truncated JPEG has a grey band at the bottom. A half-written CSV has a row with the wrong number of columns. Those artifacts exist because the format has global structure that the truncation violates. English prose has almost no global structure — a paragraph that stops after two sentences is a two-sentence paragraph — so there is nothing for the truncation to violate.

Turned around, that comparison says exactly when reading the output does work, and the condition is narrower than it sounds. A cut becomes visible precisely when the output carries a structure spanning the whole of it that the cut breaks: an unclosed brace, a table whose final row is half a row, a numbered list that stops at the third of five promised items. The visibility is a property of the format rather than of the model, and across the formats a single service produces in a day it ranges from total to none. A correctness check resting on it is therefore resting on which format the prompt happened to ask for, and it stops working on the day somebody changes the prompt to ask for prose.

Check the return value, not the buffer#

There is an old habit that transfers exactly here. When you call read(fd, buf, 4096) and get back 1200, you do not inspect the 1200 bytes to work out whether more data was coming. You look at the number the call returned, because that is the channel through which the kernel tells you what happened. Reading the buffer to infer the outcome of the read would be an obvious mistake, and nobody makes it, because the API's contract puts the outcome somewhere else.

Model APIs have the same shape and the habit has not transferred. The finish reason is present on every response, it is the provider's statement about what ended the loop, and it is the only place that statement is made. The text is the buffer.

The parallel is close, and the place it stops holding is where this whole topic lives. read gives you a count you can compare against the count you asked for, so a short read is detectable by arithmetic even by a caller who ignores the contract entirely. A completion offers no such comparison. You never asked for a number of tokens; you asked for an answer, and how long that answer ought to have been is not a quantity anybody in the system knows, including the model. Strip the finish reason away and there is no second source to fall back on. That is why reading it is not the tidier of two available methods. It is the only method.

The disambiguation problem in the second row bites people moving code between providers. On Anthropic, a stop-sequence match and a natural ending are different values, so a single equality check distinguishes them. On OpenAI both report stop, and the only way to tell them apart is to examine whether the content ends where one of your stop strings would have cut — which is exactly the read-the-buffer mistake, forced on you by the API. If you are on that API and you use stop strings, the safest posture is to treat stop as ambiguous and rely on structural validation of the content instead.

Write the check as an allowlist#

The instinct is to write the check as a denylist: fail on max_tokens, fail on refusal, proceed otherwise. That gets the polarity backwards, and the reason is in the table above. refusal, pause_turn and model_context_window_exceeded are all values that did not exist in earlier versions of the API. Every one of them was added because a new failure mode became reportable, and every denylist written before the addition silently classified the new value as success on the day it started appearing.

class IncompleteGeneration(Exception):
    def __init__(self, stop_reason: str):
        super().__init__(f"generation did not complete: {stop_reason}")
        self.stop_reason = stop_reason


COMPLETE = frozenset({"end_turn"})          # add "tool_use" in an agent loop


def require_complete(response) -> str:
    if response.stop_reason not in COMPLETE:
        raise IncompleteGeneration(response.stop_reason)
    return response.content[0].text

An allowlist inverts the risk. A new stop reason you have never heard of raises immediately, in the environment where it first appears, with the unknown value in the message. That is a five-minute investigation. The denylist version is a silent behaviour change that shows up as a data quality question from another department, which is the six-week version of the same investigation.

Walking through the day a new value first appears shows why the polarity matters so much. Suppose a service checks if response.stop_reason == "max_tokens" and raises, proceeding otherwise, and suppose the provider starts returning refusal on a thin slice of inputs. Nothing raises. The comparison is false, control falls through to the line that reads response.content[0].text, and on a refusal the content array can be empty. Code that indexes it directly at least fails loudly, with an index error thrown from a line that has nothing to do with the cause. Code written defensively — a default of "", a try wrapped around the extraction, a helper that returns an empty string when anything goes wrong — writes the empty string into the column and books the request as a success. The defensive version is the one that costs six weeks, which makes a denylist and a swallowed default a genuinely dangerous pair rather than merely an untidy one.

Treating a value as failed also needs a policy, because the obvious one does not work. Retrying a max_tokens response with the same request reproduces it. The prompt is unchanged, the cap is unchanged, and although sampling is not deterministic, how long the answer wants to be is a property of the task rather than of luck, so the second attempt runs into the same ceiling at about the same place. A retry helps only when something has moved: a larger cap, a smaller task, a prompt that asks for less. Retrying unchanged turns one truncated response into two and bills you for both.

The COMPLETE set is deliberately a named constant rather than an inline string, because the right membership differs by call site. A single-shot extraction accepts only end_turn. An agent loop accepts end_turn and tool_use, since a tool call is the loop continuing normally. A call using server-side tools also accepts pause_turn and resumes. Making the set explicit forces each call site to state which endings it is prepared to handle rather than inheriting somebody else's answer.

Carrying the reason into the exception rather than discarding it matters for the same reason. When this raises in production, the first question is which ending fired, and the difference between the answers is the difference between raising a cap, fixing a stop sequence, compacting a conversation, and escalating a policy refusal to someone who can decide what to do about it. An exception that says the response was incomplete and nothing else forces whoever is on call to go back to the request log to find out what the process already knew.

Log the reason on every response, not only on the failing ones. A field with six possible values is cheap to store and it is the only way to answer the question "has this always been happening at this rate, or did it start on Tuesday?" — which is the first question anyone asks after discovering a rate like this, and the one that is unanswerable if the field was only ever recorded at the moment somebody decided to start looking.

Streaming moves the signal into a different event#

None of this is harder when streaming, but the field is somewhere less obvious, and stream handlers written by accumulating text tend to miss it entirely. Anthropic streams over SSE, or server-sent events, which is a single long-lived HTTP response whose body is a sequence of labelled events the client reads as they arrive. The text arrives on content_block_delta events and the stop reason arrives on a single message_delta event near the end, alongside the final usage numbers. A handler that concatenates deltas and returns when the connection closes never touches the field.

with client.messages.stream(model="claude-opus-5", max_tokens=4096,
                            messages=messages) as stream:
    for text in stream.text_stream:
        emit_to_user(text)
    final = stream.get_final_message()

require_complete(final)   # same check, same allowlist

Using the SDK's get_final_message() reassembles the whole response — content, usage and stop reason — so the same check applies unchanged. If you are consuming raw SSE, read delta.stop_reason off the message_delta event and carry it out of the loop.

Streaming also introduces the one ending that has no reported reason at all. If the connection drops partway through, you have some tokens, no message_delta, and no message_stop. The stream simply ends. Since your accumulator has been appending text the whole time, the natural shape of the code is to return what it has — which converts a network failure into a plausible-looking short answer. The defence is to make the terminal event mandatory: initialise the stop reason to None, and treat None at the end of the loop as a failure rather than as an absent complication.

That case deserves attention out of proportion to its frequency, because it is the one where the user has already seen the output. In a chat product the tokens were rendered as they arrived, so a dropped stream shows the user a message that stops mid-sentence and then sits there looking finished. The server may have no record of a problem at all, since from its side the generation may have completed normally and only the delivery failed. Products that handle this well mark the message as interrupted in the UI and offer to continue; products that do not are the ones where users learn to copy their question and ask it again.

The limit of the signal#

stop_reason: "end_turn" means the model reached a point where it judged the turn finished. It does not mean the answer is right, or relevant, or that it addressed all four parts of a four-part question. A model can invent a payment term, emit its end-of-sequence token, and hand you a confident end_turn on a fabrication. Every hallucination you have ever seen arrived with a natural stop reason.

So the check is narrow on purpose. It separates "the generator ran to a boundary it chose" from "the generator was interrupted", and it is the only reliable way to make that separation. Correctness is a different problem with different tools — evaluation sets, verification passes, cross-checks against the source — and none of them substitutes for this check, because a truncated response will fail them in confusing, expensive-to-diagnose ways rather than in the one obvious way.

The check earns its place by being the cheapest reliable thing in the topic: one field, present on every response, compared against a set you wrote down. Read habitually and logged on every call, it converts truncation from a class of bug that surfaces as a data quality question months later into a metric with a baseline and an alert. Read never, it leaves you inferring process facts from document contents, which is the activity the rest of this topic is about failing at.

There is also a gap between the check existing and the check running. Most pipelines have several layers between the API call and the durable write, and a truncated response has to get past all of them. Whether it does depends on something that varies enormously between codebases: how loudly the content itself breaks when it is cut in half.