Stop sequences and the consumed delimiter

Stop sequences exist so that a caller can end generation at a string of their choosing rather than waiting for the model to decide it is finished. You supply one or more of them in the request, the serving layer watches the output as it is produced, and the first time any of them appears the loop stops and the response comes back with stop_reason: "stop_sequence". That much is what the parameter's name promises.

What the name does not tell you is what happens to the matched text, and the answer surprises nearly everybody the first time: it is treated as a marker for where to cut, not as content, and the cut lands immediately before it. This chapter is about that behaviour — where in the serving path the cut happens, why the default was chosen this way, what it does to a streaming response, and what to do when your parser needs the delimiter that was eaten.

Where the cut happens#

The response makes the behaviour unambiguous once you look at the right two fields. A classification request that wraps its verdict in tags, with the closing tag registered as the terminator, looks like this:

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    stop_sequences=["</answer>"],
    messages=[{
        "role": "user",
        "content": "Classify this document. Reply as <answer>LABEL</answer>.\n\n" + doc,
    }],
)

{
  "content": [{ "type": "text", "text": "<answer>contract" }],
  "stop_reason": "stop_sequence",
  "stop_sequence": "</answer>",
  "usage": { "input_tokens": 892, "output_tokens": 12 }
}

Three things are being reported at once. stop_reason says the run ended because a supplied sequence matched, not because the model was done. stop_sequence says which of your sequences did it, which matters when you have supplied several. And content holds the text with the delimiter stripped. The output_tokens count still includes the tokens the model spent producing </answer> — generation genuinely happened, and the serving layer's decision was only about where to slice the text it hands back.

The matching runs over decoded text rather than over token IDs, which is why a stop sequence works even when it does not line up with the tokenizer's boundaries. A tokenizer might well encode </answer> as three or four tokens, and it might encode t</answer> differently again depending on what preceded it; none of that matters, because the serving layer accumulates the decoded output as tokens arrive and checks the tail of that buffer after each one.

It is worth working out why the matcher has to be built that way, because the obvious alternative — compare the ids coming out of the sampler against the ids your string encodes to — does not survive contact with a tokenizer. A tokenizer is the component that splits text into the vocabulary units a model actually reads and writes, and it splits according to what surrounds each piece, so a string has no single canonical id sequence. </answer> standing on its own might encode as </, answer, >. Directly after a letter, the first of those pieces may merge with that letter into a different unit. After a newline, different again. A serving layer matching on ids would therefore have to hold every id sequence that could possibly decode to your string, and which of them will occur depends on the text immediately before the match — text that does not exist until the model has produced it. Matching on decoded characters removes the entire problem. Whatever ids arrive, they decode to characters, and characters are the thing you specified.

Two consequences fall out of that choice, and both surprise people later. The first is that a cut can land inside what was, at the instant it was sampled, a single token. Suppose the model emits one token whose text is contract</answer> in one go. The match begins partway through it, the cut is defined by the character position rather than the token boundary, and the response carries contract while the token that produced it was sampled in full and counted in full. The second consequence is that the matcher has to keep a decoded buffer at all — it cannot decide anything from the token just sampled, only from the tail of everything so far — and that buffer is what makes the prefix behaviour below a requirement rather than a refinement.

The prefix branch is the part most people never think about, and it is the reason streaming behaves the way it does. If you have registered "</answer>" and the model emits <, the serving layer cannot forward that character yet, because the next few tokens might complete the sequence — in which case the < should never have been sent. So it holds the partial match, samples on, and releases the held text only once the match is ruled out.

Holding back is forced, and the reason is that streaming has no undo. Characters go out over an open connection in the order they are produced, and once one has been written it is in a client buffer, on a screen, appended to somebody's accumulator string. No later event retracts it. So at every character that might turn out to be the first character of your delimiter, the serving layer faces two options: send it immediately and be irreversibly wrong if the sequence goes on to complete, or withhold it and be a few tokens late if it does not. Only the second mistake can be repaired, so the matcher makes the second one. What that delay buys is a guarantee — that no part of a delimiter you asked to have removed ever reaches you — and a layer that has already spoken cannot offer it.

Register a sequence that starts with a common character and you have quietly introduced a small stutter into your token stream, because the matcher spends a lot of time holding characters back.

The effect is measurable in a chat UI. A stop sequence beginning with "\n" means every newline the model produces is withheld until the following token proves the sequence did not match, which reads to the user as a pause at the end of every line. A sequence beginning with a rare character costs nothing noticeable. This is a small consideration next to correctness, but it is the kind of thing that gets attributed to network jitter for months when the actual cause is a parameter three layers away.

Several sequences, and which one fired#

You may register more than one, and the semantics are the ones you would guess with one wrinkle worth knowing. Generation stops at the earliest position where any of your sequences completes, so the sequences have no priority order among themselves — position wins. If two of them could complete at the same position, the response's stop_sequence field names one, and relying on which one is a bad idea.

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    stop_sequences=["</answer>", "\n\nUser:", "###END"],
    messages=messages,
)

match response.stop_reason:
    case "end_turn":
        text = response.content[0].text
    case "stop_sequence" if response.stop_sequence == "</answer>":
        text = response.content[0].text + "</answer>"   # reconstruct
    case "stop_sequence":
        raise UnexpectedBoundary(response.stop_sequence) # a guard rail fired
    case other:
        raise IncompleteGeneration(other)

The shape of that branch carries the design. Two of these three sequences are guard rails — they exist to catch the model wandering into a fabricated conversation turn or running past the end of the document — and if either fires, something has gone wrong that you want to know about. One is the intended terminator, and it is the only one whose match is a success. A single if response.stop_reason == "stop_sequence" branch that reconstructs unconditionally would append "\n\nUser:" to the answer when the guard rail fired, which produces a garbled result from a signal that was trying to warn you.

Reconstruction is correct only because the reason was checked first. Appending unconditionally would corrupt every response that ended for a different reason, and appending response.stop_sequence when that field is null would raise on exactly the responses that finished normally.

The end_turn branch at the top of that match is not a formality, and it is where the second misconception about this parameter lives. Registering a stop sequence does not make the model produce the string. It arms a rule that fires if the string appears, and whether it appears remains an ordinary question about what the model generates. A run can therefore end with the model's own end-of-sequence token, having never emitted your terminator anywhere, and the response comes back with stop_reason: "end_turn" and stop_sequence: null.

Hold that case next to the consumed-delimiter case, because the text can be character-for-character identical. With </answer> registered, a response whose content is <answer>contract and whose reason is stop_sequence is a complete verdict with the closing tag eaten by the serving layer. A response whose content is <answer>contract and whose reason is end_turn is a model that stopped without ever closing the tag, which is a formatting failure in the generation itself. The repair for the first is wrong for the second: appending a closing tag there manufactures a well-formed document out of an answer the model never finished writing, and every downstream consumer will treat the result as authoritative. Same characters, opposite meanings, and the only thing separating them is the field beside the text.

The other repair is to stop overloading the terminator. If your parser needs </answer> to be present, choose a boundary marker that is not </answer> — a sentinel line such as "\n###END" that the prompt asks the model to emit after the closing tag. The tag comes back inside the content because nothing cut there; the sentinel gets consumed because that is its job. This costs a few tokens per response and buys you a pipeline where the text you receive is valid input to your parser with no reconstruction step at all, which is worth it wherever more than one consumer reads the same output.

The approach that looks like a repair and is not is switching off stop sequences and relying on the prompt alone to make the model stop after the closing tag. It usually works, because modern instruction-tuned models — models given a training stage specifically on following directions written in the prompt, which is why you can ask for a format at all — do follow that instruction, and "usually works" is precisely the property that produces an incident six weeks later. A stop sequence is an enforcement mechanism; an instruction is a request. If you drop the enforcement you should do so knowingly, and you should still be checking why generation stopped, because now the answer will be end_turn on the good cases and something else on the bad ones.

Why consuming is the sensible default#

The behaviour looks arbitrary until you know what stop sequences were built for. The parameter predates chat-shaped APIs. On completion-style endpoints, a conversation was a single formatted string that the model continued, and the model — having learned the format — would happily continue past the assistant's reply into a fabricated user turn, then another assistant turn, on and on until the token budget ran out. The canonical fix was to stop generation the moment the string that begins the next turn appeared, which is where the well-known "\n\nHuman:" stop sequence came from.

In that setting, returning the matched text would be actively wrong. "\n\nHuman:" is not part of the assistant's answer; it is the first fragment of hallucinated dialogue that you are trying to discard. Consuming the delimiter is exactly what the caller wants, every time, and no caller ever wants it appended back. The default was chosen for that use case and it has not changed, because for the terminator-of-a-conversational-turn job it is still right.

What changed is what people use the parameter for. A closing tag or a fence marker is a structural boundary rather than a turn terminator, and structural boundaries are usually part of the grammar the consumer expects. The default silently converts your grammar into a different grammar — one where every document is missing its final delimiter — and hands that to a parser written against the original.

There is a way of holding the whole behaviour that predicts the answer in cases this chapter does not cover, and it is worth carrying instead of the rule. A stop sequence is not part of the document. It is a message from the generated text to the serving layer, saying cut here, and a message addressed to the transport is removed by the transport once it has been acted on. Anything you also need inside the document has to be carried separately, because the delimiter's job is over at the instant it is recognised. That framing accounts for the consumption without appealing to history at all. It also accounts for the existence of the stop_sequence field — a layer that removed one of your strings owes you an account of which one — and it tells you in advance, without running an experiment, that any attempt to make a single string serve as both boundary marker and grammar will cost you the copy in the document.

The limits of a substring terminator#

A stop sequence does not prevent the model from generating the text. The tokens are sampled, decoded, counted in output_tokens, and billed. If your reason for adding one was to avoid paying for output past a boundary, you get part of that — generation does end there, so everything after the boundary is never produced — but the delimiter itself is on your bill.

It does not filter content. A stop sequence has no notion of what the text means; it is a substring match. If the string you registered can appear inside a legitimate answer, generation will end there, in the middle of a sentence, and you will get stop_reason: "stop_sequence" on a response you consider mangled. Choosing "---" as a delimiter and then asking the model to write documentation containing horizontal rules is the classic version, and it is a special case of a general rule: a terminator has to be a string the content cannot contain. Three characters of ASCII punctuation almost always fails that test. A sentinel such as "###END_OF_ANSWER###" almost always passes it, and the extra tokens cost a fraction of a cent per call.

The failure is also asymmetric in a way that makes it hard to catch in testing. Short, simple test inputs rarely provoke the model into emitting a horizontal rule or a blank line in the middle of an answer, so the sequence never fires early during development. Long, real, messy production inputs provoke exactly that, which is why this class of bug reliably surfaces after launch rather than before it.

The behaviour is not an Anthropic quirk, so both repairs travel. OpenAI's stop parameter excludes the matched string from the returned content in the same way, and for the same historical reason. The difference is that its finish_reason reports stop for both a natural ending and a stop-string match, so the branch above cannot be written as an equality check on the reason — you have to track which strings you supplied and decide from that whether reconstruction applies, which is exactly the kind of inference the reason field exists to spare you.

A matched stop sequence, then, tells you three things and withholds a fourth. It tells you the loop was ended from outside, which of your rules ended it, and that the delimiter is missing from the text you hold. What it cannot tell you is whether the model had more to say, because a rule you wrote fired on a string that happened to appear, and the model's own judgement was never consulted.

Both parameters covered so far are set at request time and neither is visible to the model, but the stop sequence at least announces itself: there is a distinct stop_reason value and a field naming the culprit. The other request-time parameter people reach for when they want to control output has no such honesty, because it never reaches the model at all.