Model refusals and platform blocks

A content filter is a classifier the provider runs around the model — separate software from the model's weights, scoring the text you send and the text the model produces against a fixed set of policy categories such as violence, hate, sexual content and self-harm, and refusing to pass anything that scores above a configured threshold. It exists because the company selling model access carries the consequences of everything its platform emits, across every customer at once, while a model's own trained-in judgment is probabilistic, arguable and different in each version; a classifier wrapped around the model gives the provider one enforcement point it can tune, audit and change without retraining anything. You meet it the moment you call a hosted model rather than running one yourself: Azure OpenAI filters every deployment by default and will not disable it without an eligibility review, Amazon Bedrock packages the same idea as Guardrails, Google's Gemini API exposes safety settings per harm category, and Anthropic's API runs classifiers around the model on some request types. Every cloud model platform ships one, whether or not it came up during procurement.

Those four descriptions are a snapshot of how each platform was configured when this was written, and the details move. Defaults are tightened, categories are added, a detector that was on for everybody becomes configurable, and the eligibility process for switching one off is rewritten. What does not move is the reason any of them exist, which is the paragraph above: a company selling access to a model to thousands of customers at once cannot enforce policy by retraining, so it enforces policy in software it can redeploy. Treat the specifics as something to verify against current documentation for the platform you are on, and the shape as something you can rely on.

The physical arrangement is worth being concrete about, because most of what follows is a consequence of it. The model's weights sit behind an endpoint the provider operates, and your request reaches those weights only by passing through provider code first. That code is where the classifier runs. It is a separate function with its own inputs and its own output — text in, a score per policy category out — and it has no share in the model's generation. Two consequences follow immediately, and both are load-bearing later. The classifier can refuse before the model has seen anything, because it sits upstream of the weights. And it does not read instructions, because scoring text is all it does: there is no step in it at which a sentence addressed to an assistant is treated as a sentence addressed to anybody.

When that classifier refuses, this topic calls the result a platform block, as distinct from the model itself deciding, in prose, that it would rather not answer — a model refusal. Those two failures come from different systems, are decided at different points in the request, cost different amounts, and have different fixes. Merging them is the single most consequential mistake in this topic, because every downstream decision — retry policy, alerting, user messaging, escalation — depends on knowing which one you are holding. This chapter is about telling them apart from the response alone.

The response tells you which system said no#

A model refusal is content. The model ran, consumed your input tokens, produced output tokens, and what it produced happens to be a decline. From the transport's point of view nothing went wrong at all: HTTP 200, a normal usage block, a finish reason indicating the model stopped because it was done talking. The decline lives inside content[0].text, which means your code has to read English to notice it — and English is a terrible thing to pattern-match against, because "I can't provide the exploit code, but here is how the vulnerability works" is a partially successful response, not a failure.

The reason there is no field to read instead is worth following through, because it explains why a refusal cannot be given one. A model produces its answer by scoring every candidate next token against the text so far, drawing one, and repeating. A decline is that loop running normally and landing on the words I can't help with. Nothing in the loop distinguishes those tokens from any others, so there is no point at which the provider could have set a flag saying a refusal is under way — the refusal is not a decision the machinery took, it is a description of what the text turned out to say. The English is not a rendering of a verdict held somewhere else. It is the whole of the event, which is why reading English is the only way to find it and why the reading is unreliable.

A platform block is envelope. Something other than the model decided the request or the response could not be served, and the shape of what comes back is not a message at all. Depending on the provider it is an HTTP 400 with an error object, or a 200 with an empty content array and a stop reason that names the block. What it never contains is a model-authored sentence explaining itself, because the model either did not run or its output was discarded before you saw it.

// Model refusal: HTTP 200. The model ran and declined in prose.
{
  "id": "msg_01Qw8Z3nR7",
  "model": "claude-opus-4-8",
  "stop_reason": "end_turn",
  "content": [
    { "type": "text",
      "text": "I can't help with a working exploit for CVE-2024-3094, but I can explain how the backdoor was injected into the build." }
  ],
  "usage": { "input_tokens": 412, "output_tokens": 52 }
}

// Platform block, input side: HTTP 400. No model output exists.
{
  "error": {
    "code": "content_filter",
    "message": "The response was filtered due to the prompt triggering Azure OpenAI's content management policy.",
    "innererror": {
      "code": "ResponsibleAIPolicyViolation",
      "content_filter_result": {
        "hate":     { "filtered": false, "severity": "safe" },
        "self_harm":{ "filtered": false, "severity": "safe" },
        "sexual":   { "filtered": false, "severity": "safe" },
        "violence": { "filtered": true,  "severity": "medium" },
        "jailbreak":{ "filtered": true,  "detected": true }
      }
    }
  }
}

Two structural details in that second body matter more than the prose. There is a per-category verdict with a severity, which is the provider telling you why it blocked in machine-readable terms. And there is a jailbreak entry alongside the harm categories, which is a separate detector — a prompt-injection classifier — that fires on request shapes rather than on subject matter. A request can be blocked by that detector while every harm category reads safe, and engineers who only look at the four harm categories conclude the filter fired for no reason.

Where in the request each decision happens#

The ordering is the part that most changes what you do next. A platform filter is not one thing sitting in one place; it is at least two checkpoints, and they have different economics.

An input-side block happens before the model runs. You are billed nothing, the latency is tens to low hundreds of milliseconds, and no completion ever existed. An output-side block happens after generation is complete, or in streaming mode after part of it has already reached you: the tokens were produced, they are on your invoice, and then they were withheld. On a 2,000-token prompt producing an 800-token answer, an output-side block on a mid-tier model costs roughly the same as a successful call — call it $0.018 at $3 per million input tokens and $15 per million output — for a response nobody will ever read.

That figure is two multiplications and an addition, and it is worth doing them in the open, because the surprise for most people is not the size of the number but that there is a number at all. Two thousand input tokens at three dollars a million is six thousandths of a dollar. Eight hundred output tokens at fifteen dollars a million is twelve thousandths. Six thousandths plus twelve thousandths is the eighteen thousandths above, and every one of those thousandths is charged for work that was genuinely performed: the model read the prompt and generated the answer, and it did both before anything looked at what it had written.

The reason none of it comes back is the order of the steps rather than a billing policy anyone chose to be harsh about. Usage is metered where the tokens are produced, and the output filter reads a completion the model has already finished producing. By the time the verdict exists, the meter has already counted. An input-side block escapes the charge for the mirror-image reason and not because the provider is being generous — the classifier sits before the weights, so the prompt is never processed by the model, no tokens are generated, and there is nothing for the meter to count. This is the single most useful thing to know about a block before you decide what to do with it, because it means the same event can cost either nothing or a full call depending only on which side of the model the classifier that refused it was standing.

There is a third shape worth naming now so it does not ambush you later. Some providers deliver a platform verdict inside a successful response rather than as an error. On the Anthropic API a safety classifier that declines a request returns HTTP 200 with stop_reason: "refusal", an empty or partial content array, and a stop_details object carrying a category such as cyber or bio. Code that does response.content[0].text without checking stop_reason first will throw an index error, or worse, will silently treat an empty string as a valid answer.

# The only branch order that is safe across all three shapes.
try:
    resp = client.messages.create(model=MODEL, max_tokens=1024, messages=msgs)
except anthropic.BadRequestError as e:
    if getattr(e, "type", None) == "invalid_request_error" and "content" in str(e):
        return PlatformBlock(stage="input", detail=e.response.json())
    raise
if resp.stop_reason == "refusal":                 # platform, post-model
    cat = resp.stop_details.category if resp.stop_details else None
    return PlatformBlock(stage="output", category=cat)
text = "".join(b.text for b in resp.content if b.type == "text")
if not text:                                       # empty 200: still not a refusal
    return EmptyCompletion(request_id=resp.id)
return ModelAnswer(text)                           # may still contain a prose decline

Note what the last line does not do: it does not scan the text for apology words. A prose decline is a content outcome, and content outcomes belong to whoever owns the prompt, not to whoever owns the error taxonomy. If you want to measure prose declines — and you should, because a rise in them is a real signal — measure them with a classifier you own, on a separate metric, and accept that the measurement is fuzzy. What you must not do is let that fuzzy measurement contaminate the exact one.

Streaming makes the shape harder, not different

Everything above assumes a single response object. Under streaming the same three outcomes arrive as event sequences, and two of them can look identical for the first several hundred milliseconds.

A model refusal streams normally: message_start, then text deltas that happen to spell out a decline, then a normal stop. An input-side block never produces a stream at all — the HTTP request fails before the first event, which is the easiest case to handle because your existing error path catches it. An output-side block is the awkward one. The provider evaluates the completion in chunks, so a stream can deliver 300 tokens of a perfectly good answer and then terminate with a block verdict in the final event. The user has already watched half a paragraph appear.

Chunked evaluation is not a design shortcut, and following the constraint that forces it is what makes the behaviour predictable rather than alarming. Streaming exists so that text appears while the model is still writing. That means tokens have to leave the provider before the completion is finished. Whatever the output filter does, therefore, it cannot have read the whole completion at the moment the first tokens go out, because the whole completion does not exist yet. It has two options and only two: hold everything until generation ends, which is not streaming, or score what has accumulated so far and score it again as more arrives. A provider offering streaming has chosen the second. So a stream that stops on a block verdict is not a filter that woke up late; it is a filter that could not have run earlier on text that had not been written. Any product decision you make here is downstream of that, and no provider setting removes it.

That leaves a product decision with no clean answer: leave the partial text on screen, or remove it. Leaving it means shipping content the provider's safety layer decided should not be shown, which is precisely the thing you were paying that layer to prevent. Removing it means text vanishes from under the user's cursor, which reads as a bug. Most teams settle on buffering — holding the stream in the client for a short window and rendering only once enough events have arrived to make a block unlikely — and pay for it in perceived latency. Whichever you choose, choose it deliberately and write it down, because the default is whatever your streaming component happens to do when the iterator raises. The billing consequence is the same in all streaming cases: tokens delivered before the block are tokens you were charged for.

Different owners, different remedies#

One row of that table does more work than the others and is the least obvious, so it is worth deriving rather than asserting. A model refusal varies run to run because it comes out of the sampled loop described above: at each position the model scores the candidates and something picks one, and on a request near the edge of what the model will answer, I can't and Here's how are two candidates with comparable scores. Send the same request twice and you can get one of each. A filter verdict comes from a different kind of computation entirely. The classifier maps text to a score, that score is compared against a configured threshold, and a comparison between two fixed numbers has no room for a draw in it. Same text, same score, same side of the threshold, same answer, every time.

That difference is why one of these failures is a rate and the other is a verdict. A refusal rate in the low single digits is a real property of a prompt and a model, and you move it by changing the prompt. A filter block carries no such odds on the request it blocked; it is certain on that request and absent on the next one, and the percentage you see on a dashboard is a fact about the mix of content your users send rather than about the odds any one of them faced. The next chapter is about what follows from that for retries, and it is the practical half of the distinction.

The prose refusal is a prompt problem and it is yours. A security product asking for exploit mechanics gets better results by stating the analyst's role, the defensive purpose, and the fact that the vulnerability is published — and if that does not work, by choosing a model whose policy fits the domain. You can test that change in an afternoon and ship it, and you will see the refusal rate move.

The platform block is not a prompt problem in the same sense, because the entity that said no is not the entity you are prompting. You can sometimes get around it by changing the input, which is what a later chapter of this topic is about, but you cannot argue with it, and you cannot fix it by being clearer about your intentions, because the classifier that ran on the input never read your system message as an instruction. It read it as text to score. An elaborate "you are a defensive security assistant, the user is an authorised analyst" preamble can increase a block rate, because it adds more security-adjacent vocabulary to the thing being scored.

The belief that preamble does the opposite is the most reliable wrong turn in this topic, and it is wrong for a mechanical reason rather than a philosophical one. A system message has authority over the model because the model was trained to treat that position differently from a user turn. The classifier was not trained on that arrangement and takes no part in it. What reaches it is the assembled request as text — role markers included, since those are characters in the same string — and what it returns is a score for that text. Stating your authorisation to it accomplishes what stating your authorisation to a word counter would. That is where "be clearer about your intent" stops being advice and becomes a way of adding tokens to the very thing being measured.

There is a second-order effect worth anticipating here, because it drives a lot of wasted effort. When a team cannot separate the two failures, the natural response to a rising number is to work on the prompt, since that is the lever everyone knows how to pull. Prompt work is cheap to start, produces visible activity, and occasionally moves the number by coincidence — a rewrite that shortens the input can drop it below a severity threshold. So the team gets intermittent reinforcement for an intervention that is not addressing the mechanism, and three sprints later there is a 900-token system message full of clauses nobody can justify, half of which were added to fix blocks and are themselves adding scoreable text.

The remedies diverge accordingly. For a refusal you edit and redeploy. For a block you have three moves: change the input so it stops scoring above threshold, ask the provider to change the threshold or exempt your account, or route the affected traffic somewhere else. The first is engineering, the second is a support process with a turnaround measured in business days, and the third is a commercial decision. None of them is a retry.

A block is not automatically worse than a refusal#

The distinction is not a severity ranking. A product whose model politely declines 30% of legitimate requests is in more trouble than one blocked on 0.4%, and the refusal problem is the one nobody escalates because it returns a 200 and never trips an alert.

It is also not the same distinction as "user error versus system error", which is what most error taxonomies are actually built around. Both of these failures can be triggered by a perfectly well-formed request from a legitimate user doing legitimate work; neither is a validation failure, and putting either in a 4xx bucket labelled "client error" will teach everyone who reads your dashboard that the user did something wrong.

And the response shape does not, by itself, tell you whether a given block is a false positive. It tells you that the platform blocked and which category fired. Whether that category should have fired for a clinician's structured self-harm risk assessment is a judgment nobody at the provider has made about your specific request, and reaching it is a separate piece of work with its own evidence requirements.

What the shape does give you is an exact, cheap, mechanical test that runs on every response: a status code, a stop reason, and a category field, none of which requires reading English. Two lines in a log that used to be one become two classes with different owners, different costs and different remedies, and every technique in the rest of this topic — the taxonomy, the captured signal, the reduction, the segmented metric, the appeal — depends on that split having been made at the boundary rather than reconstructed later from prose.

The next question is what your infrastructure does with each of them once separated, and the answer most systems reach for by default — treat the failure as a transient upstream error and let exponential backoff sort it out — is precisely wrong for one of the two, in a way that costs money on every attempt.