The messages array

You will meet this document behind every product surface built on a chat model, usually without seeing it: the assistant in the corner of an application, the "summarise this ticket" button, the agent framework three dependencies deep in a lockfile. Every one of them assembles the same body and posts it, and the moments when it stops being invisible are the expensive ones — a 400 naming an array index in a request your code never explicitly built, a token bill growing faster than traffic, a conversation in which the assistant forgets something it was told a minute ago. Each of those has a chapter in this topic, and each is much easier to reason about once you have built the body yourself.

A chat API is the interface a provider exposes for holding a conversation with a model: you send the conversation as it currently stands, and what comes back is the next thing the assistant would say. The conversation travels as an array of messages, and each message carries a role naming who produced it — the operator who set the standing rules, the person typing, or the model itself — together with a content field holding what was said. A multi-turn conversation is that same array with more entries in it, sent again in full on every request. There is nothing else in the protocol that represents the conversation.

Building the call by hand#

Here is a complete, working request. No SDK, no wrapper, nothing hidden.

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-opus-5",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Name the capital of Portugal."}
    ]
  }'

Four headers and four fields, and only one of them is interesting. The endpoint is a plain POST; there is no handshake, no connection to open, and no session to establish. The anthropic-version header pins the wire contract to a date, so that a provider can change response shapes for new callers without breaking yours — it is a version number for the JSON, not for the model. The model field names which weights run, and max_tokens is a hard ceiling on the length of the reply, enforced by cutting generation off when it is reached rather than by asking the model to be brief.

That distinction catches people, and the mechanism under it is short enough to state in full. Generation proceeds one token at a time: the model scores the candidates for the next piece of text, ordinary code outside the model picks one, appends it, and calls the model again. max_tokens is a counter held by that outer loop, and the model's input does not contain it. So the ceiling cannot make the model pace itself or close a sentence early — when the counter runs out the loop stops calling the model, mid-word if that is where it happens to be. A ceiling of 1024 does not produce a shorter answer to the same question. It produces the first 1024 tokens of the answer the model was going to give anyway.

The messages array is the part that matters. It is an ordered list, and each element is an object with exactly two fields you must supply: a role drawn from a closed set, and content. The order of the list is the conversation — there are no timestamps, no ids, and no threading; position 0 happened before position 1 because it is at position 0. The roles are a vocabulary the provider defines, not labels you invent: send "role": "customer" and you get a 400 back naming the offending index.

Why that vocabulary is closed is worth working out rather than accepting, because the answer explains most of the rest of this topic. A language model reads one flat sequence of tokens and nothing else. It has no notion of a list, a field, an object or a speaker. An array of role-tagged messages therefore cannot be what the model receives; something on the provider's side has to render it down into a single sequence first. And the only way to keep several speakers apart inside one sequence is to write a marker before each speaker's text and another after it. Those markers are themselves tokens, and the model was built around a particular set of them — one span opener for the operator, another for the person typing, another for the model's own turns.

A role name is therefore a lookup key into that fixed set of markers. That is the whole reason customer fails. It is not a validator being fussy about spelling: there is no marker to render customer as, and a provider that invented one would be feeding the model a delimiter it had never been trained to read as a boundary, which the model would take for ordinary text. The closed set is a consequence of how the sequence is assembled, not a policy laid on top of it.

The same fact accounts for why the array is validated at all rather than accepted as free-form JSON. The provider is not filing your document away; it is rendering it into the sequence the model reads, and an entry with no content is a rendering instruction with a hole in it. So the request is checked before any model runs, and the error points at the position that would not render.

content accepts two forms, and it is worth seeing both now because the second is the one everything later in this topic is built on. A bare string is sugar; the provider expands it into a list containing a single text block. These two entries are identical by the time they reach the model:

{"role": "user", "content": "Name the capital of Portugal."}

{"role": "user", "content": [
  {"type": "text", "text": "Name the capital of Portugal."}
]}

The block form exists because a turn can carry more than text — an image, a document, a request to call a tool, the result of one — and each of those is a typed block sitting in the same list. For a plain question the string form is fine, and most SDK helpers emit it. For anything structural you will be writing the list.

Reading the envelope#

The response is a JSON object with the generated text buried a little deeper than people expect:

{
  "id": "msg_01XyzAbc123",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-5",
  "content": [
    {"type": "text", "text": "Lisbon."}
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {"input_tokens": 16, "output_tokens": 5}
}

The text sits at content[0].text, and the indexing is not a formality — a response that used a tool, or that thought before answering, has other block types in that list, and code that reaches for content[0].text unconditionally will read the wrong block or crash on a response whose first block is not text. Iterate the list and match on type.

The reason a block other than text can turn up at position zero is the generation loop again. Blocks appear in the list in the order the model produced them, because the list is a record of a reply being emitted rather than a sorted index of its parts. A model that reasons before answering emits its reasoning first, so a thinking block sits at index 0 and the text is at index 1. A model that decides to call a tool before saying anything puts the tool block first, and one that narrates before deciding puts the text first. Neither order is a special case, and neither is announced in a field you can branch on before parsing. So content[0].text has encoded an assumption about model behaviour into a line that reads like plumbing, and the assumption stops holding the day somebody switches on reasoning.

Notice that the response carries "role": "assistant". The envelope is itself a message in the same shape as the ones you sent, which is what makes multi-turn work: you take the content list from the response and append it to your array as an assistant entry. The model field echoes the model that actually served the request, which is the cheapest available check that a deploy pinned what you thought it pinned.

usage reports what you are billed for, split into what you sent and what came back. Those two numbers are priced differently — output is typically several times the price of input per token — and they behave differently as a conversation grows, which is the subject of a later chapter. For now the useful habit is to log both on every call, because a token bill you cannot attribute per request is a bill you cannot debug.

stop_reason says why generation ended, and it is the field most often ignored until it causes trouble. A truncated answer and a complete answer are the same shape of JSON; the only difference is one string.

A parser that assumes valid JSON in the reply, hits max_tokens at 400 tokens, and reports "the model returns malformed JSON" is one of the most common false diagnoses in this area. The model returned exactly what it was allowed to.

Follow that failure through and it stops being a surprise. The loop's counter runs out somewhere inside the object — after an opening brace, a key and half a value, say. The bytes that were produced are correct JSON as far as they go; what is missing is everything after the cut. Your parser sees an unclosed object and reports a syntax error, which is the only thing it could report, because a truncated document and a malformed one are the same document to a parser. The HTTP layer agrees with the parser rather than with you: the status says success, the envelope is well formed, and the content block holds a string exactly as it would on a complete reply. Every visible signal says the request succeeded and the model produced rubbish, and the one signal that says otherwise is a string in a field most code never reads. Branch on stop_reason before you parse anything.

Turning one call into a conversation#

A second turn is the first body with two more entries in the array:

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-opus-5",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Name the capital of Portugal."},
      {"role": "assistant", "content": "Lisbon."},
      {"role": "user", "content": "And its population, roughly?"}
    ]
  }'

The second user turn contains no noun. "Its" resolves only because the two earlier entries are sitting in the same array, and the resolution happens inside the model as ordinary attention over one long input, not by any lookup of a stored thread. Run this and the usage.input_tokens on the second call comes back around 34 rather than the 16 of the first — you paid for the whole array again, because the whole array is what you sent.

Those two figures are worth taking apart, because the gap between them is the economics of everything later in this topic. The first request's sixteen input tokens covered the question and the marker tokens written around it. The second request contains that same question and those same markers, unchanged, then adds the assistant's one-word reply with its own pair of markers, then the new user turn with its. Add those two spans to the sixteen already there and the total lands near thirty-four. No discount applied to the part the provider had seen a moment earlier: the first turn cost what it cost the first time, and you paid for it again. So you are not billed for the question you just asked. You are billed for the conversation, once per turn, and a twenty-turn conversation has sent turn one twenty times — a growth curve worse than the linear one most people assume, which is why a later chapter of this topic, on the quadratic growth of input tokens, is about nothing else.

That is the shape of every multi-turn client that exists: read the stored turns, build the list, send it, append the response, store it. The libraries differ in ergonomics and in nothing else. It is worth seeing what each convenience actually does to the body, because the mapping is short:

The last row is the one that quietly costs money, and it gets a chapter of its own at the end of this topic. Streaming deserves a sentence here because it changes so little: the request body is unchanged apart from a flag, the response arrives as a sequence of server-sent events that each carry a fragment, and the client assembles them into a message with the same content, stop_reason and usage fields as the single-object form. Nothing about the array changes, and the assembled result is the thing you append to it.

The array replaced a string you concatenated yourself#

This shape is recent enough that its predecessor is still in living memory, and knowing what it replaced explains several of its oddities. The first widely used interfaces to these models took a single prompt string and returned a continuation, so a multi-turn conversation was built by concatenating the turns yourself with agreed-upon markers. A rendering looked something like \n\nHuman: where is my order\n\nAssistant: , and the trailing marker was what told the model whose turn it was. The client owned every byte of that rendering, including the blank lines, which were load-bearing.

Three problems came with that arrangement. A mistake in the markers or the whitespace produced no error at all, just a quietly worse response, because a malformed prompt is still a valid prompt. A user who typed the marker sequence into their message could forge a turn, appearing to speak as the assistant or to end the conversation and begin a new one. And there was nowhere to put anything that was not text, so images, tool calls and structured results had no representation.

The forgery deserves its own step, because it is the clearest case in this chapter of a mechanism doing something nobody designed it to do. The marker was plain text in the same stream as everything else. When the client wrote \n\nAssistant: it wrote those characters into the string; when a user typed the same characters into a support form, the client concatenated them into the string as well. By the time the model read the sequence there was no difference between the two — not a difference the model ignored, but none present in the bytes at all. A boundary made of ordinary characters can be written by anyone who can get ordinary characters into the document, and the person typing into your chat box always can.

The messages array solves all three by moving the rendering to the provider's side. The provider owns the delimiters, so your text is inserted into a role's span rather than concatenated into a stream where it might be mistaken for a boundary. The structure can be validated, which is why you get an error naming an index instead of a confusing answer. And because each entry's content is a list of typed blocks, there is a natural home for everything that is not prose. The cost is that you can no longer see the string the model receives — which is exactly why building the body by hand once is worth the hour.

That solution is narrower than it sounds, and the next chapter depends on knowing exactly how narrow. What the provider-owned delimiters remove is the ability of participants to forge a boundary — to make the model believe a new speaker has started. They do not remove the ability of participants to write whatever they like inside their own span, including sentences that instruct. "Ignore your previous instructions" arrives intact, correctly attributed to the user, and the model still reads it. Escaping a marker is a solved problem; persuading a reader is not, and the second is the one that remains.

Not a session, not a log, not portable#

The body is not a session. Nothing in the request or response identifies a conversation, and the id on the response is a receipt for one generation rather than a handle you can send back. There is no server-side transcript being appended to as you go, which has consequences severe enough that a later chapter is devoted to proving it empirically rather than asserting it.

That absence follows from the layers below it rather than from nobody having got round to it. The function that scores the next token reads its input and writes nothing back — weights are read at inference and never modified by a request, which is what topic 1.0 establishes when it separates training from inference. Nothing at the model layer could remember a conversation even in principle. A provider can of course keep a transcript beside the model, and provider-hosted conversation state is a real product with a topic of its own later in this part of the book. The default endpoint does not, because it would cost the property that makes the fleet cheap: requests for one conversation would have to reach the machine holding it, instead of any machine that happens to be free.

The belief that the endpoint remembers is not careless, and naming why it is tempting is more useful than calling it wrong. The response carries an id, and in nearly every other HTTP API an id returned on creation is a handle you send back. Chat products show a thread that survives closing the tab. Most decisively, the behaviour is indistinguishable from memory whenever the client is doing its job, because a client that resends the history gets answers that refer back correctly, and nothing on the surface separates "it remembered" from "you reminded it". The experiment that settles it costs one request: send the third entry of the array above on its own, without the two before it, and watch "its" resolve to nothing.

It is not a log, either, and the distinction matters when you start storing turns. Your database can hold whatever your application needs — cancelled generations, internal annotations, moderation verdicts, the id of the worker that served each turn. The array you send is not that record; it is the complete input to one function call, and every entry in it is text the model will read and be influenced by. Anything you put there because it seemed like a convenient place to keep it is something you are also paying to send and asking the model to interpret.

It is also not portable in the way the family resemblance suggests. Chat APIs across providers converge on the same idea — an ordered list of role-tagged turns — and diverge on nearly every detail around it: what the array is called, whether the instruction channel is a top-level field or an array entry, whether the ceiling parameter is named max_tokens or something else, whether consecutive same-role entries are merged or rejected. Code written against one shape and pointed at another fails in the validator, which is the good case, or silently drops a field, which is not. Read the reference for the provider you are on; the concepts transfer and the field names do not.

Some of that divergence is arbitrary and some of it is not. A field being called one thing rather than another is arbitrary. Whether two consecutive user entries are merged or rejected is not, because each provider's model was trained around its own markers and its own idea of a well-formed rendering, and a validator that accepts an arrangement the model never saw during training hands you a request that gets answered oddly rather than one that fails.

One JSON document, four fields that matter, and an ordered array whose positions are the conversation. Everything the rest of this topic deals with is a property of that array: what a validator will and will not accept in it, how large it gets, who holds it between turns, and what happens when the same one is sent twice. You now have a body you can build, an envelope you can read, and a multi-turn call that works without a library between you and it, which is the hour that makes all of those legible.

What the body does not yet contain is an instruction — the standing rule that governs how the assistant behaves across every turn, the "you are a support agent for a bicycle retailer and you never quote prices" that has to survive whatever the user types next. On the provider used above that instruction does not go in the array at all, and where it goes changes how strongly the model weights it.