Stateless APIs and conversation history

A model API holds no session: the conversation is an array your client owns, and every call sends the whole of it — system prompt, tool definitions, every earlier user and assistant turn, and every tool result those turns produced. The same tokens are therefore billed again on each turn, which is why a multi-turn feature's bill is dominated by input rather than by the answers it writes. This chapter traces one conversation's usage, itemises the four components estimates routinely omit, and separates the changes that reduce those tokens from the ones that only move where they are stored.

Token economics — read the current version at https://daviddominguez.dev/learn/en/token-economics/stateless-apis-and-conversation-history

Back to top