The token budget

Write the subtraction out with every line visible: a 128k effective window, a 4k system prompt, 6k of tool schemas and a guaranteed 2k answer leaves a little under 116k, and less once the template, the user's message and the conversation history land. The answer is the residual claimant and the only line item that cannot object, so max_tokens does not reserve anything — your assembler does. Compute the remaining room per request and decide in code what gives way when the content will not fit.

Context window — read the current version at https://daviddominguez.dev/learn/en/context-window/the-token-budget

Back to top