Probing the enforced context limit

A context limit is a property of a deployment rather than of a set of weights, so the number on the model card and the number your endpoint enforces routinely differ. Send requests and find the boundary: one probe with a one-token output request and one with a large one, because some endpoints validate input alone and some validate input plus reserved output. Record the advertised figure, the effective ceiling and the date, and treat the output cap as a separate limit with its own error.

Context window — read the current version at https://daviddominguez.dev/learn/en/context-window/probing-the-enforced-context-limit

Back to top