Sampling temperature

Temperature divides every logit before the softmax, which stretches or flattens the gaps between candidates without touching their order. The direction of the effect is fixed by the arithmetic; the size of it depends entirely on how far apart the top logits already were, which is a number you can read from a log-probability dump before you change anything. This chapter covers the maths, how to predict a change, how temperature interacts with nucleus sampling, and the failures it cannot remove.

Sampling — read the current version at https://daviddominguez.dev/learn/en/sampling/top-p-sampling

Back to top