Top-p

Also known as: Nucleus Sampling

Top-$p$ (nucleus) sampling keeps the smallest set of tokens whose cumulative probability mass exceeds $p$ (e.g., 0.9), then samples from that set—usually after temperature scaling.

It adapts the candidate set to the distribution’s shape: tighter when the model is confident (mass on few tokens), wider when uncertain. That is why it is often preferred over fixed top-$k$ alone.

Knobs

  • $p \approx 0.9$–$0.95$ is a common chat default
  • Lower $p$ → safer, more conservative text
  • Combine with temperature; do not crank both wildly at once

Nucleus sampling is a decoding / sampling choice at inference time—it does not change trained weights. Measure task metrics (factuality, diversity, format adherence) when you retune $p$.

← Back to AI Glossary