Module 3 · Running models in production · Lesson 6
Sampling and hallucination
How a model turns logits into text (temperature, top-k, top-p and their limits in today's APIs), and why models state false things fluently, from pretraining statistics to grading incentives, with grounding checks that catch it.
By the end of this lesson you can
- Compute next-token probabilities under different temperatures and apply top-p and min-p filtering by hand
- Explain why temperature 0 is neither deterministic nor a cure for hallucination
- Explain the statistical and incentive-based causes of hallucination identified by recent research
- Design and implement a grounding check that verifies quoted evidence against the source document