8.3 Decoding and sampling
You can control how deterministic or creative a model is.
Before:07. Natural Language ProcessingUnlocks:09. Agentic AI12. Frontier Topics
Decoding settings — temperature, top-k, top-p — are the dial between deterministic and creative output, and grammar-constrained decoding can force valid structure outright. It sits early in the LLM module because every call made after it turns these knobs. The reproducibility myth needs retiring: temperature zero does not make outputs identical, because batching and floating-point nondeterminism still vary them, so systems must be built to tolerate variation.
Work through these
Greedy, beam, temperature, top-k, top-p
The settings that control how the next token is chosen, from always taking the most likely to sampling from a restricted set. These are the controls behind creative versus predictable output.
Repetition penalties
Penalties that discourage the model repeating itself, which addresses a characteristic failure of sampling. They are blunt and need tuning per use.
Constrained and grammar-guided decoding
Forcing output to follow a grammar or schema so it can be parsed. This is far more reliable than asking politely and checking afterwards.
Seeds, determinism and why outputs still vary
Even with a fixed seed, output can vary because of how computation is batched and parallelised. Knowing this prevents a long hunt for a bug that is not there.
Sign in to keep your progress.
Free resources
These videos are on YouTube. Opening the link takes you to YouTube's page. Pressing "Watch here" loads YouTube's player into this page — nothing loads from YouTube until you do. Either way the video comes from Google and uses much more mobile data than a page of text. Something wrong with a link here?
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.