5.18 Hyperparameter optimization
You can tune systematically instead of by feel.
Before:03. Data Handling & AnalysisUnlocks:06. Deep Learning10. Production & MLOps13. Capstones, Portfolio & Interviews
Hyperparameter search turns tuning from folklore into procedure: random search beats grid in high dimensions, and Bayesian methods like Optuna spend a budget where it pays, with halving schemes killing weak candidates early. It sits late in the module because it optimises models already understood. The failure it warns against is validation overfitting — tuning against one validation set until the score is optimistic by construction, which is leakage conducted in slow motion.
Work through these
Grid vs. random search
Trying every combination or trying random combinations, with the surprising result that random search finds good settings faster when only a few settings matter. That result is worth knowing before writing any search.
Bayesian optimization and Optuna
Methods that model which settings look promising and search there, along with a widely used library. They pay off when each evaluation is expensive.
Successive halving and Hyperband
Giving many candidates a small budget and only continuing with the survivors, which finds good settings for far less computation. It suits training runs that can be stopped early.
Budgeting search and avoiding validation overfit
A search that is allowed to run long enough will fit the validation data, and there needs to be a budget decided in advance. Otherwise tuning becomes another route to a score that will not hold.
Sign in to keep your progress.
Free resources
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.