5.4 Regularization
You can control overfitting with a knob you understand.
Before:03. Data Handling & AnalysisUnlocks:06. Deep Learning10. Production & MLOps13. Capstones, Portfolio & Interviews
Regularisation is the standard control for overfitting: L2 shrinks weights smoothly, L1 pushes some to exactly zero, elastic net blends the two. Seen through the maximum-likelihood lens from earlier, it is simply a prior on the weights. It sits right after the first models because they are the ones it protects. The mechanical trap is regularising unscaled features — the penalty then punishes features for their units rather than their usefulness, and the fix is one scaler away.
Work through these
Ridge (L2) and its shrinkage effect
Penalising the sum of squared coefficients shrinks them all toward zero without eliminating any. It is the standard first defence against a model fitting noise.
Lasso (L1) and sparsity
Penalising the sum of absolute coefficients drives some of them to exactly zero, which performs feature selection as a side effect. That difference from the previous item is the whole reason both exist.
Elastic net and the tuning trade-off
Combining both penalties, with a setting controlling the mix, which handles correlated features better than either alone. Tuning two knobs instead of one is the cost.
Regularization as a prior
Every penalty corresponds to a prior belief about what the coefficients should look like. Seeing that equivalence makes the choice between penalties a modelling statement rather than a hyperparameter.
Sign in to keep your progress.
Free resources
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.