5.12 Bagging and random forests
You can train a strong baseline in one command and tune it sensibly.
Before:03. Data Handling & AnalysisUnlocks:06. Deep Learning10. Production & MLOps13. Capstones, Portfolio & Interviews
Random forests average many decorrelated trees — bootstrap samples plus feature subsampling — turning an unstable learner into a robust one that is genuinely hard to badly mistune, with out-of-bag error as nearly free validation. It sits directly after single trees as their redemption. The reading to distrust is the default feature-importance chart: impurity-based importances inflate high-cardinality features, and permutation importance is the honest replacement.
Work through these
Bootstrap aggregation and variance reduction
Training many models on resampled copies of the data and averaging them reduces the sensitivity that a single unstable model has. This is the general technique before it is applied to trees.
Feature subsampling and decorrelation
Also choosing a random subset of features at each split makes the individual models less alike, and less alike models average better. That extra randomness is what separates a forest from plain averaging.
Out-of-bag error
Each model was trained without some of the data, and those left-out examples give a free performance estimate. It removes the need for a separate validation split in some workflows.
Feature importance and its biases
The built-in importance measures are convenient and biased in known ways, particularly toward features with many possible values. Knowing the bias prevents a confident wrong conclusion.
Sign in to keep your progress.
Free resources
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.