5.11 Decision trees
You can read a tree and explain a prediction to a manager.
Before:03. Data Handling & AnalysisUnlocks:06. Deep Learning10. Production & MLOps13. Capstones, Portfolio & Interviews
A decision tree is the most explainable model you will ever train: a prediction is a path that can be read aloud to a manager. Depth and leaf-size controls are what stand between a useful tree and a memorised dataset. It sits before the ensembles because they are built from it. The honest weakness is instability — a single deep tree changes shape on small data changes, which is not a reason to abandon trees but the exact reason ensembles exist.
Work through these
Splitting criteria: Gini, entropy, MSE
How a split is chosen: two impurity measures for classification and squared error for regression. The criterion decides the tree's shape and is worth understanding rather than accepting.
Depth, leaf size, pruning
Left unrestricted a tree memorises the training data perfectly, so its size has to be limited during growth or reduced afterwards. These controls are the main defence against overfitting.
Handling categoricals and missing values
Trees handle categorical features and missing values more naturally than most methods, and the details differ between implementations. Knowing your implementation's behaviour avoids surprises.
Instability and why ensembles follow
Small changes in the data can produce a completely different tree, which is the weakness that motivates the two topics after this one. The instability is the point, not a footnote.
Sign in to keep your progress.
Free resources
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.