5.13 Gradient boosting: XGBoost, LightGBM, CatBoost

What this is and why it exists

Gradient boosting is what wins tabular problems. Where a forest builds many independent trees and averages them, boosting builds trees in sequence, each one fitting what the ensemble still gets wrong. That difference makes it more accurate and more delicate — it can overfit in a way a forest essentially cannot — so this topic is as much about the guard rails as about the method.

The vocabulary

• Boosting — building models in sequence, each correcting the previous ensemble.
• Residual — what the current ensemble still gets wrong; more precisely, the gradient of the loss.
• Weak learner — a deliberately shallow tree, contributing a small correction.
• Learning rate (shrinkage) — how much of each new tree's correction is applied.
• Early stopping — halting when the validation score stops improving.
• Subsample — fitting each tree on a fraction of the rows, for regularisation and speed.
• Histogram binning — bucketing continuous features so split-finding is fast.
• Leaf-wise growth — growing whichever leaf promises most, rather than level by level.

The mental model

Start with a constant prediction — the average — and then repeat: look at what the ensemble currently gets wrong, fit a small tree to that error, and add a fraction of it to the ensemble. After a few hundred rounds, the accumulated small corrections describe a very detailed function.

The word "residual" is a helpful simplification and the accurate version is worth having, because it explains the flexibility. Each tree is fitted to the negative gradient of the loss with respect to the current predictions. For squared error that is exactly the residual; for other losses it is something else, and that is precisely why the same machinery handles classification, ranking, quantile regression and custom objectives — you change the loss and the gradients change with it.

The learning rate is the central setting and its behaviour is the opposite of intuition: a smaller learning rate is better, and costs more trees. Each tree contributes only a fraction of its correction, so the ensemble approaches the answer in small careful steps and generalises better; you need more of them. The practical consequence is that the learning rate and the number of trees are one decision, not two — halve the rate and you need roughly twice the trees.

Which makes early stopping the essential guard rather than a nicety. Hold out a validation set, watch its score after each round, and stop when it has not improved for a set number of rounds. Boosting will keep reducing training error indefinitely; the validation curve is the only thing that says when the additional detail stopped being signal. A boosted model trained without early stopping and without a validation set is overfitted by construction, and the training score will not tell you.

Then the tuning order, because the parameters interact and poking at all of them at random is how a day disappears. Fix the learning rate low — small enough to be safe — and let early stopping choose the number of trees. Then tune complexity: depth, or the number of leaves, plus the minimum examples per leaf. Then tune the sampling of rows and features per tree, which is regularisation with a speed bonus. Then, if it matters, the explicit penalties. Doing it in that order means each setting is chosen against a sensible value of the ones before it, which random search across all of them at once does not give you.

The three widely-used implementations differ mostly in engineering rather than in principle. All three bin continuous features into histograms, which is what makes split-finding fast. They differ in how trees are grown — level by level, or leaf by leaf choosing whichever leaf promises the most improvement, which is faster and more prone to overfit without a leaf limit — and, most usefully for a practitioner, in how they handle categorical features. One expects you to encode them yourself; one accepts declared categorical columns directly; one is built around a target-based categorical encoding computed with an ordering scheme designed to avoid the leakage the feature-engineering topic warned about. If your data is heavily categorical, that difference is the one that will decide which you pick.

And the honest comparison. Boosting will usually beat a random forest on tabular data once tuned, and it will do worse if it is not tuned or if early stopping is missing. So the sequence that costs least and teaches most is: baseline, then forest in one command, then boosting with early stopping and the tuning order above. If boosting does not beat the forest, something is wrong with the setup rather than with the data.

What you should now be able to explain or do

Say how boosting differs from bagging in one sentence. Explain what each tree is actually fitted to, and why that generalises to other losses. State the learning-rate rule and say why it makes the rate and tree count one decision. Explain why early stopping is the essential guard and what a training curve will not tell you. Give the tuning order and say why the order matters. Name the practical difference between the three implementations that will decide your choice. Say what it means if boosting does not beat your forest.

Check yourself