6.6 Regularization and augmentation
You can close a train/validation gap deliberately.
Before:05. Classical Machine LearningUnlocks:07. Natural Language Processing11. The ECE Bridge — Signals, Edge & Embedded AI
Deep learning regularisation is a toolkit — dropout, weight decay, early stopping, augmentation, mixup — aimed at closing the train-validation gap deliberately rather than hopefully. Augmentation is often the cheapest large win, especially on images. It sits in the craft run because generalisation is trained, not wished for. The stacking trap is real: every technique at once, at full strength, can leave a model unable to fit even the training set.
Work through these
Dropout and its placement
Randomly switching off units during training forces the network not to depend on any one of them, and where it is placed matters. It is less used in modern architectures and still worth understanding.
Weight decay and early stopping
Penalising large weights, and stopping when held-out performance stops improving. Early stopping is the cheapest regularisation available and it needs a validation set to work.
Data augmentation for images, audio, text
Transforming training examples so the model sees more variety than you collected, with different techniques for each data type. It is frequently the largest single improvement available on small datasets.
Mixup, CutMix, label noise
Blending examples and their labels together, and deliberately imperfect labels, both of which improve robustness in ways that seem counter-intuitive at first.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.