6.10 Debugging deep learning
You can find the bug instead of adding layers.
Before:05. Classical Machine LearningUnlocks:07. Natural Language Processing11. The ECE Bridge — Signals, Edge & Embedded AI
Deep learning bugs rarely crash; they converge to mediocrity silently, which is why debugging is its own skill with its own first move: overfit a single batch, always — a healthy setup must be able to memorise ten examples, and one that cannot has a bug, not a capacity problem. It closes the craft run. The reflex it replaces is adding layers or epochs in response to a bad curve, when shapes, shuffling and normalisation are the usual culprits.
Work through these
Overfit one batch first — always
Before anything else, check the model can memorise a single batch. If it cannot, the bug is in the code rather than the method, and no amount of tuning will help.
Learning-rate range test
Sweeping the step size across a range and watching where the loss falls fastest gives a starting value in minutes. It replaces guesswork with a short experiment.
Inspecting gradients, activations, weight norms
Watching the size of gradients, activations and weights during training tells you which layer is misbehaving. These are the instruments; adding layers is not a diagnostic.
Common silent bugs: shapes, shuffling, normalization
The bugs that produce a model that trains without error and learns nothing: mismatched shapes that silently broadcast, data that was never shuffled, and normalisation applied inconsistently between training and inference.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.