6.2 Backpropagation by hand
You can derive gradients for a two-layer net on paper.
Before:05. Classical Machine LearningUnlocks:07. Natural Language Processing11. The ECE Bridge — Signals, Edge & Embedded AI
Backpropagation on paper for a two-layer network is the rite of passage: chain rule over the graph, local gradients, accumulation — and after it, framework errors about shapes and graphs become legible forever. It follows the perceptron topic because the mechanics need a network to run on. The safety habit taught alongside is gradient checking: a numerical finite-difference comparison catches sign and transpose errors that reread derivations miss.
Work through these
Chain rule over the computational graph
Training is the chain rule applied over a graph of small operations, accumulating derivatives from the loss backwards. This is the entire algorithm, and it is worth deriving rather than accepting.
Gradients for common layers and losses
The derivative expressions for the layers and losses you will actually use. Having these at hand is what makes hand-derivation feasible on a real network.
Implementing backprop in NumPy
Writing the backward pass yourself in array code, without a framework doing it for you. Everyone who does this once reads framework code differently afterwards.
Gradient checking
Comparing your analytic gradients against numerical estimates catches derivation errors that otherwise show up as a model that trains slowly for no visible reason. It is a short check with a large payoff.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.