1.8 Chain rule and computational graphs
You can hand-derive backpropagation for a small network.
Before:00. Orientation & SetupUnlocks:03. Data Handling & Analysis04. Classical AI — Agents, Search & Knowledge Representation
Backpropagation is the chain rule organised over a computational graph and applied backwards, and this topic derives it by hand for a small network. Forward versus reverse mode explains why training computes gradients the way it does: reverse wins when parameters are many and the loss is one. It sits right after gradients because it is their choreography. The confusion it dissolves is backprop as mystery — after one hand derivation, framework errors become legible forever.
Work through these
Chain rule in multiple variables
Composing functions means multiplying their derivatives, and in several variables that becomes a product of matrices. This is the entire mathematical content of training a neural network.
Forward vs. reverse mode differentiation
Derivatives can be accumulated from inputs toward outputs or from outputs back toward inputs, and the two cost very different amounts. Which one is cheaper depends on the shape of the problem.
Computational graphs and node-wise local gradients
Drawing a calculation as a graph of small operations, each of which knows its own local derivative, is how automatic differentiation is implemented. It turns a hard derivation into bookkeeping.
Why reverse mode wins when outputs are few
When a function has many inputs and one output, which is exactly the case for a loss function, working backwards is dramatically cheaper. That asymmetry is why backpropagation exists in the form it does.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.