1.7 Derivatives, gradients, Jacobians, Hessians
You can compute the gradient of a multivariate function correctly.
Before:00. Orientation & SetupUnlocks:03. Data Handling & Analysis04. Classical AI — Agents, Search & Knowledge Representation
The gradient is the direction of steepest ascent, and every neural network trains by following its negative — this topic makes computing it correct and comfortable, then extends to Jacobians for vector outputs and Hessians for curvature. It opens the calculus sequence because backpropagation is nothing but these objects organised. The quiet bug it warns against is fuzzy indexing: being unsure whether Jacobian rows are outputs or inputs silently breaks derivations later.
Work through these
Partial derivatives and directional derivatives
A partial derivative varies one input while holding the others fixed; a directional derivative asks how the function changes along any chosen direction. Both are needed before gradients make sense.
Gradient as the direction of steepest ascent
The gradient collects the partial derivatives into a vector that points in the direction of fastest increase. Every training method in this subject is a variation on moving against it.
Jacobian for vector-valued functions
For a function producing several outputs, the derivatives form a matrix rather than a vector. This is the object that the chain rule composes when networks are stacked.
Hessian, curvature and second-order information
Second derivatives describe curvature, which distinguishes a valley floor from a saddle and tells you how large a step is safe. Methods that use this information converge faster and cost more per step.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.