1.9 Convexity and the shape of loss surfaces
You can say whether an optimizer will find the answer or just an answer.
Before:00. Orientation & SetupUnlocks:03. Data Handling & Analysis04. Classical AI — Agents, Search & Knowledge Representation
Convexity is what makes an optimisation problem honest: any minimum found is the minimum. Deep networks are not convex and train anyway, and knowing why — saddle points are escapable, plateaus dominate, most local minima are fine — changes how training curves get read. It sits between the gradient machinery and the optimisers that use it. The common confusion is fearing local minima in deep learning; the real enemies are plateaus and saddle regions, which look identical from the loss curve.
Work through these
Convex sets and convex functions
A convex function has a single valley with no local traps, which is the property that makes optimisation guaranteed rather than hopeful. Recognising convexity tells you what a solver can promise.
Local vs. global minima, saddle points, plateaus
In a non-convex landscape a method can stop at a local dip, a saddle, or a flat region, and each has a different remedy. Distinguishing them from the symptoms is a practical diagnostic skill.
Why deep nets are non-convex and still trainable
Deep networks have wildly non-convex loss surfaces and are trained successfully anyway, which is a genuine tension worth understanding rather than glossing over. The reasons are partly geometric and partly empirical.
Lipschitz continuity and step-size intuition
A bound on how fast a function can change tells you how large a step can be taken without overshooting. It is the theory behind advice about learning rates that otherwise sounds like folklore.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.