1.6 Norms, projections and least squares
You can derive linear regression from geometry alone.
Before:00. Orientation & SetupUnlocks:03. Data Handling & Analysis04. Classical AI — Agents, Search & Knowledge Representation
Least squares is the geometry of finding the closest point in a subspace, and from that single picture linear regression falls out with no calculus at all. Norms decide what close means, which is why L1 and L2 regularisation behave so differently downstream. It closes the linear algebra run by cashing the geometry in for the course's first model. The trap is solving the normal equations blindly on ill-conditioned data — the QR route exists because the textbook formula can be numerically treacherous.
Work through these
L1, L2, L-infinity and Frobenius norms
Different ways of measuring the size of a vector or matrix, each preferring a different shape of answer. Which norm a method uses is frequently the whole difference between two otherwise identical techniques.
Orthogonal projection onto a subspace
Projecting a point onto a subspace finds the closest point in it, and that is the geometric content of fitting a model. Everything about least squares follows from this one picture.
Normal equations and the QR route
Two routes to the same fitted line: solving directly, or using a decomposition that is numerically better behaved. Knowing why the second is preferred in software is worth the detour.
Why regularization is a constrained projection
Adding a penalty to a fit is the same thing as restricting the answer to a region, seen from the other side. That equivalence makes regularisation feel principled rather than arbitrary.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.