5.8 k-nearest neighbours and distance methods
You understand the simplest possible learner and its cost.
Before:03. Data Handling & AnalysisUnlocks:06. Deep Learning10. Production & MLOps13. Capstones, Portfolio & Interviews
k-nearest neighbours is the simplest possible learner — store everything, vote among neighbours — and it teaches two lessons that outlive it: distance depends entirely on feature scaling, and high dimensions break geometric intuition. It sits early among the algorithms as the baseline with a lesson. The unscaled-feature failure is worth experiencing once deliberately: with metres in one column and millimetres in another, the distance is measuring units, not similarity.
Work through these
Distance metrics and feature scaling
The method depends entirely on what counts as close, and features on different scales quietly dominate any distance. Scaling first is not optional here.
Choosing k; the curse of dimensionality
How many neighbours to consult trades noise against blurring, and in many dimensions everything becomes roughly equidistant. That second effect is why the method fades on wide data.
KD-trees, ball trees, approximate search
Data structures and approximations that make neighbour lookup fast enough to use at scale. Without them the method is quadratic and unusable.
Where k-NN still wins
Despite its simplicity it remains competitive on small, low-dimensional problems, and it is the natural baseline for similarity search. Knowing where it wins keeps it in your toolkit.
Sign in to keep your progress.
Free resources
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.