1.1 Vectors, spaces and geometric intuition

What this is and why it exists

Every example a model ever sees is a vector — a house is (area, rooms, age, price), a word is 300 numbers, an image is thousands. Linear algebra is the grammar of such lists, and this topic installs its first habit: reading a vector equation and SEEING a picture. The dot product alone, understood geometrically, unlocks half of machine learning's vocabulary — similarity, projection, attention scores.

The vocabulary

• Vector — one object, three faces: a point in space, an arrow from the origin, a list of feature values. Fluency is switching faces mid-thought.
• Dot product — multiply matching components and add; geometrically, the product of the lengths times the cosine of the angle between.
• Projection — the shadow of one vector along another; how much of THIS points along THAT.
• Cosine similarity — the dot product with the lengths divided out: pure direction agreement, from 1 (same direction) through 0 (unrelated) to −1 (opposite).
• Linear independence — no vector in the set is a combination of the others; nothing is redundant.
• Span, basis, dimension — everything a set of vectors can reach; a minimal independent set that reaches it; how many vectors that minimal set needs.
• Subspace — a flat region through the origin (line, plane, …) closed under adding and scaling.

The mental model

The dot product is a similarity meter. Two feature vectors pointing the same way — customers with the same tastes, words used in the same contexts — have a large positive dot product; unrelated ones sit near zero; opposites go negative. Divide out the lengths and cosine similarity remains, which is exactly how search engines rank documents and how attention in a transformer scores which words to look at. When the dot product is zero the vectors are orthogonal — carrying independent information — and when you project b onto a, you split b into "the part along a" plus "the part a knows nothing about". That decomposition, repeated, IS most of applied linear algebra: least squares (la6) and PCA (la4, la5) are both projections.

Span, basis and dimension answer "how big is this space, really?". Three vectors that all lie in one plane span only that plane — one of them is redundant, dependence in action. A basis is a coordinate system: independent vectors whose span covers the space, so every point has exactly one address. Dimension counts them. The punchline for ML: a dataset with 100 columns rarely fills 100 dimensions — heights and weights correlate, pixels repeat their neighbours — so the data hugs a lower-dimensional subspace inside the ambient one. Finding that subspace and dropping the rest is dimensionality reduction, and it is why the geometry of this topic is a practical tool rather than a formality.

What you should now be able to explain or do

Read a vector three ways and pick the useful face per problem. Compute a dot product and state its geometric meaning. Project one vector onto another and name the two parts. Test small sets for independence, and explain in one sentence why real data lives below its column count.

Check yourself