7.3 Word embeddings
You can explain why king − man + woman lands near queen.
Before:06. Deep LearningUnlocks:08. Large Language Models
Word embeddings place words in a space where distance means relatedness — the famous king minus man plus woman geometry — and they are the conceptual ancestor of every embedding used in search and retrieval today. It sits between the sparse methods and the neural ones as the idea that changed the field. The honesty requirement travels with them: embeddings inherit the biases of their training text, and systems built on them inherit those in turn.
Work through these
word2vec: skip-gram and CBOW
Learning word vectors by predicting context from a word or a word from its context. These two arrangements are the original approach and the ideas remain foundational.
GloVe and co-occurrence factorization
An alternative built by factorising a matrix of how often words appear together. It reaches similar representations from a different direction, which is itself informative.
fastText and subword vectors
Building word vectors from character fragments, which gives sensible representations for words never seen in training. This matters a great deal for morphologically rich languages.
Bias in embeddings and its consequences
Word vectors absorb the associations present in their training text, including ones nobody wants a system to act on. This is a real deployment concern rather than a footnote.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.