7.2 Bag of words, TF-IDF and n-grams
You can build a strong text baseline in minutes.
Before:06. Deep LearningUnlocks:08. Large Language Models
Bag-of-words and TF-IDF turn text into sparse vectors that linear models handle brilliantly — a baseline built in minutes and embarrassingly hard to beat on many classification tasks. It sits early in the module as the honest yardstick. Skipping it is the trap with a cost attached: without this baseline there is no way to tell whether a transformer's extra expense bought anything, and often it did not.
Work through these
Count vectorization and sparsity
Representing a document as counts of its words, which produces a very wide and mostly empty matrix. Understanding that sparsity explains the storage and the methods that follow.
TF-IDF weighting
Weighting words by how informative they are rather than how often they appear. It is a small adjustment that improves nearly every count-based method.
n-grams and feature explosion
Using short sequences of words instead of single words captures some word order, and multiplies the number of features alarmingly. Managing that growth is the practical problem.
Baseline text classification with linear models
A linear model on weighted counts is a strong baseline that is fast to train and hard to beat on many tasks. Establishing it before reaching for anything larger is the professional habit.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.