5.22 Recommender systems

What this is and why it exists

Recommenders decide what a very large number of people see every day, and they are the natural closing topic for this module because building one uses almost everything in it — a matrix decomposition, a similarity measure, a ranking metric, a split that respects time, and a baseline that is embarrassingly hard to beat. That baseline is the point. Recommending the most popular items is accurate, safe and useless, and the actual work is beating it honestly.

The vocabulary

• Feedback matrix — users down one side, items across the other, filled with what happened between them.
• Explicit feedback — a rating somebody deliberately gave.
• Implicit feedback — a click, a view, a play; abundant, noisy, and positive-only.
• Collaborative filtering — recommending from patterns of who interacted with what.
• Matrix factorization — learning a short vector for every user and every item so their dot product reproduces the matrix.
• Embedding — that short vector, learned rather than designed.
• Content-based filtering — recommending from the properties of the items themselves.
• Cold start — a user or item with no history to learn from.
• Popularity bias — the tendency of a system to keep recommending what is already popular.

The mental model

Everything starts from one table. The Google recommendation course describes the training data as "a feedback matrix in which: Each row represents a user. Each column represents an item", and every method in this topic is a different answer to the same question: the matrix is mostly empty, so what belongs in the empty cells?

Collaborative filtering answers it from the pattern of interactions alone, using — in the course's words — "similarities between users and items simultaneously to provide recommendations". There are three levels of sophistication and they are worth separating.

User-based: find people whose interactions resemble yours, and recommend what they liked that you have not seen. Intuitive, and it scales badly, because the neighbour search grows with the number of users and users change faster than items do.

Item-based: find items that are interacted with by the same people, and recommend items similar to the ones you already like. This is the workhorse of the two neighbourhood methods, because item-to-item similarities are far more stable than user-to-user ones and can be computed in advance.

Matrix factorization is the classical strong approach. The course calls it "a simple embedding model": learn a user embedding matrix in which "row i is the embedding for user i", learn an item embedding matrix likewise, and predict an interaction as the dot product of the two. Both are learned together by fitting the observed entries. What makes this powerful is that the embedding dimensions are discovered rather than designed — nobody labels a film as slow-paced or a shopper as price-sensitive, and the model finds whatever axes explain the data. The course names this as the headline advantage: "We don't need domain knowledge because the embeddings are automatically learned", along with serendipity, since "the model can help users discover new interests".

Then the shift that matters most in practice: real feedback is implicit. You do not have ratings, you have clicks and plays and purchases. Every entry you observe is a positive, and there are no negatives at all — an empty cell means the person did not interact, which may mean dislike or may mean they never saw it. Fitting only the observed entries in that setting fails in a specific and instructive way. The course states it directly: summing "only over values of one is not a good idea — a matrix of all ones will have a minimal loss" and produces recommendations that are useless. The standard repair is to include the empty cells at a lower weight: weighted matrix factorization "decomposes the objective into the following two sums: A sum over observed entries" and "A sum over unobserved entries (treated as zeroes)", with a weight controlling how much the second matters. That weight is the model's statement about how strongly a non-interaction should be read as a negative, and it is one of the most consequential settings you will choose.

Content-based filtering answers the empty-cell question differently: describe the items by their own properties — text, category, author, tags — describe a user by the properties of what they have engaged with, and recommend on the match. It needs no other users at all, which is exactly the property that makes it valuable, because a brand new item with no interactions still has a description. Its weakness is the mirror image: it recommends more of the same, and it never surprises anybody.

Hybrids are what production systems actually run, because the two failure modes are complementary. Collaborative filtering is strong where history exists and helpless where it does not; content-based filtering works from the first moment and narrows over time. Combine them by blending scores, by generating candidates with one and reordering with the other, or by feeding item and user features into the same model alongside the learned embeddings — the course notes that side features can be incorporated by augmenting the input matrix with feature blocks, so that the system learns embeddings for those features as well.

Evaluation is where recommenders are most often flattered. Error measures are the wrong instrument, because nobody sees a predicted rating — people see an ordered list, and only the top of it. Use ranking measures: precision at a cut-off, recall at a cut-off, and normalised discounted cumulative gain, which scikit-learn's reference describes procedurally — "sum the true scores ranked in the order induced by the predicted scores, after applying a logarithmic discount", then "divide by the best possible score (Ideal DCG, obtained for a perfect ranking) to obtain a score between 0 and 1". The logarithmic discount is the part that matters: a relevant item at position two counts for much more than the same item at position twenty, which is how people actually read a list.

Two rules keep those numbers meaningful. Split by time, never at random — the same lesson as forecasting. Randomly holding out interactions lets the model learn from a person's later clicks and be scored on their earlier ones, which is a fantasy score. Cut at a date, or hold out each user's most recent interactions. And always report the popularity baseline next to your model. Recommending the most popular items scores well on every ranking metric, because popular items are by definition the ones most people interact with. A model that beats it by a small margin has not earned deployment, and finding that out from your own report is far better than finding it out afterwards.

Cold start is structural, and it comes in three forms. A new item has no interactions, and the course is blunt about the consequence: "If an item is not seen during training, the system can't create an embedding for it and can't query the model with this item." The remedies are content features, which work from the first moment, and — as the course puts it — "heuristics to generate embeddings of fresh items", such as averaging the embeddings of items in the same category. A new user has no history, so use context you do have (time, device, where they arrived from), ask a few onboarding questions, and fall back to popular-within-category until history accumulates. A new system has neither, which is why the first version of a recommender is usually content-based or rule-based, and that is the correct decision rather than a compromise.

Popularity bias is the last idea and the one people underestimate, because it is a loop rather than a flaw. The system recommends popular items; those items get shown more; being shown more, they get more interactions; the next model trains on that data and finds them more popular still. Within a few cycles the catalogue has collapsed to a few hundred items and nobody chose that. The training data you have tomorrow is produced by the model you deploy today. Three countermeasures are standard: reserve a small share of recommendations for exploration, so items get a chance to be seen on merit; weight training examples against how much exposure each item received, so a click on something rarely shown counts for more; and hold out a slice of traffic served randomly, which is the only data that can tell you what people would have liked rather than what they were offered. Report catalogue coverage — the share of items ever recommended to anyone — alongside your accuracy numbers, because a system with an excellent score and two percent coverage is a popularity chart wearing a costume.

What you should now be able to explain or do

Describe the feedback matrix and say what every method here is answering about it. Distinguish user-based, item-based and matrix factorization, and say why item-based similarities are the more stable pair. Explain what matrix factorization learns and why the dimensions need no labels. Say what changes under implicit feedback, why fitting only the observed entries fails, and what the weight on unobserved entries means. Say what content-based filtering can do that collaborative filtering cannot, and design a hybrid on that basis. Choose ranking metrics over error metrics and explain the logarithmic discount. Split by time and report the popularity baseline. Name the three forms of cold start with a remedy for each. Explain the popularity feedback loop and give three countermeasures.

Check yourself