13.1 Capstone 1 — tabular ML, deployed

What this is and why it exists

This is the first of four finished projects, and it will probably be the strongest single artefact you own: a live service that takes an input and returns a calibrated prediction, with a write-up somebody can follow. Everything tabular you have learned is exercised at once. The decision that matters most is made before any modelling — choose data attached to a real decision, because interviewers ask about the decision, the leakage risks and the limitations, and a scrubbed competition dataset has nothing to say about any of them.

The vocabulary

• Real decision — an action somebody would actually take differently given the prediction.
• Leakage — information in the features that would not exist at prediction time.
• Calibration — predicted probabilities meaning what they say.
• Baseline — the simplest sensible alternative, reported alongside.
• Service — the model behind an interface a stranger can call.
• Limitations section — the written account of what the model cannot do.

The mental model

Choose the dataset by asking what would change, and keep asking until the answer is concrete. Who acts on this prediction, what do they do differently, and what does being wrong cost them? A dataset with a good answer gives you everything the rest of the project needs: a metric that means something, a threshold with a reason, a cost of error you can name, and a limitations section that writes itself.

A scrubbed competition dataset gives you none of that. It has been cleaned by somebody else, the leakage removed by somebody else, the metric chosen by somebody else, and the decision it attaches to does not exist. You can score well on it and have nothing to say about it, which is precisely the conversation an interview turns into.

Better sources: a public dataset from a government or civic body, where the decision is real even if you are not the one making it; data from a domain you know, where you can tell a plausible value from an implausible one; something you assembled yourself, which is more work and is distinctively yours. Prefer a small messy dataset attached to a real decision over a large clean one attached to nothing.

Then the modelling sequence, done honestly, which exercises the whole classical module.

Exploration first, and it is not decoration. Look at distributions, missingness, obvious errors, duplicated rows and the target's balance. Find out how the data was generated, because that decides everything about the split. And look for the feature that is too good — the one correlating almost perfectly with the target — because it is nearly always leakage, and finding it now is far better than in an interview.

Feature engineering with the leakage rule in front of you: every feature must be computable at prediction time, from information that exists then. A field filled in by a person after the outcome is known is the classic trap, and it looks wonderful.

Model selection starting from a baseline you report. Predict the majority class, or a simple rule from the domain, then a linear model, then a gradient-boosted one. Report all of them, because the gap between the baseline and your best model is the honest measure of what the modelling contributed, and a report without it is a number without a reference point.

Validation that respects how the data was generated: split by time if the data is temporal, by group if rows cluster by customer or session or device, stratified if a class is rare. Randomly splitting grouped data is the leak the sensor topic named and it inflates every number.

Calibration is what makes it a prediction rather than a score, and it is the detail that distinguishes this project. A model that says seventy percent should be right about seventy percent of the time when it says that. Plot the reliability curve, calibrate on held-out data if it is off, and choose the decision threshold from the cost of each error rather than accepting one half. Being able to explain your threshold is worth more in a conversation than a higher score.

Deploying it is what makes it showable. A model that lives in a notebook cannot be shown to anybody — a reviewer will not run your notebook, and "it works locally" is what everybody says.

So: a small service with validated input and output shapes, the model loaded once at startup, a warm-up prediction, a health endpoint, and structured logs — all of it from the serving topic. Containerise it, deploy it somewhere a stranger can reach it, and keep it running. Free tiers are adequate for a portfolio service, and note that a service scaled to zero has a slow first request, so say so in the README rather than letting a reviewer conclude it is broken.

The write-up is the part most people underinvest in and reviewers read first. Four sections.

The problem: what decision, for whom, and what being wrong costs. Two paragraphs, in plain language, with no technique in them.

The approach: what you did and, more importantly, why — why that split, why that metric, why that threshold. The reasons are the content; the list of models tried is not.

The results: your number beside the baseline, with the per-segment breakdown and the confusion matrix, and the run-to-run spread so a reader knows what a difference means.

The limitations: what the model cannot do, which populations it is likely worse on, what would make it degrade, what you would need to deploy it for real, and what you would do next. This is the section a knowledgeable reader looks at first, and a project without one reads as unfinished regardless of the score.

Two closing habits. Write the README as though the reader has three minutes and no context — what it does, a picture or a sample request and response, how to run it, and the honest state of it. And record what you would do differently, because that answer is asked in nearly every interview and thinking about it once, in writing, is what makes it a good answer rather than an improvised one.

What you should now be able to explain or do

Choose a dataset by asking who acts on the prediction and what error costs, and say why a scrubbed competition set fails that test. Explore for generation process, leakage and the too-good feature. Engineer features under the availability rule. Report a baseline alongside your model and say what the gap measures. Split by time, group or stratum as the data requires. Calibrate, and choose a threshold from costs you can name. Deploy a validated, containerised service a stranger can reach. Write the four-section account, with the limitations section written honestly.

Check yourself