10.9 Data pipelines and feature stores
Training and serving features cannot silently disagree.
Pipelines and feature stores exist to kill training-serving skew — the disease where features computed one way in training are computed differently live, degrading the model invisibly. Orchestrators make multi-step data flows reliable, and quality checks act as gates. It sits in the advanced production run. The root cause is duplication: feature logic written twice will disagree eventually, and the feature store is the architectural answer to writing it once.
Work through these
Orchestration: Airflow, Prefect, Dagster
Tools that run pipelines on a schedule with dependencies between steps, of which three are widely used. The choice matters less than having one rather than a collection of scheduled scripts.
Batch and streaming ingestion
Data arriving in scheduled batches or as a continuous flow, with different tooling for each. Many systems need both, for different features.
Training/serving skew and the feature store answer
Features computed one way for training and another way for serving will disagree, and the model will quietly degrade. A feature store exists to compute them once and serve both.
Data quality checks as pipeline gates
Checks that stop a pipeline when the data looks wrong, rather than letting bad data reach a model. Failing loudly at ingestion is far cheaper than debugging predictions later.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.