2.13 Type hints, mypy and Pydantic

What this is and why it exists

Type hints turn your editor into a proof-reader: it can tell you that a function returns None on one path before you run anything, and it can complete attributes it now knows exist. They also make a signature self-documenting, which matters more in a data project than almost anywhere else, because "what shape is this thing" is the question you ask constantly. And they come with a limit you must know precisely, because getting it wrong produces false confidence.

The vocabulary

• Annotation — the type written after a colon on a parameter or after an arrow on a return.
• Static type checker — a tool that reads annotations and reports contradictions without running the code; mypy is the common one.
• Optional — a value that may be the type given or None.
• Union — a value that may be any of several types.
• Literal — a value restricted to specific constants.
• TypeVar — a placeholder standing for "some type", used to say that two places must agree.
• Protocol — a structural description: anything with these methods satisfies it, whatever it inherits from.
• Runtime validation — checking actual values as they arrive, which annotations do not do.

The mental model

The single most important sentence about annotations is the reference's own: "the Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc." So a hint is a statement of intent that a tool can check and the interpreter ignores entirely. Annotate a parameter as an integer and pass a string, and Python runs it happily until something breaks further down.

That fact draws the line this topic exists to teach. Annotations catch your mistakes; validation catches the world's. Inside your own code, where you control both sides, hints plus a checker are enough. At the boundary — data arriving from a file, an interface, a user, a model's output — the values are outside your control and something must actually look at them. Hints alone at a boundary are a comment with better syntax.

Which is where Pydantic comes in: it uses the same annotations, and at runtime it checks and converts incoming data against them, raising a clear error listing exactly which fields were wrong. The same declaration therefore does two jobs — it documents the shape for your editor and enforces it where data enters. Use plain dataclasses for internal structures and Pydantic models at the edges, and the division of labour is clean.

The types worth knowing early are few. Optional is a value or None, and the reference notes it "is equivalent to X | None" — the modern spelling with a vertical bar is fine and shorter. A union covers several possibilities. Literal restricts a value to specific constants, which is the type-checker equivalent of an enum for the small closed sets that live in function arguments. And a Protocol is the structural version of an interface: the reference's example shows a class satisfying a protocol without inheriting from it, purely by having the right method — which is duck typing made checkable, and exactly right for a codebase where anything with fit and predict should be acceptable.

TypeVar is the piece people postpone and should not, because one use of it is genuinely everyday: saying that a function returns the same type it was given. Without it you would annotate a return as "some object" and lose everything the checker knew; with it, passing a list of integers means the checker still knows integers come back.

Running the checker in CI is what makes any of this real. Hints nobody verifies drift out of date exactly like comments, and drifted hints are worse than none because they are believed. Start permissively on an existing codebase, tighten as you go, and treat a checker failure like a test failure.

In code

Checked against the typing reference.

from typing import Literal, Protocol, TypeVar
from pydantic import BaseModel

T = TypeVar("T")


def first_or_none(items: list[T]) -> T | None:
    """The return type follows the input type, which is what TypeVar buys."""
    return items[0] if items else None


Split = Literal["train", "validation", "test"]


def load(split: Split, limit: int | None = None) -> list[dict]:
    ...


class Estimator(Protocol):
    def fit(self, x, y) -> None: ...
    def predict(self, x) -> list[float]: ...


class ScoreRow(BaseModel):
    """A boundary type: this one is checked at runtime, not only by the editor."""

    name: str
    subject: str
    score: float


row = ScoreRow.model_validate({"name": "asha", "subject": "maths", "score": "88"})
print(row.score)      # 88.0 — checked and converted, or a clear error

Pydantic's own documentation describes models as "classes which inherit from BaseModel" with fields declared as "annotated attributes", and model_validate as validating the given object — casting input "to force it to conform to model field types", ensuring the result conforms, and raising a validation error "whenever it finds an error in the data it's validating". Per-field constraints go through its Field() function, which the reference says customises "default values, JSON Schema metadata, constraints"; the constraints available differ by type, so look the argument names up in its reference rather than guessing them.

Note what each part is doing. Split as a literal means passing "trian" is an error your editor shows while you type, not a mystery at runtime. first_or_none returns whatever type it was given or None, so the checker keeps following the types through your code. And ScoreRow is the only one of these that actually inspects a value — because it sits where untrusted data arrives, and it converts the string "88" into a number rather than letting it travel on as text.

What you should now be able to explain or do

Quote the runtime-enforcement rule and say what follows from it. Draw the line between where hints suffice and where validation is required, and say what a hint at a boundary really is. Use Optional, a union and a Literal correctly. Say what a Protocol expresses that inheritance does not. Explain what TypeVar preserves in a return type. Say why an unchecked hint is worse than no hint.

Check yourself