2.17 Profiling and making Python fast

What this is and why it exists

Python is fast enough for far more than people assume, and when it is not, the slow part is reliably not the one you suspected. This topic is about replacing that suspicion with evidence, and then applying the fixes in the order that actually pays: better algorithm first, vectorisation second, caching third, and compilation last — which is the reverse of the order most people try.

The vocabulary

• Profiling — measuring where the time actually goes.
• Deterministic profiler — one that records every call; cProfile is the standard library's.
• Line profiler — one that reports time per line inside a chosen function.
• Cumulative time — time spent in a function including everything it called.
• Total (own) time — time spent in the function itself, excluding calls it made.
• Hot path — the small part of the code where most of the time is spent.
• Vectorisation — replacing a Python loop with an array operation that runs in compiled code.
• Compilation — turning Python into machine code ahead of or during the run, as Numba and Cython do.

The mental model

Measure first, and mean it. A change made without a profile is a guess, and the usual cost of guessing is readable code turned into unreadable code of the same speed. Run cProfile over a representative workload — representative matters, because a profile of a toy input tells you about the toy — and read two columns. Cumulative time finds which part of the program owns the problem; own time finds the function actually burning the cycles. Then narrow with a line profiler on that one function, because "this function takes eight seconds" and "this line inside it takes eight seconds" are different amounts of information.

Then the fixes, in order.

Algorithm first, because nothing else competes. A membership test against a list inside a loop, a nested loop doing work a dictionary would answer directly, a file re-read on each iteration — these are the finds that turn minutes into seconds, and no amount of compilation rescues an approach that does too much work. This is the containers topic arriving with a stopwatch attached: the reason to know that sets test membership by hashing is that you will meet the loop where it matters.

Vectorisation second, for numerical work. NumPy performs an operation over a whole array in compiled code, so a Python loop over a million elements becomes one call. The change is usually large — often an order of magnitude or more — and it happens to be less code rather than more, which makes it the rare optimisation that improves readability. It also has a second payoff mentioned in the previous topic: those operations release the interpreter lock, so they were never competing for it in the first place.

Caching third. If the same expensive answer is computed repeatedly with the same inputs, remember it — the decorator from the earlier topic does this in one line. The conditions are the ones stated there: the arguments must be hashable, and the function must be pure, or you will serve a stale answer forever.

Compilation last. Numba compiles a numerical function at runtime; Cython compiles annotated Python ahead of time. Both can be dramatic on a genuinely tight numerical loop that will not vectorise. Both also add a build step, a dependency and a debugging cost, so they earn their place only after the first three have been tried and measured. The question to ask before reaching for them: has this loop already been vectorised, and is it still the hot path?

One more practical note. Memory is often the real constraint in data work rather than processor time, and the symptom is different — the machine slows to a crawl or the process is killed, rather than the work merely taking longer. When that happens the fix is usually the generators topic: process in a stream rather than loading everything, or work in chunks. A memory profiler tells you which structure is holding the space, and the answer is frequently one intermediate list that nobody needed to keep.

In code

Checked against the cProfile and timeit references.

import cProfile
import pstats

cProfile.run("build_features(rows)", "profile.out")
stats = pstats.Stats("profile.out")
stats.sort_stats("cumulative").print_stats(15)
stats.sort_stats("tottime").print_stats(15)

The two sorts answer the two different questions: cumulative for which area owns the time, own time for which function is burning it.

And the algorithmic find, which is the one you will make most often:

# Before: membership against a list, inside a loop over rows.
known = load_known_names()                 # a list of 200,000 names
flagged = [r for r in rows if r["name"] in known]

# After: one line changed, the same result, a different order of magnitude.
known = set(load_known_names())
flagged = [r for r in rows if r["name"] in known]

Then vectorisation, which is the second:

import numpy as np

# Before: a Python loop over a million values.
normalised = []
for v in values:
    normalised.append((v - mean) / std)

# After: one array operation, in compiled code.
values = np.asarray(values)
normalised = (values - mean) / std

What you should now be able to explain or do

Profile a real workload and read both the cumulative and the own-time columns, saying what each answers. Narrow to a line with a line profiler. Apply the four fixes in order and say why the order is what it is. Recognise the membership-in-a-list find. Vectorise a numerical loop and explain the two reasons it is faster. Say when compilation is justified and what it costs. Distinguish a memory problem from a speed one by its symptom.

Check yourself