3.6 Exploratory data analysis as a discipline

What this is and why it exists

Exploratory data analysis is a repeatable procedure, not an inspiration. Done as a checklist it surfaces the problems that would otherwise surface in production, and it is where most genuine findings actually come from — more than from any model. Done as a mood it produces a notebook of pretty pictures that nobody, including its author three weeks later, can extract a conclusion from.

The vocabulary

• Univariate — looking at one column at a time.
• Bivariate — looking at two together, usually a feature against the target.
• Multivariate — looking at several at once, including correlations between them.
• Segment — a subset defined by a meaningful attribute: a region, a course, a device.
• Cohort — a group defined by when it started, followed over time.
• Distribution — the shape of a column's values, which averages hide.
• Finding — a written statement with the evidence attached.
• Open question — something the data raised and could not settle.

The mental model

Run the same passes every time, in the same order, and the order is chosen so that each pass would be wasted without the one before it.

Shape first. How many rows and columns, what are the dtypes, how much is missing per column, how many rows are exact duplicates. This takes two minutes and it decides whether the rest of the analysis is worth doing at all — a column that is 90 percent missing is not a feature, and a table where half the rows are duplicated is not the table you think you have.

Then univariate. For each numeric column, the distribution — not the mean, the distribution. Averages hide bimodality, hide a spike at zero, hide the sentinel value of minus one that somebody used for "unknown". For each categorical column, the distinct values and their counts, which is where you meet "Maths" and "maths" as two categories and the trailing space that made a third.

Then bivariate, each feature against the target. This is where the promising signals and the too-good-to-be-true ones both appear, and a feature that separates the target almost perfectly should be treated as a leakage suspect before it is treated as a discovery.

Then multivariate, including how features relate to each other. Two nearly-identical columns are one column measured twice, and knowing that now saves confusion when a model treats their importance as split between them.

Then segments and cohorts, which is where the real findings usually live. An overall average is a mixture of populations, and the moment you split by region, by course, by device or by month, the mixture separates: a metric that looks flat overall is often one segment improving and another declining. Cohorts add time to that — grouping people by when they arrived and following each group forward separates "the product changed" from "the people arriving changed", which are constantly confused and have opposite implications.

Then the part that makes it analysis rather than decoration, and it is worth being blunt about. An exploration that ends without written claims and open questions was decoration. Every pass should end with a sentence somebody could disagree with: not "here is a chart of scores by region", but "southern-region scores are about eight points higher, and the gap is concentrated entirely in one subject". The one-page summary is the deliverable — the shape of the data, three to five findings each with its evidence, the data-quality problems you found, and the questions you could not settle. If you cannot write that page, you have not finished, whatever the notebook looks like.

Two habits keep it honest. Write the finding immediately, while you still remember what you did to see it. And record the questions you could not answer, because those are what the next piece of work is for, and they are the first thing to be forgotten.

What you should now be able to explain or do

Run the five passes in order and say why each depends on the one before. Look at distributions rather than averages, and name three things an average hides. Treat an unusually strong feature as a suspect first. Split by segment and by cohort, and say what each separates that the other does not. Write a one-page summary with findings, data-quality problems and open questions. Recognise the difference between a chart and a stated finding.

Check yourself