1.21 Hypothesis testing and p-values
You can run a test correctly and explain what a p-value is not.
Before:00. Orientation & SetupUnlocks:03. Data Handling & Analysis04. Classical AI — Agents, Search & Knowledge Representation
Hypothesis tests are the standard machinery for asking whether an effect is real or noise, and this topic covers the common tests along with the concepts that outrank them: error types, power, effect size. It sits mid-sequence because experiment reading depends on it. The field's most common mistake gets named directly — the p-value is not the probability the hypothesis is true, and treating it that way, or peeking and testing repeatedly, manufactures discoveries out of randomness.
Work through these
Null and alternative hypotheses
Stating what you are testing and the alternative it is being weighed against, before looking at any results. Deciding these afterwards is what makes a test meaningless.
t-test, chi-square, ANOVA — when each applies
Three standard tests and the situations each is designed for. Choosing the wrong test is a more common error than computing the right one incorrectly.
Type I/II error, power, effect size
The two ways a test can be wrong, the chance of detecting a real effect, and the size of the effect itself. Effect size is what actually matters and is what usually goes unreported.
p-hacking, multiple comparisons, corrections
Running many tests until one is significant, and the corrections that exist because people do. This item is about recognising the practice in other people's work as well as avoiding it in your own.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.