EC-8.6 Designing an Experiment and Reporting It

What this is and why it exists

An accurate reading of the wrong quantity is still wrong.

This is the topic that makes the rest of the module worth something. The instruments and the error analysis tell you how well you measured. Nothing so far tells you whether the measurement answers a question.

It is also the topic that transfers furthest. Three habits carry over: state the question first, change one thing at a time, and report the uncertainty. They apply to a circuit, a piece of software and a manufacturing line alike.

The vocabulary

• Hypothesis — the outcome you expect, written down before measuring.
• Control — a measurement taken with nothing changed, to show how much the setup moves on its own.
• Independent variable — the quantity you deliberately change.
• Dependent variable — the quantity you measure in response.
• Residual — the difference between a measured point and the value a fitted model predicts.
• Least squares fit — a line or curve chosen to make the sum of squared residuals as small as possible.
• Reproducibility — whether another person following your report gets your result.

The mental model

Write down the question, and what result would change your mind, before touching an instrument. It takes ten minutes. It is the single thing that most often separates a productive day at a bench from a day of readings that answer nothing.

The second half of that sentence matters as much as the first. A prediction that no possible result would contradict is not a prediction. If you cannot say what outcome would surprise you, the experiment is not yet designed.

Controls come next. If two things change between two readings, the difference cannot be attributed to either of them. That is obvious when stated and violated constantly. Usually something changed that you did not choose to change: the temperature, the supply voltage, the warm-up state of an instrument.

A control measurement, with nothing deliberately changed, tells you how much your setup moves on its own. Any effect smaller than that movement has not been observed. Taking the control first, and again at the end, also catches drift during the session.

Choosing points is a small skill with a large payoff. Points clustered where nothing happens waste effort. A range too narrow hides the very behaviour you were looking for. A coarse wide sweep followed by a fine one over the interesting region almost always beats a single even sweep.

How many depends on what you are doing. If you are fitting two parameters, you need many more than two points. The extra points are what let you estimate the uncertainty of the fit.

Plotting is a thinking tool and not a presentation step. Choose axes that make the expected relation a straight line. A departure from a straight line is visible, and a departure from a curve is not. Logarithmic axes turn a power relation into a line and an exponential one into a line on a semi-logarithmic plot.

Error bars belong on the plot. Without them a reader cannot tell whether two points differ. Plotting the residual rather than the raw value is the sharpest version of the same idea. The residual plot is where a wrong model announces itself.

Fitting a line is where confidence outruns evidence. A least-squares fit returns a slope and an intercept whatever the data looks like, including data that is not a line at all. A high correlation is not evidence that the model is right.

The residuals are the test. If they scatter randomly about zero, the model is doing its job. If they curve, the model is wrong and the fitted parameters are meaningless however good the correlation looked.

The report is the last step and it has a test attached. Could a competent stranger repeat this work from your report and get your answer? If not, something is missing, and it is usually a setting, a part number or a condition that seemed too obvious to write down.

The structure that satisfies that test is short: purpose, setup with parts and settings, method, data, uncertainty, result, and what you concluded. Anything that would change the result belongs in the setup section, including the things you did not choose, such as the ambient temperature.

What you should now be able to explain or do

• Turn a question into a measurement plan with a stated prediction and a stated way to be wrong.
• Use a control measurement to establish how much a setup moves on its own.
• Choose the number and spacing of points for the behaviour you are looking for.
• Plot data on axes that make a departure from the expected relation visible.
• Fit a line and use the residuals to test whether the model is right.
• Write a report that another person could follow to your result.

Check yourself