EC-14.4 Prototyping, Verification and the Design Review

What this is and why it exists

A prototype that tries to be a small version of the whole product answers nothing.

This topic argues for prototypes with a single question each, and then attaches the verification plan that turns requirements into evidence.

The design review is included because it is the professional practice most students never experience before they are expected to survive one.

The vocabulary

• Prototype — something built to answer a specific question.
• Verification — demonstrating that the design meets its requirements.
• Validation — establishing that the requirements were the right ones.
• Verification plan — a table mapping each requirement to a test and its evidence.
• Corner case — a combination of conditions at the extremes of the allowed ranges.
• Design review — a meeting where others examine a design to find problems.

The mental model

A prototype answers one question.

Can this sensor resolve what we need? Will this run for a day on this battery? Does this algorithm fit in the available memory? A prototype built to answer one question is quick to build, quick to interpret, and conclusive.

A prototype built to be a small version of the product answers nothing clearly. When it does not work you cannot say which of its ten unproven assumptions was wrong. It also takes five times as long.

The verification plan is a table with one row per requirement: the requirement, the test that demonstrates it, and the evidence that test produces.

Write it while the requirements are fresh. Doing so means the tests exist before the schedule makes them unwelcome. It also exposes a requirement that cannot be tested at all, which is better discovered now.

Verification and validation are different questions and the distinction matters more than it sounds.

Verification asks whether you built the thing right, against the requirements. Validation asks whether the requirements were the right ones. A design can pass verification completely and still be useless, because it does exactly what was asked and what was asked was wrong.

The way to validate is to show the thing, or a prototype of it, to the people who will use it. Do it early enough that the answer can change the design.

Testing belongs at the edges rather than in the middle. A design that works at nominal conditions may fail at the extremes of temperature, supply voltage, input range or load.

The interesting tests are at the boundaries and a little outside them. What happens at the maximum input? At the minimum supply? At both at once? Corner cases are where the assumptions that were never written down reveal themselves.

The design review exists to find problems while they are cheap. That is worth stating because most people experience reviews as approval events, and treat finding a problem as a failure of the author.

Reviewers who find nothing have usually not been given enough time or enough of the design. A review with no findings is a warning sign rather than a success.

As an author, present the decisions and the reasons rather than defending the outcome. The reviewers cannot help if they only see the conclusion. As a reviewer, ask about assumptions and failure cases rather than about style. Record what was raised, so that it can be answered rather than remembered.

Both roles are learnable and both are usually learned badly by accident. Doing one deliberately, once, is worth more than reading about them.

What you should now be able to explain or do

• Build a prototype that answers a single stated question.
• Write a verification plan mapping each requirement to a test and its evidence.
• Distinguish verification from validation and say how each is done.
• Design tests at the corners of the allowed conditions rather than at nominal.
• Take part in a design review as an author, presenting decisions rather than defending them.
• Review someone else's design usefully and record what was raised.

Check yourself