4.20 Rational decisions: utility and decision networks

What this is and why it exists

The first topic in this module defined a rational agent as one that acts to maximise its performance measure, and then quietly left the performance measure as something the designer writes down. This is where it is made precise. It turns out that any preferences obeying a short list of reasonable conditions can be written as a single number per outcome, and that the agent maximising the expected value of that number is exactly the rational agent the whole approach has been pointing towards. The result also explains a great deal of ordinary behaviour that looks irrational when it is measured in money and stops looking irrational when it is measured properly.

The vocabulary

• Lottery — a set of outcomes with probabilities, meaning any uncertain choice.
• Preference — that one outcome or lottery is wanted over another.
• Utility — a number attached to an outcome, ordered as preference is.
• Expected utility — each outcome's utility weighted by its probability.
• Risk averse — preferring a sure thing to a gamble of equal money value.
• Certainty equivalent — the sure amount as good as a given gamble.
• Decision network — a Bayesian network plus decision and utility nodes.
• Value of information — the gain in expected utility from learning something first.

The mental model

Utility is not assumed, it is constructed. A handful of conditions on preferences do the work. Any two options can be compared. Preference does not go round in a circle. If you prefer one option to a second and the second to a third, there is a mixture of the first and third you value exactly as much as the second. Mixing in an irrelevant option the same way on both sides changes nothing. Every one of these is hard to argue against, and together they force the existence of a single number per outcome whose expected value ranks the choices exactly as your preferences do. This is why maximising one quantity is not an arbitrary modelling convenience — it is what having coherent preferences amounts to.

The rule itself is one line. Weight the utility of each possible outcome by how likely it is, add them up, and choose the action with the highest total. That is the formal content of acting rationally under uncertainty, and every decision method later is a way of computing it when the direct calculation is out of reach.

Utility and money come apart, and that is the useful part. The second thousand rupees genuinely matters less to you than the first, so utility as a function of money bends: it rises, and it rises more slowly as the amount grows. A person with that shape prefers a certain five hundred to an even chance of a thousand or nothing, even though the two are worth the same in money. That is risk aversion, and it is not a defect. It is the correct response to a curve of that shape, and it explains insurance — where both sides gain, because the same loss sits at a much steeper part of one party's curve than the other's. The opposite curve explains why a lottery ticket can be a reasonable purchase for someone to whom a very large sum would be transformative and a small stake would not be missed. When behaviour looks irrational, check what it is being measured in first.

A decision network adds two kinds of node to a Bayesian network. The network you already have says what is likely and how the variables depend on one another. Add decision nodes for the things you may choose, and a utility node for what you care about, with edges from the variables that affect it. The result answers a different question from the one the original network answered: not what is probably true, but what to do. Evaluating it is mechanical — for each possible decision, set that node, work out the probability of each outcome, compute the expected utility, and keep the best.

The last piece prices a measurement. Before deciding, you can often learn something first — run a test, take a reading, ask an expert — and the question is whether it is worth it. The value of information answers it: compute the expected utility of deciding now, then compute the expected utility of deciding after learning the result, averaged over what the result might be, and take the difference. It is never negative and it is often zero, and the zero case is the one worth understanding. Information that would not change your decision is worth nothing, however interesting it is. That single observation stops a great deal of pointless measurement, and it is the calculation that says when to stop gathering evidence and act.

What you should now be able to explain or do

State the preference conditions and explain what their satisfaction guarantees. Compute expected utility for a set of actions and choose between them. Explain risk aversion using the shape of the utility curve, and use it to account for insurance and for lotteries. Extend a Bayesian network with decision and utility nodes, and evaluate it decision by decision. Compute the value of information for a proposed test, and explain what it means when the answer is zero.

Check yourself