4.23 How a network learns: the five learning rules

What this is and why it exists

Learning has a precise meaning here, and it is worth holding to it: the free parameters of a network are adapted through stimulation by the environment the network sits in. What kind of learning it is, is settled by the manner in which those parameters change. Five rules cover the ground, and they are genuinely different ideas rather than five arrangements of one. Two of them repay working by hand. The delta rule, because every supervised method afterwards descends from it. And Hebb's rule, because it learns with no teacher at all, and because its obvious failure — weights that can only grow — is fixed by a device you will see again in several other places.

The vocabulary

• Free parameters — the weights and biases that adaptation changes.
• Learning rule — the recipe deciding how they change.
• Error signal — desired output minus actual output.
• Delta rule — weight change proportional to error times input.
• Learning rate — the size of the step taken on each update.
• Memory-based learning — store all examples, answer from the nearest ones.
• Nearest neighbour rule — answer with the single closest stored example.
• Activity product rule — weight change proportional to the two activities.
• Forgetting factor — a term shrinking the weight to bound its growth.
• Competitive learning — units compete and only the winner adapts.

The mental model

Fix the definition first. The weights change under stimulation from outside, and the manner of that change names the type of learning. Holding to this keeps the five rules comparable, which they otherwise are not — they come from different traditions and are usually met one at a time in different courses.

Error-correction learning is the one everything descends from. The network produces an output, the desired output is known, and the difference between them is the error signal. Each weight moves by the learning rate times that error times the input arriving on that weight. Repeat, and the corrections shrink until they stop. It is also called the delta rule, or the Widrow-Hoff rule after the two people who wrote it down. Two things are worth noticing. The rule is local — a weight's update needs only the error at its own unit and the signal on its own input — and the learning rate is the whole of the tuning: too small and it crawls, too large and the corrections overshoot and the weights oscillate rather than settle.

Memory-based learning does no work at all until asked. Every example is stored with its answer. When a new input arrives, find the stored examples nearest to it and answer from them. It needs two things and neither is optional: a distance, so that nearest means something, and a rule for what to do with the neighbourhood once you have it. The nearest neighbour rule takes the single closest example's answer. The k-nearest neighbour version takes several and lets them vote, which is more robust when the stored examples contain mistakes. It is the plainest possible learning rule and it is genuinely competitive on small problems, which is a useful thing to remember before reaching for something elaborate.

Hebb's rule learns without a teacher, and it is stated in two halves. If two units on either side of a connection are active at the same time, the connection strengthens. If they are active out of step, it weakens. The weight change is the product of the two activities and a learning rate, which is why it is called the activity product rule. Nothing external is needed — no desired output, no error signal — and that is the point of it. The learning is local in the strongest sense: the connection changes on the basis of what happens at its own two ends and nothing else.

Left alone, that rule grows without limit, and the fix is a forgetting term. Two active units strengthen their connection, which makes them more likely to be active together, which strengthens it further. Nothing pushes back. Subtract a small multiple of the current weight on every update and the growth becomes bounded — the larger the weight, the larger the subtraction, until the two balance. The multiple is small, typically a hundredth to a tenth, so that the network forgets slowly rather than losing what it has learned.

Work it through once on the smallest example you can build. Initialise the weights to small random values. Compute the output. Update every weight by the product rule. Increase the count and go round again. Doing the arithmetic by hand for three or four rounds is what turns the rule from a formula into something you can reason about, and it is where the effect of the learning rate and the forgetting factor become visible rather than described.

The remaining two rules are different in kind and worth naming properly. In competitive learning the output units compete for each input and only the winner adapts, moving its weights towards that input. Repeated, this divides the input space among the units so that each specialises in one region — a division of labour arrived at with no supervision, and the basis of the self-organising methods. In Boltzmann learning the units are stochastic, taking one of two states with a probability set by the others, and the weights change according to a rule taken from statistical physics that compares the network running with its inputs held fixed against the network running free. It is the most demanding of the five computationally and the one you are least likely to implement, but it is where the idea of a network with an energy to minimise comes from.

What you should now be able to explain or do

Define learning as adaptation of the free parameters, and use that definition to compare the five rules. Apply the delta rule and explain why it is local and what the learning rate controls. Apply the nearest neighbour and k-nearest neighbour rules and name the two things memory-based learning requires. State Hebb's law in both halves and write the activity product rule. Explain why the rule grows without bound and how the forgetting factor prevents it. Work a small Hebbian example by hand for several rounds. Say what makes competitive and Boltzmann learning different in kind from the other three.

Check yourself