4.24 Learning paradigms, and what a network is asked to do

What this is and why it exists

The three learning paradigms differ in exactly one thing, and it is not the network. It is what comes back from the environment. A teacher supplies the answer that was wanted. A critic supplies a single number saying how well things went, and nothing more. Unsupervised learning gets nothing at all and has to find whatever structure is already in front of it. Every architecture in the previous topics can be trained under any of the three, which is why the distinction is drawn here on feedback rather than on structure. The learning tasks that follow are the shapes an applied problem takes, and one of them is where an electronics background meets this material head on.

The vocabulary

• Teacher — a source of the desired response for each input.
• Critic — a source of a single number rating what happened.
• Error surface — mean square error drawn as a function of the weights.
• Operating point — where the current weights sit on that surface.
• Pattern association — storing and recalling paired patterns.
• Autoassociation — recalling a pattern from a corrupted version of itself.
• Heteroassociation — recalling a different pattern paired with the input.
• Pattern recognition — assigning an input to one of several classes.
• Function approximation — fitting an unknown mapping from examples.
• Beamforming — steering sensitivity towards one direction.

The mental model

Learning with a teacher is a closed loop. The teacher knows the environment as a set of input and output examples, and supplies the desired response. The error — desired minus actual — adjusts the weights, step by step, until the network emulates the teacher. Picture the mean square error as a surface over the weights, with the current weights as a point on it. Learning is that point moving downhill towards a minimum. This picture is worth carrying because it makes several later things obvious at a glance: why the step size matters, why a rough surface is harder than a smooth one, and why the process can stop somewhere that is not the lowest point. When the training is done the teacher can be dismissed and the network works alone, with what it learned held in fixed weights.

Reinforcement learning replaces the teacher with a critic. No desired output is supplied. The system acts, the environment responds, and a single number says how good that was. A critic turns that raw signal into something more useful before it reaches the learning machinery. It sits between the other two paradigms: more feedback than none, far less than a full answer, and the learning happens through continued interaction rather than from a fixed set of examples. The characteristic difficulty follows from that — the number arrives after the action, sometimes long after, and working out which action deserves the credit is the problem the whole area is built around.

Unsupervised learning has no teacher and no critic. There is no external signal at all, so the network can only find structure that is already present in the data. Clustering is the familiar example, and the distinction that matters is that this is descriptive rather than predictive: it tells you how the data is arranged, not what a new input should be labelled. The competitive rule from the previous topic is the natural fit here, because dividing the input space among units is precisely finding structure without being told what to find.

Set the three side by side and the pattern is clear. Supervised learning uses labelled examples, produces a predictive result, and covers classification and regression. Unsupervised learning uses unlabelled data, produces a descriptive result, and covers clustering. Reinforcement learning uses interaction with an environment and produces a policy. The contrast is about feedback throughout, not architecture — the same multilayer network appears in all three columns.

The learning tasks are the shapes an applied problem takes. Pattern association stores pairs and recalls them, which is a memory built out of a network. Pattern recognition assigns an input to one of a fixed set of classes, with the input represented as a point in a space and the training deciding where the boundaries between regions fall. Function approximation fits an unknown mapping from examples, and it has two important uses: identifying an unknown system from its inputs and outputs, and building the inverse of a known one. Most applied problems are one of these three wearing local clothes, and recognising which saves a great deal of time.

Three more tasks, and the last one is the interesting one here. Control drives a system towards a desired output, with the network learning what to do rather than being designed for it. Filtering cleans a signal, separating what you want from what you do not — noise removal, echo cancellation. Beamforming steers sensitivity towards one direction while suppressing interference arriving from others, when the target's direction is unknown and nothing is known about the interference. Radar and sonar do it, and so does the echolocating bat, which broadcasts a short frequency-modulated call and uses two ears to attend to one insect in a cluttered return. This is the point where the signals half of an electronics degree and this material stop being separate subjects.

Pattern association splits in two, and the difference is worth keeping. Autoassociation stores a set of patterns, then recovers a stored pattern from a noisy or partial version of itself. Heteroassociation pairs one set of patterns with a different set and recalls the partner. The first is a memory that tolerates corruption; the second is a mapping. Both are useful and confusing them makes the results of either look wrong.

What you should now be able to explain or do

Describe learning with a teacher as a closed loop and use the error surface to explain what learning is doing. Describe reinforcement learning, the role of the critic, and the difficulty that follows from delayed feedback. Describe unsupervised learning and say why its result is descriptive. Contrast the three paradigms on feedback, data and output. Name and describe pattern association, pattern recognition and function approximation. Describe control, filtering and beamforming, and say why beamforming is the meeting point with signal processing. Distinguish autoassociation from heteroassociation.

Check yourself