OE-11.1 Introduction to Artificial Neural Networks

What this is and why it exists

The first model in this subject dates from 1943 and is deliberately crude. Fixed weights, a threshold, no learning at all.

That crudeness is exactly why it is the right place to start. It shows the shape — inputs, weights, a sum, a threshold, an output — with nothing else in the way.

One caution belongs at the top, and it is worth stating plainly. The biological analogy is a historical motivation, not a statement of equivalence. Modern networks resemble brains about as much as an aeroplane resembles a bird. Something real was borrowed, and the two are not the same kind of thing.

The vocabulary

• Neuron — the unit: several inputs, one output.
• Weight — how much one input counts.
• Weighted sum — inputs multiplied by weights and added.
• Threshold — the level the sum must reach to produce an output.
• Feedforward — signals travel one way, input to output.
• Recurrent — some connections lead backwards, giving the network memory.
• Layer — a group of neurons at the same depth.
• Excitatory and inhibitory — an input that pushes towards firing, or against it.

The mental model

A biological neuron receives signals from many others, and produces an output when enough of them arrive together. The artificial version keeps that shape and discards everything else. Inputs, each multiplied by a weight, added together, compared to a threshold, and an output produced.

The 1943 model fixes the weights by hand and has no way to change them. So it can be *designed* to compute a logical function and it cannot learn one. That absence is instructive, because it makes obvious what every later development supplies — a rule for changing weights from experience.

The connection patterns are worth learning as a small set. Signals travelling one way, input to output, gives a feedforward network, which computes a function of its input and nothing more. Allow connections backwards and you get a recurrent network, whose output depends on what came before — it has state, and therefore memory.

That single distinction decides what a network can do at all, so it is worth more attention than its length in a syllabus suggests.

On the biological analogy, be careful in both directions. It is not nothing: the idea of many weak weighted influences summing to a decision genuinely came from neuroscience. But a network's units are not neurons in any strong sense, and its learning is not how brains learn. Explaining a network's behaviour by appeal to brains explains nothing. Use the analogy for intuition and drop it before making an argument.

The characteristics that follow are what make these worth using. Learning from examples, tolerating noisy input, and degrading gradually rather than failing outright. They are also why they are hard to inspect: the knowledge is spread across the weights rather than written anywhere.

What you should now be able to explain or do

Describe the artificial neuron as inputs, weights, a sum and a threshold. Say what the 1943 model can and cannot do, and what its absence of learning reveals. Distinguish feedforward from recurrent connection and say what the difference makes possible. State what the biological analogy legitimately supplies and where it stops. Name the characteristics that make these networks useful, and the inspection difficulty that comes with them.

Check yourself