7.6 The Transformer architecture in full
You can draw the whole architecture and explain every block.
Before:06. Deep LearningUnlocks:08. Large Language Models
The transformer is the architecture of the current era, and this topic learns every block: attention with positional encodings, feed-forward layers, residuals and norms — plus the quadratic cost of attention, which explains the context-length economics of the entire LLM world. It sits at the centre of the module as the load-bearing topic. Positional encodings deserve real attention on the way through: without them the model cannot tell one word order from another.
Work through these
Encoder–decoder structure
The original architecture has one stack that reads and one that writes, and modern variants keep one or the other. Knowing the full form makes every variant readable.
Positional encodings: sinusoidal, learned, RoPE, ALiBi
Attention has no inherent sense of order, so position has to be injected, and there are several schemes for doing it. The newer ones are what allow context lengths to be extended.
Feed-forward blocks, residuals, layer norm placement
The parts of a block other than attention: the per-position network, the shortcut connections, and where normalisation sits. Where normalisation is placed genuinely changes training stability.
Complexity, context length and the quadratic cost
Attention compares every position with every other, so cost grows with the square of the sequence length. This single fact drives most research on long context.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.