6.8 PyTorch: tensors, autograd, modules
You can build any architecture you can draw.
Before:05. Classical Machine LearningUnlocks:07. Natural Language Processing11. The ECE Bridge — Signals, Edge & Embedded AI
PyTorch is the lingua franca of research and much of industry: tensors on devices, autograd recording operations, modules organising parameters. Fluency here means any architecture you can sketch, you can build. It sits at the centre of the module because everything after is written in it. The leaks to guard against are device and graph — CPU and GPU tensors mixed mid-computation, and histories accumulated by keeping tensors that still require gradients.
Work through these
Tensors, devices, dtypes, and .to()
The array type, where it lives, what it holds, and the method that moves it between processor and accelerator. Most beginner errors are a mismatch in one of these three properties.
autograd, requires_grad, detach, no_grad
The automatic differentiation system, the flag that marks something to be differentiated, and the two ways of stepping outside the recording. Knowing when gradients are being recorded prevents both wasted memory and missing gradients.
nn.Module, parameters, buffers
The base class every model component inherits from, and the distinction between things that are learned and things that are merely stored. That distinction decides what gets saved and optimised.
Saving and loading state dicts
Saving a model means saving its parameter dictionary rather than the object itself, which is what makes a checkpoint portable. Getting this right is the difference between a reloadable model and a broken one.
Sign in to keep your progress.
Free resources
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.