6.4 Loss functions
You choose a loss because of what it penalizes, not habit.
Before:05. Classical Machine LearningUnlocks:07. Natural Language Processing11. The ECE Bridge — Signals, Edge & Embedded AI
The loss is the objective everything optimises, chosen for what it penalises: mean squared error punishes large mistakes hardest, Huber tolerates outliers, focal loss attends to hard examples, label smoothing softens overconfidence. It sits in the training-craft sequence because the loss shapes every gradient after it. The numerical lesson has teeth — computing softmax and log separately overflows, and the fused, stable forms exist because of real crashes.
Work through these
MSE, MAE, Huber
Three regression losses differing in how harshly they punish large errors, with the third a compromise between the first two. The choice should follow from whether large errors are outliers or emergencies.
Cross-entropy, focal loss, label smoothing
The standard classification loss, a variant that concentrates on hard examples, and a technique that stops the model becoming overconfident. Each addresses a specific observed problem.
Contrastive and triplet losses
Losses that pull similar things together and push dissimilar things apart, which is how representations get learned without labels. They underpin much of the self-supervised work later in this module.
Custom losses and numerical stability
Writing your own loss is straightforward and the numerical pitfalls are not, particularly around logarithms of small numbers. Knowing the stable formulations prevents silent failures.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.