PE1-6.5 AI Applications in Signals, Images & Speech

What this is and why it exists

This topic is why the course sits in an electronics department rather than a computing one. The inputs are signals, and preparing them is signal processing under another name.

Hand-designed features and learned representations are two answers to the same question. Which wins depends mostly on how much data you have.

The vocabulary

• Pre-processing — preparing raw data before any learning: filtering, resampling, normalising.
• Feature — a derived quantity summarising something useful about the input.
• Feature extraction — computing those quantities from the raw signal.
• Feature mapping — turning raw input into a representation a model can use.
• Learned representation — features the model discovers itself rather than being given.
• Source-filter model — describing speech as a sound source shaped by the vocal tract.
• Formant — a resonance of the vocal tract, visible as a peak in the spectrum.
• Phoneme — the smallest unit of speech sound that distinguishes words.

The mental model

Good features beat a better model on this kind of data, and that is where an engineer with domain knowledge adds the most value.

Pre-processing comes first and it is ordinary signal processing. Remove the drift, filter out what is not wanted, resample to a consistent rate, and normalise so that recordings made at different levels are comparable. None of it is glamorous and all of it decides whether anything downstream works. A model trained on data with an artefact in it learns the artefact.

Feature extraction then summarises the signal into quantities that carry the information and discard the rest. From a time series that might be statistics of the values, or energy in frequency bands. It might be rates of change, or how the signal correlates with itself. The choice encodes what you know about the domain. That knowledge is why an engineer is better placed to do it than a general programmer.

Images need their own preparation, and only a subset of image processing matters here. Resizing so every input is the same shape, and adjusting contrast. Also altering the training images in harmless ways, so the model is less sensitive to position and lighting.

Feature mapping generalises the same idea. Raw input is turned into a representation a model can use, and the question is who chooses that representation.

Classical practice is that you do, using domain knowledge, producing a small number of meaningful quantities. The modern alternative is that the model discovers its own representation from raw input during training, given enough examples.

Both are answers to one question, and the trade is real. Hand-designed features need far less data, they are interpretable, and they cap performance at how good your understanding is. Learned representations need a great deal of data and computation, and they are hard to interpret. Given enough of both, they surpass what hand design achieves.

So the honest guidance is about circumstances rather than fashion. With a few hundred examples, engineered features. With a hundred thousand, learned ones. In between, and in most real projects, some of each.

The image classification exercise is worth doing end to end, and the point is the pipeline rather than the accuracy achieved. Split the data before doing anything else. Prepare inputs identically for training and testing. Watch training and validation performance separate as overfitting begins, and evaluate once on data the model has never seen. That sequence is the transferable part.

Speech closes the course, and one model of production underlies every technique in it. Sound is produced by a source: the vibration of the vocal folds, or turbulent noise. That source is then shaped by the vocal tract, a filter whose resonances change as the tract moves.

Separating the two is the idea to carry. The source carries pitch and voicing. The filter carries which sound is being made, and its resonances are what distinguish one vowel from another. Almost every speech feature is an attempt to describe the filter while discarding the source.

Classifying speech sounds follows directly. Sounds differ by whether the vocal folds vibrate, by where in the tract the constriction is, and by how the air is released. Those distinctions have acoustic consequences, and knowing them is what makes a recogniser's features meaningful rather than arbitrary.

Recognition using the time-structured model from the knowledge topic closes the loop. The hidden state is what is being said, which you cannot observe. The observations are the acoustic features extracted every few milliseconds. The model's assumptions fit. The sound being produced now depends on the one immediately before it, and what you measure depends on what is being produced now.

That is one model, from an earlier topic in this course, doing real work end to end on a real problem. It is a good place to finish.

What you should now be able to explain or do

List the pre-processing steps and say why a learned artefact is a failure of preparation. Extract features from a time series and say what domain knowledge contributes. State the trade between engineered and learned representations, and choose by the amount of data. Describe an image classification pipeline as a sequence rather than a result. Explain the source and filter split and why the time-structured model fits speech.

Check yourself