11.3 How speech is produced and classified

What this is and why it exists

Speech is a source driving a filter: something makes a sound, the vocal tract shapes it, and the shape changes several times a second. That one model explains why vowels are identified by resonances, why voiced and unvoiced sounds look completely different on a spectrogram, and why speech features are built exactly as the previous topic built them. Skip it and the coefficient features stay a recipe you follow; understand it and they become a representation you can reason about.

The vocabulary

• Source — what generates the sound: vocal fold vibration, or turbulent airflow.
• Filter — the vocal tract, shaping the source into a particular sound.
• Fundamental frequency — the rate of vocal fold vibration, heard as pitch.
• Harmonics — multiples of that rate, present in a voiced source.
• Formant — a resonance of the vocal tract, appearing as a peak in the spectrum.
• Voiced — produced with the vocal folds vibrating; unvoiced — without.
• Plosive — a sound made by stopping the airflow and releasing it.
• Fricative — a sound made by forcing air through a narrow constriction.

The mental model

Follow the chain once and most of what a speech signal contains is explained.

The lungs supply a steady flow of air — the energy, and nothing about the identity of the sound.

The larynx is where the source is made. The vocal folds may vibrate, opening and closing many times a second, chopping the airflow into a periodic train of pulses; that rate is the fundamental frequency, heard as pitch, and its pulse train contains that frequency and a long series of harmonics above it. Or the folds may stay apart, and the source is turbulent noise made at a constriction further along — broadband, with no periodicity at all.

The vocal tract — throat, mouth, tongue, lips, and sometimes the nasal passage — is a tube whose shape you change continuously. Like any tube it has resonances, and it amplifies frequencies near them and suppresses the rest. Those resonances are the formants, and their positions are decided by the shape.

The lips radiate the result, which adds a mild emphasis to higher frequencies — the reason speech feature pipelines conventionally apply a pre-emphasis step, compensating for it before analysis.

The source-filter model is the abstraction to take away, and it is the single most useful one in the topic: the source decides the pitch and whether the sound is periodic; the filter decides which sound it is. Two people saying the same vowel at different pitches have different sources and very similar filters. One person singing a vowel up a scale is changing the source and holding the filter still. Identity of the sound lives in the filter; melody and voice quality live in the source.

Two consequences follow immediately and are worth stating.

It explains the coefficient features exactly. In the frequency domain the source and the filter multiply. Taking a logarithm turns that product into a sum, and the two parts vary at very different rates along the frequency axis: the filter's formant structure is a slow undulation, the source's harmonics are a fast ripple. A further transform separates slow from fast, and keeping the slow part keeps the vocal tract and discards the pitch. That is the whole derivation, and it is why those features identify what was said and are poor at identifying who said it.

And it explains speech synthesis and coding. Model the source and the filter separately, transmit or generate each, and recombine — which is what low-rate speech coding has done for decades, and it is why a coded voice can sound thin while remaining perfectly intelligible: the filter survived and the source was approximated.

Now the sound classes, which are the classes an acoustic model has to separate.

Voiced sounds — all vowels, nasals, and consonants such as the ones in "buy", "day", "go", "zoo", "van" — have a vibrating source. Unvoiced sounds — the ones in "pie", "tie", "key", "sue", "fan" — have a turbulent source and no periodicity. Notice the pairs: several consonants differ only in voicing, with the same tract shape, which is why that distinction has to be represented and why it is a common confusion in a weak acoustic model.

Vowels are voiced, with the tract open and a stable shape held for a comparatively long time. Their identity is carried almost entirely by the first two formants — where the lowest two resonances sit — which is decided by tongue height and how far forward the tongue is. This is the clearest link between anatomy and acoustics in the topic.

Plosives stop the airflow completely and release it: a silence, then a burst. They are brief and their information is in the transition.

Fricatives force air through a narrow gap, producing sustained turbulent noise with energy concentrated high in the spectrum, and they can last as long as breath allows.

Reading these off a spectrogram is what makes speech data interpretable rather than opaque, and each has a signature you can learn in an afternoon.

Voiced regions show horizontal striations — evenly spaced harmonics — and their spacing is the pitch, so the stripes converge as pitch rises. They also show clear dark bands where the formants are.

Vowels are the steady stretches with two or three obvious formant bands; watching them move as the mouth changes shape is the whole of the source-filter model made visible.

Plosives are a near-vertical gap with a sharp vertical spike after it — silence, then a burst that spans many frequencies at once.

Fricatives are a cloud of energy concentrated at the top of the display, with no harmonic structure, sustained rather than brief.

Silence between words is not silent: it is low-level noise, and the boundary between silence and a quiet unvoiced sound is genuinely difficult, which is why endpoint detection is a real problem rather than a threshold.

Two practical consequences to carry into the next topic. The tract shape changes several times a second, so analysis windows have to be short enough to catch one configuration — a few tens of milliseconds, which is exactly the window length speech pipelines conventionally use, and now you know why rather than merely that. And because voiced and unvoiced sounds differ so completely in structure, a spectrogram is genuinely readable: when a recogniser fails, looking at the spectrogram frequently shows the reason — the sound was clipped, the fricative was lost to noise, the recording is band-limited by a codec that removed the high frequencies fricatives live in.

What you should now be able to explain or do

Trace the production chain and say what each stage contributes. State the source-filter model and use it to separate what carries identity from what carries pitch. Derive the coefficient features from it, and say why they suit what-was-said and not who-said-it. Classify sounds as voiced or unvoiced, vowel, plosive or fricative, and say what distinguishes each. Recognise each class by its spectrogram signature. Explain why speech analysis windows are a few tens of milliseconds. Diagnose a recognition failure by looking at the spectrogram.

Check yourself