PE1-3.3 The General Gaussian Problem

What this is and why it exists

Assume everything is Gaussian and the algebra closes into a form you can read. That is why this assumption is made everywhere.

It is also often true. Thermal noise is the sum of very many small independent contributions, and the central limit theorem says such sums tend towards a Gaussian shape.

The vocabulary

• Covariance matrix — the table describing how the components of a random vector vary and how they vary together.
• Diagonal covariance — components that are uncorrelated with each other.
• Non-diagonal covariance — components that are correlated.
• Whitening — a transformation that turns correlated components into uncorrelated ones.
• Linear detector — one that forms a weighted sum of the observations and compares it against a threshold.
• Quadratic detector — one that must square the observations before comparing.
• Same-covariance case — the two hypotheses differ in their means.
• Same-mean case — the two hypotheses differ in their variances.

The mental model

The key question of this topic is what shape the detector takes, and the answer depends on how the two hypotheses differ.

Take the same-covariance case first. Both hypotheses have the same noise structure and differ in their mean. Form the likelihood ratio, take its logarithm, and the quadratic terms are identical under both hypotheses, so they cancel. What survives is linear in the observation. The detector becomes a weighted sum compared against a threshold, which is a dot product and nothing more.

With diagonal covariance, that dot product has a satisfying reading. The test reduces to comparing the distance from the observation to each of the two candidate means, and choosing the nearer one. That is the same nearest-point picture the digital modulation topic uses, arrived at from a different direction.

With non-diagonal covariance the components are correlated, and distance no longer means what you want it to mean. The fix is whitening. Apply a transformation that makes the transformed noise uncorrelated with equal variances, transform the signals the same way, and then apply the diagonal argument unchanged. Whitening is the idea to carry away from this topic, because it recurs throughout signal processing whenever correlated noise gets in the way.

Now the same-mean case. Here the two hypotheses have the same average and differ in variance. The mean carries no information at all, so a weighted sum of the observations tells you nothing. What differs is the spread, and measuring spread means squaring. The quadratic terms no longer cancel, and the detector is quadratic: form a weighted sum of squares and compare it against a threshold. With equal variances across components the weights are the same; with unequal variances the components are weighted differently, according to how informative each one is.

That is the summary the whole topic is heading for. If the hypotheses differ in mean, the detector is a weighted sum. If they differ in variance, the detector must square. Knowing which structure your problem has tells you the receiver's shape before you compute anything.

Work the diagonal case fully before touching the correlated one. The correlated case is the same argument after a whitening step, and treating it as a separate problem doubles the work for nothing.

What you should now be able to explain or do

Say why Gaussian assumptions are both convenient and often justified. Show that the same-covariance case gives a linear detector, and read it as a distance comparison. Explain what whitening does and why it reduces the correlated case to the diagonal one. Show that the same-mean case gives a quadratic detector. Predict the detector structure from how the hypotheses differ.

Check yourself