S4-5.3 Stochastic Processes: Temporal & Spectral

What this is and why it exists

A random variable is one number drawn once. A stochastic process is randomness that unrolls in time — the model for every real signal that matters, because a signal known in advance carries no information and noise never repeats itself. This unit builds the working assumptions (stationarity, ergodicity) that make such signals analysable, and its central theorem — the Wiener-Khinchin relationship — reveals correlation in time and power across frequency as two views of one object.

The vocabulary

• Random process — an ensemble of possible waveforms with probabilities; one experiment selects one member function.
• Wide-sense stationary (WSS) — constant mean, and autocorrelation depending only on the time gap τ, not on when.
• Strict-sense stationary — every statistic, of every order, immune to time shifts; stronger and rarely verifiable.
• Ergodicity — time averages along one member function equal ensemble averages across members; one long recording suffices.
• Autocorrelation function — the expected product of the process with itself τ apart; its value at zero is the average power.
• Cross-correlation and covariance — the same comparison between two processes, and with means removed.
• Gaussian random process — every finite set of samples jointly Gaussian; WSS then implies strict stationarity.
• Power spectral density (PSD) — how the process's power is distributed over frequency; non-negative, and even for real processes.
• Wiener-Khinchin relationship — the PSD and the autocorrelation are a Fourier transform pair.

The mental model

Picture a wall of oscilloscopes, each showing one possible noise waveform from identical circuits — that wall is the ensemble, and the process is the whole wall plus its probabilities. Ensemble statistics read down the wall at one instant; time statistics read along one screen. Stationarity says the wall looks statistically identical whenever you sample it: no drift in the rules. WSS asks this only of the first two moments — mean and autocorrelation — which is the practical contract, since those are what measurements deliver. Ergodicity is the working engineer's licence: one screen, watched long enough, reveals the whole wall's statistics. It cannot be tested from one recording — it is an assumption, usually reasonable for noise, false for any process with a randomly chosen constant (each screen then shows a different constant forever).

Autocorrelation is the process interviewing itself: how alike are two samples τ apart? At τ of zero the answer is perfect likeness — numerically the mean square, the average power. How fast likeness decays as τ grows measures the process's memory: slow decay means slow wandering, low frequencies; a sharp collapse means fast fluctuation, high frequencies. Said aloud, that is a frequency statement extracted from a time function — and Wiener-Khinchin makes it exact: Fourier-transform the autocorrelation and the result is the power spectral density, the honest answer to "how much power lives at which frequencies". A narrow autocorrelation gives a wide PSD and the reverse — the same reciprocal-width law Fourier pairs always obey. Integrate the PSD over all frequency and the average power returns: the books balance. Cross-correlation extends the interview to two processes — alignment, shared content, relative delay (the principle radar ranging runs on) — with its own cross-power spectrum on the frequency side. The Gaussian process ties the strands: specify its mean and autocorrelation, and because all sample sets are jointly Gaussian, everything is specified — the second-order theory this unit builds is, for Gaussian processes, the entire theory.

What you should now be able to explain or do

Test a given process for WSS from its mean and autocorrelation. Explain ergodicity as time-versus-ensemble and produce a process that fails it. Use autocorrelation properties: power at τ zero, evenness, maximum at the origin. Move between autocorrelation and PSD via Wiener-Khinchin and verify total power both ways.

Check yourself