S4-5.1 Probability & Random Variables

What this is and why it exists

This is the load-bearing unit of the course: every result after it assumes fluency here, and the whole of communication engineering is applied probability — noise is random, signals carry information only because receivers cannot foresee them. The distribution list has an EE accent on purpose: Gaussian because noise, Rayleigh because fading envelopes, Poisson because arrivals. If you have met probability in the AI/ML material, this is the same core wearing communication-engineering examples — the overlap is deliberate.

The vocabulary

• Conditional probability — the probability of A given that B occurred; probability updated by knowledge.
• Total probability — the probability of an event assembled from its probability under each cause, weighted by the causes' probabilities.
• Bayes' theorem — the reversal engine: from the probability of the observation given the cause to the probability of the cause given the observation.
• Independence — B's occurrence changes nothing about A; joint probability factors into the product.
• Random variable — a number assigned to each outcome of an experiment; the object distributions describe.
• Distribution and density — the cumulative function and its derivative; area under the density is probability.
• Gaussian, uniform, exponential, Rayleigh — the continuous regulars: noise, ignorance-within-limits, waiting times, fading envelope.
• Binomial, Poisson — the discrete regulars: successes in n trials, and counts of rare events at a known average rate.

The mental model

Bayes' theorem is the receiver's theorem. Every digital receiver lives inside one question: given what was RECEIVED, what was SENT? Physics hands you the forward direction — given that a zero was sent, here is the density of what arrives (signal plus Gaussian noise). Bayes reverses the arrow, weighting by how often each symbol is sent, and the receiver decides for the cause with the larger posterior. Detection theory, decoding, even spam filtering are that one reversal at scale — learn it as the direction-swapper and the formula stops needing memorisation.

The distributions earn their places physically. Gaussian: thermal noise is the sum of countless tiny independent electron agitations, and sums of many small independent effects go Gaussian (the next unit proves it) — so noise voltage is Gaussian, not by convention but by mechanism. Rayleigh: take TWO independent Gaussians — the in-phase and quadrature parts of a multipath radio signal — and the length of that random vector is Rayleigh; deep fades on a moving receiver follow it. Exponential: waiting time to the next event when events have no memory. Poisson: how many such events land in a window — photons on a detector, calls at an exchange, packets at a router. Binomial: n independent shots, each with the same success probability — bit errors in a block.

Joint and conditional densities extend the machinery to pairs: the joint density covers the plane, marginals are its shadows on the axes, and conditioning is slicing the joint at what you learned and renormalising the slice into a proper density. Receivers condition on the received value; that slice is where detection happens.

What you should now be able to explain or do

Run a Bayes computation for a binary channel: priors, likelihoods, posterior, decision. Name the mechanism behind each regular distribution rather than only its formula. Move between joint, marginal and conditional densities. State independence three equivalent ways.

Check yourself