Probability and Random Processes for Communication Systems

Probability provides the mathematical language for modeling uncertainty in communication systems—e.g., noisy observations, fading channels, and random interference. We build from Probability to Random variable and then to Random process.\n\nIn communication, typical questions include: given a transmitted symbol, what is the probability of each received value? Conversely, given an observation, what is the probability of each transmitted hypothesis? These are answered using conditional probability and Bayes’ theorem, then analyzed using distributions (especially Gaussian), and—when time matters—via random processes and stationarity.

Key objects you will repeatedly use:\n- Event\n- Conditional probability\n- Random variable (RV)\n- Probability density function (PDF)\n- Random process

Probability foundation (events, conditional probability, Bayes rule)

An Event $A$ is a subset of the sample space $\Omega$. The Probability function $\\mathbb{P}(A)$ satisfies:\n- Nonnegativity: $\\mathbb{P}(A) \\ge 0$\n- Normalization: $\\mathbb{P}(\\Omega)=1$\n- Additivity: if $A$ and $B$ are disjoint, then $\\mathbb{P}(A\\cup B)=\\mathbb{P}(A)+\\mathbb{P}(B)$\n\nWhen communications include decisions (decode/no-decode, detect one of $M$ messages), conditioning is central. If $\\mathbb{P}(B)>0$, the Conditional probability is\n$\\mathbb{P}(A\\mid B)=\\frac{\\mathbb{P}(A\\cap B)}{\\mathbb{P}(B)}.$\n\nBayes' theorem connects "given what we saw" to "what we assumed transmitted":\n$\\mathbb{P}(A\\mid B)=\\frac{\\mathbb{P}(B\\mid A)\\,\\mathbb{P}(A)}{\\mathbb{P}(B)}.$\n\nIn ML-style decoding, interpret:\n- $A$: hypothesis about transmitted symbol/message\n- $B$: received observation (or event that observation falls in a region)\n\nMermaid overview:\nmermaid\ngraph LR\n X[Transmitted symbol/message] --> Y[Channel/noise produces observation]\n Y --> B[Likelihood: P(Y|X)]\n B --> A[Posterior: P(X|Y) via Bayes]\n

Random variables and distributions (discrete vs continuous)

A Random variable (RV) $X$ assigns a real number to each outcome. For discrete $X$, probabilities come from the Probability mass function (PMF) $p_X(x)=\\mathbb{P}(X=x)$. For continuous $X$, use the Probability density function (PDF) $f_X(x)$ with\n$\\mathbb{P}(a\\le X\\le b)=\\int_a^b f_X(x)\\,dx.$\n\nTwo core ways to describe uncertainty:\n- Cumulative distribution function (CDF): $F_X(x)=\\mathbb{P}(X\\le x)$\n- Moments (expectation/variance)\n\nMean and variance:\n$\\mathbb{E}[X]=\\int x f_X(x)\\,dx \\quad \\text{or}\\quad \\sum_x x\\,p_X(x),$\n$\\mathrm{Var}(X)=\\mathbb{E}[(X-\\mathbb{E}[X])^2].$\n\n---

Random variables in communication: noise and sums

A foundational communication model is additive noise:\n$Y = X + N,$\nwhere\n- $X$ is the transmitted symbol (often modeled as random when choosing among messages)\n- $N$ is noise (commonly approximated as Gaussian in many receivers)\n- $Y$ is the received observation\n\nMany decision rules depend on the distribution of $N$ (and thus the distribution of $Y$ conditioned on $X$). When $N$ is Gaussian, the resulting $Y\\mid X$ distribution is also Gaussian, making likelihoods analytically tractable.

For Gaussian noise: $N\\sim \\mathcal{N}(0,\\sigma^2)$ implies\n$f_N(n)=\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp\\left(-\\frac{n^2}{2\\sigma^2}\\right).$\nThen $Y\\mid X=x$ is Gaussian with mean $x$ and variance $\\sigma^2$.\n\nGaussian-tail probabilities often appear through the Q-function:\nQ(x)=\\frac{1}{\\sqrt{2\\pi}}\\int_x^\\infty \\exp\\left(-\\frac{t^2}{2}\\right)dt.\nThese tails govern error probabilities in many modulation/detection settings.

Random processes: modeling time-varying channel behavior

A Random process $\\{X(t)\\}$ is a collection of RVs indexed by time $t$.\n\nTwo quantities describe many communication systems:\n- Mean function: $m_X(t)=\\mathbb{E}[X(t)]$\n- Correlation: $R_X(t_1,t_2)=\\mathbb{E}[X(t_1)X^*(t_2)]$ (use $^*$ for complex baseband processes)\n\nOften we assume wide-sense stationarity (WSS):\n- $m_X(t)$ is constant\n- $R_X(t_1,t_2)$ depends only on $\\tau=t_1-t_2$, written $R_X(\\tau)$\n\nFrom WSS, autocorrelation becomes a function of lag only:\n$R_X(\\tau)=\\mathbb{E}[X(t)X^*(t-\\tau)].$\n\nMermaid concept map:\nmermaid\ngraph TD\n Process[X(t)] --> Mean[Mean: E[X(t)]]\n Process[X(t)] --> Corr[Correlation: E[X(t1)X*(t2)]]\n Corr --> WSS[If WSS: Corr depends on lag τ]\n

Autocorrelation, spectral density, and intuition (frequency view)\nIn many systems, we want to connect time-domain randomness to frequency-domain behavior. Under standard regularity conditions, the power spectral density (PSD) of a WSS process is the Fourier transform of its autocorrelation:\n$S_X(f)=\\int_{-\\infty}^{\\infty} R_X(\\tau)\\,e^{-j2\\pi f\\tau}\\,d\\tau.$\nIntuitively, $R_X(\\tau)$ describes "how quickly the process changes," while $S_X(f)$ describes how that randomness is distributed across frequencies.

Random process classes used in channels and interference

Common models in communication include:\n- Independent and identically distributed (i.i.d.) noise (memoryless noise)\n- Markov process for channels with memory (e.g., bursty fading)\n- Ergodicity assumptions to estimate statistics from observations\n\nEven if you don’t fully formalize these immediately, recognizing “what kind of randomness” a model uses helps determine which tools apply (likelihoods for memoryless models vs state-space methods for Markov/memory models).

Quick comparison: probability vs random processes in communications