Probability and random variables
Introduced| English | Meaning |
|---|---|
| random variable/ˈrændəm ˈveərɪəbl/ | A numerical quantity whose value depends on the outcome of a random process. |
| probability distribution/ˌprɒbəˈbɪlɪti ˌdɪstrɪˈbjuːʃn/ | A description assigning probabilities to the possible values or ranges of a random variable. |
| variance/ˈveərɪəns/ | A measure of spread based on the squared deviations of values from their mean. |
| standard deviation/ˈstændəd ˌdiːvɪˈeɪʃn/ | A measure of spread equal to the square root of the variance. |
| expectation/ekspɪkˈteɪʃn/ | The probability-weighted average value of a random variable. |
Noise you cannot predict, but can measure
- Every measurement is noisy, and the noise itself cannot be predicted sample by sample.
- Its statistics, though, are stable from second to second. Instruments rely on that stability, and this lesson gives the few rules they use.
Random variables and the Gaussian
- A random variable 随机变量 takes values according to a probability distribution 概率分布. Thermal noise follows the Gaussian (normal) distribution $N(\mu, \sigma^{2})$: $f(x) = \dfrac{1}{\sigma\sqrt{2\pi}}\,e^{-(x-\mu)^{2}/2\sigma^{2}}$.
- The area under every density curve is 1. A wider curve must therefore be lower.
Mean, variance and standard deviation
- Expectation 期望 $E[X] = \mu$ is the long-run average. Variance 方差 $\text{Var}(X) = E[X^{2}] - \mu^{2}$ measures the spread.
- The standard deviation 标准差 $\sigma = \sqrt{\text{Var}(X)}$ is in the signal's own units: it is the noise level a scope shows. Noise with $\sigma = 2\,\text{mV}$ has variance $4\,\text{mV}^{2}$.
Noise has a standard deviation of 3 mV. What is its variance, in mV²?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Variance is the square of the standard deviation: $3^{2} = 9\,\text{mV}^{2}$.
The 2σ rule
- For a Gaussian, about 95% of samples lie within $\pm 1.96\sigma$ of the mean, and only 4.55% fall beyond $\pm 2\sigma$.
- With $\sigma = 2\,\text{mV}$ and $\mu = 0$, the 95% band is about $\pm 3.9\,\text{mV}$: a reading beyond it is unusual, not impossible.
For Gaussian noise with σ = 2 mV, a single reading of 5 mV is impossible.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
5 mV is 2.5σ. Readings that far out are rare (about 1%) but they do happen.
Independent noises add as variances
- For independent sources, variances add: $\sigma_{\text{total}} = \sqrt{\sigma_{1}^{2} + \sigma_{2}^{2}}$. Standard deviations do not add.
- A stage adding 2 mV to a signal already carrying 3 mV gives $\sqrt{9 + 4} = \sqrt{13} \approx 3.61\,\text{mV}$, not 5 mV. The smaller source counts for less than its face value.
Two independent noise sources of 2 mV and 3 mV combine. What is the total?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Independent sources add in quadrature: $\sqrt{4 + 9} = \sqrt{13}$. Only perfectly correlated noise would add linearly.
Independent noise sources of 6 µV and 8 µV combine. What is the total noise, in µV?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
$\sqrt{6^{2} + 8^{2}} = \sqrt{100} = 10\,\mu\text{V}$, well below $6 + 8 = 14$.
Averaging divides the noise by $\sqrt{n}$
- The average of $n$ independent samples has standard deviation $\dfrac{\sigma}{\sqrt{n}}$. Averaging 16 samples of 2 mV noise leaves $\dfrac{2}{\sqrt{16}} = 0.5\,\text{mV}$.
- In decibels, the signal-to-noise ratio improves by $20\log_{10}4 \approx 12\,\text{dB}$. Each further 6 dB needs four times as many samples.
Averaging 16 independent noisy readings improves the signal-to-noise ratio by about
Difficulty 1/5 · 100 coins + a small bonus on first completion today
$\sigma$ falls by $\sqrt{16} = 4$, and $20\log_{10}4 \approx 12$ dB. This holds only for independent noise.
Readings carry 5 mV of independent noise. How many must be averaged to bring it down to 1 mV?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
$\dfrac{5}{\sqrt{n}} = 1$ gives $\sqrt{n} = 5$, so $n = 25$.
Attack the biggest source first
- Halving the dominant 3 mV source to 1.5 mV gives $\sqrt{1.5^{2} + 2^{2}} = 2.5\,\text{mV}$: about 3.2 dB better, with no extra averaging time.
- Design order: reduce the dominant source where it starts, then average what is left.
Three common slips
- Adding standard deviations. Add variances, then take the square root.
- Mixing $\sigma$ and $\sigma^{2}$. The scope shows $\sigma$; most formulas use $\sigma^{2}$.
- Averaging correlated noise. The $\sqrt{n}$ gain needs independence; mains hum repeats every cycle and does not average away.
Averaging 100 readings reduces 50 Hz mains hum by a factor of 10.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
The $\sqrt{n}$ rule needs independent samples. Hum is correlated from reading to reading, so it does not average away; filter or shield it instead.
Recap
- Gaussian noise: mean $\mu$, spread $\sigma$; about 95% lies within $\pm 2\sigma$.
- Independent noises: $\sigma_{\text{total}} = \sqrt{\sigma_{1}^{2} + \sigma_{2}^{2}}$.
- Averaging $n$ independent samples divides $\sigma$ by $\sqrt{n}$; fix the dominant source first.