Type I, Type II, and Power
| English | Chinese | Pinyin |
|---|---|---|
| Type I error | 第一类错误 | dì yī lèi cuò wù |
| Type II error | 第二类错误 | dì èr lèi cuò wù |
| power | 功效 | gōng xiào |
Type I error
- A test can reach a wrong verdict in two ways.
- A Type I error 第一类错误 is rejecting a true $H_0$ — a "false alarm."
- You conclude there's an effect when there really isn't.
- Its probability is exactly the significance level $\alpha$.
Type II error
- A Type II error 第二类错误 is failing to reject a false $H_0$ — a "missed detection."
- You conclude there's no effect when there really is one.
- Its probability is written $\beta$.
- The two error types trade off: shrinking one tends to grow the other.
Alpha controls Type I
- The probability of a Type I error equals $\alpha$, which you choose.
- Pick a small $\alpha$ (e.g. $0.01$) when a false alarm is costly.
- But a smaller $\alpha$ makes it harder to reject, raising the Type II risk.
- Choosing $\alpha$ balances the two costs.
Power: catching a real effect
- The power 功效 of a test is the probability of correctly rejecting a false $H_0$ (power $= 1 - \beta$).
- It's the chance of detecting an effect that's really there.
- Increase power with a larger sample size, a bigger effect size, or a larger $\alpha$.
- High power = a test that rarely misses a genuine effect.
Don't swap the two errors. Type I = a false alarm (reject a true $H_0$), probability $\alpha$. Type II = a miss (fail to reject a false $H_0$), probability $\beta$. Lowering $\alpha$ guards against false alarms but raises the miss rate (lowers power). The reliable way to reduce both is a larger sample.
A medical test for a disease ($H_0$: no disease).
- Type I: telling a healthy person they're sick (false alarm), probability $\alpha$.
- Type II: telling a sick person they're healthy (missed detection), probability $\beta$.
- Power $= 1 - \beta$: correctly detecting the disease; a bigger sample raises it.
A Type I error rejects a true $H_0$ (probability $\alpha$); a Type II error fails to reject a false $H_0$ (probability $\beta$). The two trade off. A test's power $= 1 - \beta$ is its chance of detecting a real effect, increased by a larger sample, a bigger effect size, or a larger $\alpha$.
The Type I error rate α
The shaded tail area is α — the chance of a false alarm.
A Type I error is...
Type I = false alarm = reject a true null.
A Type II error is...
Type II = missed detection = fail to reject a false null.
The probability of a Type I error equals the significance level α.
By construction, P(Type I) = α.
Which increase the power of a test?
Bigger n, bigger effect, or larger α all raise power.
The power of a test equals 1 minus ___ (the Greek letter for the Type II error rate).
Power = 1 − β.