Type I, Type II, and Power
Introduced| English |
|---|
| Type I error/taɪp aɪ ˈerə/ |
| Type II error/taɪp ˈtuː ˈerə/ |
| power/ˈpaʊə/ |
An audit can wrongly flag a reliable seller
- Suppose $H_0$ says the on-time proportion is 0.90 and $H_a$ says it is lower. Rejecting when the null is actually true is a Type I error 第一类错误.
- This is a false alarm about the population rate. A test decision can be wrong even when its calculations were carried out correctly.
An audit can miss a real shortfall
- A Type II error 第二类错误 occurs when the null is false but the study fails to reject it. For example, a true rate of 0.85 might go undetected.
- Non-rejection reports insufficient evidence; it is not a declaration that no shortfall exists. The missed-detection probability depends on the actual alternative rate.
A Type I error is...
Type I = false alarm = reject a true null.
Treat the level as a conditional error target
- Under the null, a correctly calibrated test controls its rejection rate at the chosen $\alpha$. Normal-approximation procedures have approximately that rate.
- The rate is over repeated studies when the null is true. It is not the probability that the null is true or the probability of error after a particular rejection.
A Type II error is...
Type II = missed detection = fail to reject a false null.
H0 says the delivery proportion is 0.90. Match the outcome to its meaning.
Classify by the true state and the decision, not by whether the sample percentage looks large.
Power 功效 is linked to a specified effect
- Power is the probability of rejecting under a specified false-null value: $1-\beta$. Different true rates can give different power.
- For a fixed design and effect, larger samples generally improve power. A larger effect is usually easier to detect.
If β = 0.30 at p = 0.85, power is 0.70 at p = 0.85. This is not a power claim for every possible alternative.
For a test calibrated at significance level α, α is the nominal Type I error level.
α is the chosen false-rejection level under H0. Approximate and discrete procedures need not have actual error probability exactly α.
The power of a test equals 1 minus ___ (the Greek letter for the Type II error rate).
Power = 1 − β.
At a specified true proportion, the Type II error probability is 0.30. What is the power?
Power = 1 - β = 1 - 0.30 = 0.70 for that particular alternative.
Changing the threshold changes the trade-off
- With the sample size and true effect fixed, lowering $\alpha$ makes rejection harder and typically lowers power.
- Increasing sample size can reduce missed detections while retaining the same nominal Type I level. It does not automatically lower that chosen level.
At fixed nominal α, a larger sample does not automatically reduce the chosen Type I error level.
Which increase the power of a test?
Bigger n, bigger effect, or larger α all raise power.
Increasing n automatically lowers the nominal Type I error level when α stays at 0.05.
The chosen nominal level remains 0.05. Larger n usually improves power for a specified effect under the same valid model; approximate actual error rates may vary.
Identify errors using truth and decision
- Check two things: whether the null is true and whether the test rejects it. Their combination identifies an error or a correct decision.
- Without knowing the true population rate, a study cannot label its own rejection a realised Type I error solely from the observed p-value.
Check two things: whether the null is true and whether the test rejects it. Their combination identifies an error or a correct decision.