Type I, Type II, and Power

An audit can wrongly flag a reliable seller

• Suppose $H_0$ says the on-time proportion is 0.90 and $H_a$ says it is lower. Rejecting when the null is actually true is a Type I error 第一类错误.
• This is a false alarm about the population rate. A test decision can be wrong even when its calculations were carried out correctly.

An audit can miss a real shortfall

• A Type II error 第二类错误 occurs when the null is false but the study fails to reject it. For example, a true rate of 0.85 might go undetected.
• Non-rejection reports insufficient evidence; it is not a declaration that no shortfall exists. The missed-detection probability depends on the actual alternative rate.

Treat the level as a conditional error target

• Under the null, a correctly calibrated test controls its rejection rate at the chosen $\alpha$. Normal-approximation procedures have approximately that rate.
• The rate is over repeated studies when the null is true. It is not the probability that the null is true or the probability of error after a particular rejection.

Power 功效 is linked to a specified effect

• Power is the probability of rejecting under a specified false-null value: $1-\beta$. Different true rates can give different power.
• For a fixed design and effect, larger samples generally improve power. A larger effect is usually easier to detect.

Changing the threshold changes the trade-off

• With the sample size and true effect fixed, lowering $\alpha$ makes rejection harder and typically lowers power.
• Increasing sample size can reduce missed detections while retaining the same nominal Type I level. It does not automatically lower that chosen level.

Identify errors using truth and decision

• Check two things: whether the null is true and whether the test rejects it. Their combination identifies an error or a correct decision.
• Without knowing the true population rate, a study cannot label its own rejection a realised Type I error solely from the observed p-value.

Write from a power of 0.8

• The probability of missing a real shortfall is 0.2.
• That miss is a Type II error. The lesson power context uses 0.7.