Hypothesis testing and contextual conclusions
Introduced| English | Meaning |
|---|---|
| hypothesis test/haɪˈpɒθəsɪs test/ | A statistical procedure using sample evidence to assess a specified claim about a population or model. |
| null hypothesis/nʌl haɪˈpɒθəsɪs/ | The default claim about a population parameter that a statistical test assesses against an alternative. |
| p-value/piː ˈvæljuː/ | the probability of data at least this extreme if the null hypothesis were true |
| significance level/sɪɡˈnɪfɪkəns ˈlevl/ | The chosen probability of rejecting a true null hypothesis in a statistical test. |
The lady tasting tea, 1935
- In The Design of Experiments (1935), Ronald Fisher described a colleague who claimed she could tell whether milk or tea had been poured into the cup first. He proposed a test: eight cups, four of each, in random order. Guessing would pick all four correctly with probability only $\tfrac{1}{70}$.
- That is the logic of a hypothesis test 假设检验: assume the null hypothesis 原假设 (no effect), and ask how surprising the data would be. A small p-value p值 is evidence against it.
In Fisher's tea test, chance alone picks all four milk-first cups with probability 1/n. Find n.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
C(8, 4) = 70.
The steps
- State $H_0$ and $H_1$ in context. Choose a significance level, often 5%.
- Find the p-value: the probability, if $H_0$ is true, of a result at least as extreme as the one observed. If $p <$ significance level, reject $H_0$.
At the 5% level, a p-value of 0.03 means
Difficulty 1/5 · 100 coins + a small bonus on first completion today
0.03 < 0.05.
A binomial test
- With $H_0: p = 0.1$, $X \sim B(10, 0.1)$. A large observed $X$ is evidence that $p > 0.1$.
Hypothesis testing and contextual conclusions
State H₀ and H₁ in population parameters before inspecting the outcome
Compare the model with the worked case and explain one change.
Testing whether a coin is biased towards heads, which is H₁?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
One-tailed, towards heads.
Other tests in the course
- $\chi^2$ test for independence (contingency tables) and goodness of fit; $t$-test for comparing means; tests on correlation.
- The GDC gives the statistic and p-value; you state hypotheses and conclusions.
Which test checks independence in a contingency table?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Rows against columns.
Worked example: is the machine faulty?
A machine normally makes 10% faulty parts. In a sample of 10, 4 are faulty. Test at 5%.
- Hypotheses: $H_0: p = 0.1$; $H_1: p > 0.1$.
- p-value: $P(X \ge 4) = 1 - P(X \le 3) = 1 - (0.3487 + 0.3874 + 0.1937 + 0.0574) = 1 - 0.9872 = 0.0128$.
- Conclusion: $0.0128 < 0.05$, so reject $H_0$. There is significant evidence at the 5% level that the machine produces more than 10% faulty parts.
In the machine example, find the p-value P(X ≥ 4) to 4 d.p.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
1 − 0.9872.
Errors
- Type I error: rejecting a true $H_0$ (probability = significance level). Type II error: failing to reject a false $H_0$.
A Type I error is rejecting H₀ when it is true.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Its probability is the significance level.
"Not rejecting $H_0$" does not prove $H_0$. A p-value of 0.2 means the data are not surprising under $H_0$, not that $H_0$ is true. Write conclusions in context and with hedged language.
A p-value of 0.2 proves H₀ is true.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
It only fails to reject it.
State $H_0$ and $H_1$ in context, choose a significance level, compute the p-value assuming $H_0$, and reject $H_0$ if $p$ is smaller. Conclude in context; failing to reject is not proof, and a Type I error is rejecting a true $H_0$.
A die is rolled 30 times and shows six 9 times. Test at 5% whether it is biased towards six.
Hypotheses, p-value, conclusion.