Inference and Experiments
Introduced| English |
|---|
| random assignment/ˈrændəm əˈsaɪnmənt/ |
| random selection/ˈrændəm sɪˈlekʃn/ |
| statistically significant/stəˈtɪstɪkli sɪɡˈnɪfɪkənt/ |
| scope of inference/skəʊp ɒv ˈɪnfərəns/ |
Two kinds of randomness, two payoffs
- Statistics uses randomness in two different places, with two different rewards.
- Random assignment 随机分配 (to treatments) → lets you claim cause-and-effect.
- Random selection 随机选择 (of units from a population) → lets you generalize to that population.
- Knowing which one a study used tells you what its results can support.
Random assignment → causation
- Randomly assigning units to treatments balances all other variables across groups.
- So any difference in the response can be pinned on the treatment itself.
- This is why experiments — and only experiments — can prove cause.
- Without random assignment, a confounder could always be the real cause.
Random selection → generalization
- Randomly selecting units from the population makes the sample representative.
- So the conclusion generalizes to that whole population.
- Without random selection, results apply only to the units actually studied.
- Selection is about who's in the study; assignment is about what they get.
Significance and scope
- A difference is statistically significant 统计显著 if it's too large to be reasonably explained by chance.
- The scope of inference 推断范围 = what the two randomizations jointly permit.
- Random assignment + random selection → cause-and-effect that generalizes (the strongest scope).
- Only assignment → cause but not generalizable; only selection → generalizable association, not cause.
Two separate randomizations, two separate conclusions. Random assignment buys causation; random selection buys generalization. A study can have one, both, or neither — and its scope of inference is exactly what those two choices allow. Don't claim cause without random assignment, or generalization without random selection.
$60$ volunteers are randomly assigned to a new study method or the old one; scores rise significantly.
- Random assignment → the method caused the gain (a valid causal claim).
- But they were volunteers, not randomly selected → can't generalize to all students.
- Scope: cause-and-effect for these subjects, not a population-wide claim.
Random assignment to treatments licenses a cause-and-effect claim; random selection of units licenses generalization to the population. A difference is statistically significant when it's implausibly large for chance alone. The scope of inference is set by which randomizations a study used.
Random assignment to treatments
Random assignment balances other variables across treatment groups.
Match each randomization to what it permits.
Assignment → causation; selection → generalization.
Volunteers are randomly assigned to two study methods and one wins significantly. You can conclude...
Random assignment gives causation; volunteers block generalization.
A difference too large to be reasonably explained by chance is called statistically ___.
That's the meaning of statistically significant.
Random selection of units from a population is what allows you to generalize the results to that population.
Random selection buys generalization.
A study with random assignment but NO random selection supports...
Assignment → cause; without selection it doesn't generalize.