Inference and Experiments
Introduit| English | Sens |
|---|---|
| random assignment/ˈrændəm əˈsaɪnmənt/ | affectation aléatoire |
| random selection/ˈrændəm sɪˈlekʃn/ | tirage aléatoire |
| statistically significant/stəˈtɪstɪkli sɪɡˈnɪfɪkənt/ | significatif statistiquement |
| scope of inference/skəʊp ɒv ˈɪnfərəns/ | champ d'inférence |
Two kinds of randomness, two payoffs
- Statistics uses randomness in two different places, with two different rewards.
- Random assignment 随机分配 (to treatments) → lets you claim cause-and-effect.
- Random selection 随机选择 (of units from a population) → lets you generalize to that population.
- Knowing which one a study used tells you what its results can support.
Random assignment → causation
- Random assignment tends to balance known and unknown variables across groups, but chance imbalances can remain.
- Compare the observed response difference with chance variation before attributing evidence to a treatment effect.
- A well-designed randomized experiment can support a causal conclusion; random assignment alone does not prove that an effect exists.
- Without random assignment, a confounder could always be the real cause.

If random assignment alone rarely produces a difference this large, the result is statistically significant 统计显著
Random selection → generalization
- Random selection supports population inference when coverage, response, and measurement are adequate; a particular sample need not match the population exactly.
- So the conclusion generalizes to that whole population.
- Without random selection, results apply only to the units actually studied.
- Selection is about who's in the study; assignment is about what they get.
Significance and scope
- A difference is statistically significant if it's too large to be reasonably explained by chance.
- The scope of inference 推断范围 = what the two randomizations jointly permit.
- Random assignment + random selection → cause-and-effect that generalizes (the strongest scope).
- Only assignment → cause but not generalizable; only selection → generalizable association, not cause.
Two separate randomizations, two separate conclusions. Random assignment buys causation; random selection buys generalization. A study can have one, both, or neither — and its scope of inference is exactly what those two choices allow. Don't claim cause without random assignment, or generalization without random selection.
$60$ volunteers are randomly assigned to a new study method or the old one; scores rise significantly.
- Random assignment → the method caused the gain (a valid causal claim).
- But they were volunteers, not randomly selected → can't generalize to all students.
- Scope: cause-and-effect for these subjects, not a population-wide claim.
Random assignment to treatments licenses a cause-and-effect claim; random selection of units licenses generalization to the population. A difference is statistically significant when it's implausibly large for chance alone. The scope of inference is set by which randomizations a study used.
Write from this case
- Students are randomly assigned to two study apps.
- The students were not randomly chosen from all schools.
Students were randomly assigned to two study apps but were not randomly sampled from all schools. What scope of inference does the design permit, and does assignment alone establish that the apps differ?
Assignment and selection answer different questions.
What does the random assignment support?
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
Assignment supports cause. It does not create a random sample of schools.
Random assignment to treatments
Random assignment balances other variables across treatment groups.
Match each randomization to what it permits.
Difficulté 2/5 · 140 pièces + un petit bonus lors de la première réussite aujourd'hui
Assignment → causation; selection → generalization.
Volunteers are randomly assigned to two study methods and one wins significantly. You can conclude...
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
Random assignment gives causation; volunteers block generalization.
A difference too large to be reasonably explained by chance is called statistically ___.
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
That's the meaning of statistically significant.
Random selection of units from a population is what allows you to generalize the results to that population.
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
Random selection buys generalization.
A study with random assignment but NO random selection supports...
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
Assignment → cause; without selection it doesn't generalize.