Inference and Experiments

Two kinds of randomness, two payoffs

• Statistics uses randomness in two different places, with two different rewards.
• Random assignment 随机分配 (to treatments) → lets you claim cause-and-effect.
• Random selection 随机选择 (of units from a population) → lets you generalize to that population.
• Knowing which one a study used tells you what its results can support.

Random assignment → causation

• Randomly assigning units to treatments balances all other variables across groups.
• So any difference in the response can be pinned on the treatment itself.
• This is why experiments — and only experiments — can prove cause.
• Without random assignment, a confounder could always be the real cause.

If random assignment alone rarely produces a difference this large, the result is statistically significant 统计显著

Random selection → generalization

• Randomly selecting units from the population makes the sample representative.
• So the conclusion generalizes to that whole population.
• Without random selection, results apply only to the units actually studied.
• Selection is about who's in the study; assignment is about what they get.

Significance and scope

• A difference is statistically significant if it's too large to be reasonably explained by chance.
• The scope of inference 推断范围 = what the two randomizations jointly permit.
• Random assignment + random selection → cause-and-effect that generalizes (the strongest scope).
• Only assignment → cause but not generalizable; only selection → generalizable association, not cause.

Write from random assignment and cause

• Students are randomly assigned to two study apps.
• The students were not randomly chosen from all schools.