Problems with Sampling
Introduced| English |
|---|
| undercoverage/ˌʌndəˈkʌvərɪdʒ/ |
| voluntary response/ˈvɒləntəri rɪˈspɒns/ |
| nonresponse/ˌnɒnrɪˈspɒns/ |
| response bias/rɪˈspɒns ˈbaɪəs/ |
| question wording bias/ˈkwestʃn ˈwɜːdɪŋ ˈbaɪəs/ |
Who gets left out?
- Even with a plan, sampling can go wrong — usually by missing or distorting responses.
- Undercoverage 覆盖不足: part of the population has no chance of being chosen.
- Example: a phone survey misses everyone without a phone.
- The reached group then over-represents whoever was reachable.
Voluntary response
- Voluntary response 自愿回应: people choose themselves into the sample (call-ins, web polls).
- Those with strong opinions volunteer most, so the result is skewed.
- It's a self-selected group, not a random one — reliably biased.
- More volunteers doesn't help; they're the same lopsided crowd.
Nonresponse and response bias
- Nonresponse 无回应: selected people can't or won't answer; the answerers may differ from the silent.
- Response bias 回应偏差: people answer, but untruthfully — to look good, or to please the interviewer.
- Sensitive topics (income, habits) invite dishonest answers.
- Both distort results even when the sampling was random.
Question wording
- Question wording bias 问题措辞偏差: a leading or confusing question pushes answers one way.
- "Don't you agree the unfair tax should be cut?" is not neutral.
- The wording, not the respondents' true views, drives the result.
- Neutral, clear wording is part of good design.
A bigger sample does not fix any of these — they are bias, not random error. Undercoverage, voluntary response, nonresponse, response bias, and loaded wording all come from a flawed method, so collecting more the same way just cements the wrong answer. Fix the method, not the size.
A website posts: "Should the mayor resign? Click to vote."
- Voluntary response: only angry visitors bother to click → biased.
- Undercoverage: people who never visit the site have no voice.
- Adding millions of clicks doesn't make it representative.
Sampling bias creeps in through undercoverage (some can't be chosen), voluntary response (self-selection), nonresponse (selected people don't answer), response bias (untruthful answers), and question wording bias. None is cured by a larger sample — only by a better method.
A distorted response mix
Voluntary response over-represents the strongly opinionated.
A phone survey never reaches people without phones. This is...
Part of the population has no chance of selection — undercoverage.
People answer a survey about drinking untruthfully to look responsible. This is...
Untruthful answering is response bias.
A leading question like 'Don't you agree the unfair tax should be cut?' causes question wording bias.
Loaded wording pushes answers one way.
When people self-select into a poll (like a call-in), it is called ___ response bias (one word).
Self-selection = voluntary response.
Increasing the sample size will fix bias caused by a flawed sampling method.
Bias isn't cured by size — only by a better method.