Collecting Data · ⁨データ収集⁩

Enduring Understanding (VAR-1): Given that variation may be random or not, conclusions are uncertain.

Learning Objective VAR-1.E: Identify questions to be answered about data collection methods. [Skill 1.A]

• VAR-1.E.1 Methods for data collection that do not rely on chance result in untrustworthy conclusions.

Enduring Understanding (DAT-2): The way we collect data influences what we can and cannot say about a population.

Learning Objective DAT-2.A: Identify the type of a study. [Skill 1.C]

• DAT-2.A.1 A population consists of all items or subjects of interest.
• DAT-2.A.2 A sample selected for study is a subset of the population.
• DAT-2.A.3 In an observational study, treatments are not imposed. Investigators examine data for a sample of individuals (retrospective) or follow a sample of individuals into the future collecting data (prospective) in order to investigate a topic of interest about the population. A sample survey is a type of observational study that collects data from a sample in an attempt to learn about the population from which the sample was taken.
• DAT-2.A.4 In an experiment, different conditions (treatments) are assigned to experimental units (participants or subjects).

Learning Objective DAT-2.B: Identify appropriate generalizations and determinations based on observational studies. [Skill 4.A]

• DAT-2.B.1 It is only appropriate to make generalizations about a population based on samples that are randomly selected or otherwise representative of that population.
• DAT-2.B.2 A sample is only generalizable to the population from which the sample was selected.
• DAT-2.B.3 It is not possible to determine causal relationships between variables using data collected in an observational study.

Enduring Understanding (DAT-2): The way we collect data influences what we can and cannot say about a population.

Learning Objective DAT-2.C: Identify a sampling method, given a description of a study. [Skill 1.C]

• DAT-2.C.1 When an item from a population can be selected only once, this is called sampling without replacement. When an item from the population can be selected more than once, this is called sampling with replacement.
• DAT-2.C.2 A simple random sample (SRS) is a sample in which every group of a given size has an equal chance of being chosen. This method is the basis for many types of sampling mechanisms. A few examples of mechanisms used to obtain SRSs include numbering individuals and using a random number generator to select which ones to include in the sample, ignoring repeats, using a table of random numbers, or drawing a card from a deck without replacement.
• DAT-2.C.3 A stratified random sample involves the division of a population into separate groups, called strata, based on shared attributes or characteristics (homogeneous grouping). Within each stratum a simple random sample is selected, and the selected units are combined to form the sample.
• DAT-2.C.4 A cluster sample involves the division of a population into smaller groups, called clusters. Ideally, there is heterogeneity within each cluster, and clusters are similar to one another in their composition. A simple random sample of clusters is selected from the population to form the sample of clusters. Data are collected from all observations in the selected clusters.
• DAT-2.C.5 A systematic random sample is a method in which sample members from a population are selected according to a random starting point and a fixed, periodic interval.
• DAT-2.C.6 A census selects all items/subjects in a population.

Learning Objective DAT-2.D: Explain why a particular sampling method is or is not appropriate for a given situation. [Skill 1.C]

• DAT-2.D.1 There are advantages and disadvantages for each sampling method depending upon the question that is to be answered and the population from which the sample will be drawn.

Enduring Understanding (DAT-2): The way we collect data influences what we can and cannot say about a population.

Learning Objective DAT-2.E: Identify potential sources of bias in sampling methods. [Skill 1.C]

• DAT-2.E.1 Bias occurs when certain responses are systematically favored over others.
• DAT-2.E.2 When a sample is comprised entirely of volunteers or people who choose to participate, the sample will typically not be representative of the population (voluntary response bias).
• DAT-2.E.3 When part of the population has a reduced chance of being included in the sample, the sample will typically not be representative of the population (undercoverage bias).
• DAT-2.E.4 Individuals chosen for the sample for whom data cannot be obtained (or who refuse to respond) may differ from those for whom data can be obtained (nonresponse bias).
• DAT-2.E.5 Problems in the data gathering instrument or process result in response bias. Examples include questions that are confusing or leading (question wording bias) and self-reported responses.
• DAT-2.E.6 Non-random sampling methods (for example, samples chosen by convenience or voluntary response) introduce potential for bias because they do not use chance to select the individuals.

Enduring Understanding (VAR-3): Well-designed experiments can establish evidence of causal relationships.

Learning Objective VAR-3.A: Identify the components of an experiment. [Skill 1.C]

• VAR-3.A.1 The experimental units are the individuals (which may be people or other objects of study) that are assigned treatments. When experimental units consist of people, they are sometimes referred to as participants or subjects.
• VAR-3.A.2 An explanatory variable (or factor) in an experiment is a variable whose levels are manipulated intentionally. The levels or combination of levels of the explanatory variable(s) are called treatments.
• VAR-3.A.3 A response variable in an experiment is an outcome from the experimental units that is measured after the treatments have been administered.
• VAR-3.A.4 A confounding variable in an experiment is a variable that is related to the explanatory variable and influences the response variable and may create a false perception of association between the two.

Learning Objective VAR-3.B: Describe elements of a well-designed experiment. [Skill 1.B]

• VAR-3.B.1 A well-designed experiment should include the following:
  • a. Comparisons of at least two treatment groups, one of which could be a control group.
  • b. Random assignment/allocation of treatments to experimental units.
  • c. Replication (more than one experimental unit in each treatment group).
  • d. Control of potential confounding variables where appropriate.

Learning Objective VAR-3.C: Compare experimental designs and methods. [Skill 1.C]

• VAR-3.C.1 In a completely randomized design, treatments are assigned to experimental units completely at random. Random assignment tends to balance the effects of uncontrolled (confounding) variables so that differences in responses can be attributed to the treatments.
• VAR-3.C.2 Methods for randomly assigning treatments to experimental units in a completely randomized design include using a random number generator, a table of random values, drawing chips without replacement, etc.
• VAR-3.C.3 In a single-blind experiment, subjects do not know which treatment they are receiving, but members of the research team do, or vice versa.
• VAR-3.C.4 In a double-blind experiment neither the subjects nor the members of the research team who interact with them know which treatment a subject is receiving.
• VAR-3.C.5 A control group is a collection of experimental units either not given a treatment of interest or given a treatment with an inactive substance (placebo) in order to determine if the treatment of interest has an effect.
• VAR-3.C.6 The placebo effect occurs when experimental units have a response to a placebo.
• VAR-3.C.7 For randomized complete block designs, treatments are assigned completely at random within each block.
• VAR-3.C.8 Blocking ensures that at the beginning of the experiment the units within each block are similar to each other with respect to at least one blocking variable. A randomized block design helps to separate natural variability from differences due to the blocking variable.
• VAR-3.C.9 A matched pairs design is a special case of a randomized block design. Using a blocking variable, subjects (whether they are people or not) are arranged in pairs matched on relevant factors. Matched pairs may be formed naturally or by the experimenter. Every pair receives both treatments by randomly assigning one treatment to one member of the pair and subsequently assigning the remaining treatment to the second member of the pair. Alternately, each subject may get both treatments.

Enduring Understanding (VAR-3): Well-designed experiments can establish evidence of causal relationships.

Learning Objective VAR-3.D: Explain why a particular experimental design is appropriate. [Skill 1.C]

• VAR-3.D.1 There are advantages and disadvantages for each experimental design depending on the question of interest, the resources available, and the nature of the experimental units.

Enduring Understanding (VAR-3): Well-designed experiments can establish evidence of causal relationships.

Learning Objective VAR-3.E: Interpret the results of a well-designed experiment. [Skill 4.B]

• VAR-3.E.1 Statistical inference attributes conclusions based on data to the distribution from which the data were collected.
• VAR-3.E.2 Random assignment of treatments to experimental units allows researchers to conclude that some observed changes are so large as to be unlikely to have occurred by chance. Such changes are said to be statistically significant.
• VAR-3.E.3 Statistically significant differences between or among experimental treatment groups are evidence that the treatments caused the effect.
• VAR-3.E.4 If the experimental units used in an experiment are representative of some larger group of units, the results of an experiment can be generalized to the larger group. Random selection of experimental units gives a better chance that the units will be representative.