Choose a method that can answer the question · 选择能回答问题的方法
An opinion survey cannot measure every outcome
- The materials team asks, “Do you like our new label layout?” A positive answer may show preference, but it does not show whether the reader can sort the four items correctly. The method must match the outcome in the question.
- This original GAC033-aligned lesson compares an observation, a survey and a small experiment. All are useful for some questions. None is automatically the best method for every topic, and an elaborate method cannot repair a vague research question.
意见调查无法衡量所有结果
- 物料组提问：“你喜欢我们的新标签布局吗？”肯定回答可能显示偏好，但不能证明读者能正确分类四件物品。方法必须与问题中的结果指标相匹配。
- 本原创GAC033配套课程比较了观察、调查和小型实验。这三种方法对某些问题均有用处，但并非每种方法都自动适用于所有课题，且复杂的方法也无法弥补研究问题的模糊性。
What does “Do you like this layout?” measure most directly? · “你喜欢这个布局吗？”最直接衡量什么？
Liking is an opinion outcome. It cannot replace a sorting-accuracy measure. · 喜欢是一种观点结果。它不能替代分类准确度指标。
Compare what each method tells you
- An observation can record what occurs in a setting, but recording errors alone may not reveal why they happen. A survey can ask about experience or preference, but answers may differ from behaviour. Avoid reporting either as evidence of an outcome it did not measure.
- An experiment deliberately changes a feature and compares an outcome under planned conditions. For the label project, the team proposes changing layout while keeping words, item choices, time and instructions the same. Poorly controlled differences would weaken the interpretation.
| Question | Possible method | Important limit |
|---|---|---|
| What errors occur at the real trays? | Observation with approval | Motives are not directly observed |
| Which layout do readers prefer? | Survey | Preference is not sorting accuracy |
| Does accuracy differ between layouts? | Planned comparison task | Small paper task is not everyday use |
比较各方法提供的信息
- 观察可以记录特定环境中的发生情况，但仅凭记录错误本身无法揭示其成因。调查可询问经验或偏好，但回答可能与实际行为不符。避免将两者作为未测量结果的证据进行报告。
- 实验通过有意改变某一特征并在计划条件下比较结果来进行。针对标签项目，团队提议在保持文字、物品选项、时间和说明不变的情况下更改布局。若控制不佳的差异会削弱解释力。
| 问题 | 可能的方法 | 重要限制 |
|---|---|---|
| 真实托盘上出现哪些错误？ | 经批准的观察 | 动机无法直接观察 |
| 读者更喜欢哪种布局？ | 调查 | 偏好不等于分拣准确率 |
| 不同布局间的准确率是否有差异？ | 计划对比任务 | 简单的纸质任务不代表日常使用场景 |
Plan a fair small comparison
- The proposal uses 24 willing peers: 12 see A and 12 see B, assigned by shuffled slips rather than the team's preference. Each completes the same four-item paper sorting task within two minutes, without coaching. The labels' wording remains the same; spacing and alignment differ.
- Score one completely correct sheet only when all four placements match the key fixed before the task. Record the number of complete successes in each group, not a changing mixture of partial marks and whole-sheet counts. No main study has been conducted yet.
Outcome rule: four correct placements = one success; any placement incorrect or blank = no complete success. A separate error-category note may aid interpretation, but it must not silently change the primary outcome.
规划公平的小型对比
- 提案使用24名自愿同伴：12人看A版，12人看B版，通过随机抽取纸条分配而非由团队偏好决定。每人需在两分钟内完成相同的四项纸质分拣任务，且不接受指导。标签措辞保持一致；间距和对齐方式有所不同。
- 仅当四项放置均与任务前确定的标准答案完全匹配时，才计为一份完全正确。记录每组的完全成功数量，而非混合部分得分与整份计数。目前尚未开展正式研究。
结果判定规则：四项放置正确 = 一次成功；任何一项放置错误或空白 = 无完全成功。单独的误差类别备注有助于解读，但不得悄无声息地改变主要结果指标。
Which TWO features should stay the same in the proposed layout comparison? · 在拟议的布局比较中哪两个特征应保持相同？
Keeping wording and timing equal reduces alternative explanations. Unequal coaching or keys would confound the comparison. · 保持文字和时间相等可减少替代解释。不等的指导或钥匙会使比较混淆。
A sheet has three correct placements and one blank. How many complete successes does it count as? · 一张纸有三个正确放置和一个空白。它算作多少次完全成功？
The fixed rule requires all four placements correct. Partial accuracy does not meet this outcome. · 固定规则要求所有四个放置正确。部分准确不符合此结果。
Describe the sample honestly
- A sample is the participating group. Volunteers from one class are not all students everywhere. Random assignment within that group can make the A/B comparison fairer, but it is different from randomly sampling the entire school.
- State who is invited, how many are planned and what happens if fewer agree. Do not recruit only confident readers for B and less confident readers for A. With a small group, random assignment still does not guarantee identical groups in every characteristic.
Do not run a participant task before the teacher approves the procedure. Participation must be voluntary, separate from grades, and explained without pressure from teammates or teachers.
诚实地描述样本
- 样本指参与群体。来自一个班级的志愿者不能代表所有学生。在该组内进行随机分配可使A/B对比更公平，但这不同于对整个学校进行随机抽样。
- 明确说明邀请对象、计划人数及同意人数不足时的应对措施。不要只为B组招募自信的读者，而为A组招募不自信的读者。即使在小规模群体中，随机分配也不能保证各组在所有特征上都完全一致。
切勿在教师批准程序前让参与者执行任务。参与必须出于自愿，不计入成绩，且不受队友或教师的压力影响。
Randomly assigning 24 class volunteers guarantees they represent every student in the school. · 随机分配24名班级志愿者保证他们代表学校每个学生。
Assignment to conditions and sampling from a population are different. One-class volunteers still limit generalisation. · 分配到条件和从总体抽样是不同的。单一班级的志愿者仍限制推广。
Make an ethical procedure explicit
- Explain the task, purpose and use of anonymous totals before anyone agrees. Participants may stop or leave a sheet unsubmitted. Collect no names or identifying comments. Explain that after a sheet is submitted without an identifier, the team cannot reliably retrieve one person's data.
- Use ordinary printed materials, not a task that risks damage, embarrassment or disclosure. Keep records in the approved project location and agree access and disposal with the teacher. “No names” does not excuse careless handling of other identifying details.
明确伦理程序
- 在任何人同意之前，解释任务内容、目的及匿名汇总的使用方式。参与者有权中途退出或不提交表格。不收集姓名或识别性评论。说明一旦表格在无标识的情况下提交，团队将无法可靠地获取该人的数据。
- 使用普通印刷材料，避免可能导致损坏、尴尬或泄露的任务。记录应存放在已批准的项目位置中，并与教师就访问权限和销毁达成一致。“无姓名”不能成为粗心处理其他识别细节的借口。
Pilot the procedure, not your conclusion
- With approval, ask two people to check whether instructions are understandable and timing is workable. This pilot is a rehearsal of the method. If the team changes instructions afterward, keep those pilot sheets out of the main comparison.
- Write a protocol someone else could follow: recruitment, assignment, materials, instructions, clock, scoring, privacy and analysis. Mark a sentence that could lead two helpers to run the task differently and make it precise before any main collection.
Match the method to the question, define the outcome in advance, control relevant differences and make sample and ethical limits explicit.
试运行程序，而非结论
- 经批准后，请两人检查说明是否易懂、时间是否可行。此次预测试是方法的彩排。若团队后续修改了说明，这些预测试表格不得纳入主对比。
- 撰写他人可遵循的协议：包括招募、分配、材料、说明、计时、评分、隐私和分析。标记出可能导致两名助手操作不一致的句子，并在正式收集前使其精确化。
使方法与问题相匹配，预先定义结果，控制相关差异，并明确样本与伦理限制。
Order these method-development steps. · ordered these method-development steps. (请有序排列这些方法开发步骤)
Approval precedes participant work; the pilot helps refine the procedure before the main task. · 批准先于参与者工作；试点有助于在主任务前完善程序。