Practice question · Multiple choice
A researcher collects data, notices an unexpected pattern, and publishes it as the hypothesis the study set out to test. Every number reported is accurate. Why is this nonetheless a serious problem?
Hints
- Ask how many patterns the researcher could have noticed, and how many were reported.
- Every number is right. So what exactly is false in the paper?
Show the answer
B. Because the p-value assumes a single pre-specified test.
Why
Nothing in the paper is numerically wrong, which is what makes it easy to rationalise; the falsehood is structural, presenting a posterior hypothesis as prior. A dataset offers many patterns and some are striking by chance, so choosing after the fact runs many tests while reporting one. Exploratory work is indispensable and needs only to be labelled.
Practise Reporting Research Findings
The app has 5 more questions on this lesson, and keeps your place in the course. Psychology I is free to start.
More questions on Reporting Research Findings
- Sort each practice by whether it tends to worsen or to counter publication-related bias.
- Enough detail for an independent replication belongs in the Results section.
- A finding is peer-reviewed and published in a good journal. What has that process actually established?
- Select every statement that is TRUE about reporting research.
- Well-conducted studies finding nothing are much less likely to be published than studies finding an effect.…
- Order the standard APA report sections as they appear in a paper.