This week at the MRG seminar a colleague gave a talk about why, what, and how to do pre-registration in science. Before the session, he suggested us to read the following paper:
PNAS March 13, 2018 115 (11) 2600-2606; DOI: https://doi.org/10.1073/pnas.1708274114
The preregistration revolution
Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor
ABSTRACT. Progress in science relies in part on generating hypotheses with existing observations and testing hypotheses with new observations. This distinction between postdiction and prediction is appreciated conceptually but is not respected in practice. Mistaking generation of postdictions with testing of predictions reduces the credibility of research findings. However, ordinary biases in human reasoning, such as hindsight bias, make it hard to avoid this mistake. An effective solution is to define the research questions and analysis plan before observing the research outcomes—a process called preregistration. Preregistration distinguishes analyses and outcomes that result from predictions from those that result from postdictions. A variety of practical strategies are available to make the best possible use of preregistration in circumstances that fall short of the ideal application, such as when the data are preexisting. Services are now available for preregistration across all disciplines, facilitating a rapid increase in the practice. Widespread adoption of preregistration will increase distinctiveness between hypothesis generation and hypothesis testing and will improve the credibility of research findings.
Here are a few kew concepts & papers of the session:
- Publication bias (Sheel, 2020)
- The false positive problem (Simmons, 2011)
- The Garden of Forking Paths (Gelman and Loken, 2013)
- Research reproducibility (Open Science Community)
- Pre-registration (Open Science Foundation)
- Registered reports (Nosek and Lakens, 2014)
Feature image from Pexels – C0 license.