ReproducibiliTea: Musical Contrast and Chronological Rejuvenation
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
This paper, now considered a classic reference when discussing open science in psychological research, shows how unreported flexible choices in data collection, analysis, and reporting can increase false-positive rates. Using an amusing hypothetical experiment – does listening to a song about older age make people actually younger? – as well as computer simulations, the authors show how undisclosed researchers’ degrees of freedom – e.g., flexibly removing outliers, sequential testing without proper alpha correction – can increase the chances to obtain statistically significant (i.e., p < .05) results corroborating absurd conclusions.
Although this paper was written with psychologists as the main target audience, researchers’ degrees of freedom are ubiquitous in most areas of quantitative inquiry. During our session, the attendees will be encouraged to examine such practices in their own field of study, as well as possible solutions to mitigate their negative effects.
An invitation on Outlook calendar has been sent to researchers in the ReproducibiliTea mailing list (if you want to be included, contact firstname.lastname@example.org). Please explicitly Accept or Decline the invitation, so that catering and room capacity can be evaluated accordingly.
The next meeting will take place on February 12th 2020 at 11:00 in room room J7-55 (Bayle building). We will have a special session with the founders of the Journal of Trial and Error, a new initiative focusing on the importance of publishing negative and null results (details will follow)… join us!