ReproducibiliTea: Musical Contrast and Chronological Rejuvenation

The third OSCR ReproducibiliTea journal club will take place on January 22nd at 11:00 in room VB-02 (V building). I will be the discussant, and we will talk about the following paper¹:

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

This paper, now considered a classic reference when discussing open science in psychological research, shows how unreported flexible choices in data collection, analysis, and reporting can increase false-positive rates. Using an amusing hypothetical experiment – does listening to a song about older age make people actually younger? – as well as computer simulations, the authors show how undisclosed researchers’ degrees of freedom – e.g., flexibly removing outliers, sequential testing without proper alpha correction – can increase the chances to obtain statistically significant (i.e., p < .05) results corroborating absurd conclusions.

Although this paper was written with psychologists as the main target audience, researchers’ degrees of freedom are ubiquitous in most areas of quantitative inquiry. During our session, the attendees will be encouraged to examine such practices in their own field of study, as well as possible solutions to mitigate their negative effects.

An invitation on Outlook calendar has been sent to researchers in the ReproducibiliTea mailing list (if you want to be included, contact Antonio). Please explicitly Accept or Decline the invitation, so that catering and room capacity can be evaluated accordingly.