Moving to a World beyond p < .05

Abstract: Numerous authors have highlighted the limitations of the Null-Hypothesis Significance Testing (NHST) approach and the (near exclusive) reliance on p-values and significance testing. Beyond the methodological advantages and disadvantages of NHST compared to other paradigms of statistical inference, one of the main barriers to the correct use of the NHST procedure remains its complexity, often hidden by misleading intuitive interpretation. For instance, a p-value does not indicate the probability of obtaining certain data “by chance”, nor the probability of “being wrong”. Recently, The American Statistician published a special issue named “Statistical Inference in the 21st Century: A World Beyond p < 0.05”, with the intention to provide new recommendations for users of statistics (e.g., researchers, policy makers, journalists). This issue comprises 43 original papers aiming to provide new guidelines and practical alternatives to the “mindless” use of statistics and arbitrary thresholds. In the accompanying editorial, Wasserstein, Schirm, & Lazar (2019) summarise these recommendations in the form of the ATOM guidelines: “Accept uncertainty. Be thoughtful, open, and modest.” We will explore some consequences of these guidelines when applied to the analysis of empirical data, in the light of core concepts from the philosophy of statistics.