Dynamite Plots and Deceit

• Activity Number: 545
• Type: Contributed
• Date/Time: Wednesday, August 12, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Education
• Author(s): Steven Millard* and Jane Shofer
• Companies: Probability, Statistics, & Information and University of Washington
• Keywords: dynamite plot ; stripplot ; boxplot ; reproducibility ; deceit ; reviewer

Abstract

Brian Nosek, the founder of the Center for Open Science, has said: "We are incentivized to make our research more beautiful than it is," (The Washington Post, January 27, 2015, "The new scientific revolution: Reproducibility at last"). Despite the variety of graphics software available today and the repeated calls by statisticians and researchers for more informative plots (e.g., Land and Sándor, 2009), the status quo is publications with overly simplistic and potentially misleading plots to accompany data analysis. In this talk we focus on the common use of "dynamite plots" (bar plots with the top at the sample mean and only an upper error bar) to compare two or more groups, and show how some investigators may use these plots to hide very important information from reviewers and readers, even though the results are "statistically significant." We recommend statisticians 1) as collaborators, strongly encourage investigators to use stripplots (or boxplots if the data set is large) with CIs instead of dynamite plots, and 2) as reviewers, require all manuscripts to include plots (possibly as supplementary material) that illustrate the distributional properties of the data.