A prevalent cause of false positives in scientific research is confirmation bias -- the tendency to see find what we expect to find -- backed by p-hacking, i.e., searching among alternative model specifications and techniques until one or more are found that deliver the expected results with apparent statistical significance. To avoid fooling ourselves and others, various technical approaches have been recommended for protecting against p-hacking, ranging from increasing p-values to correct for multiple testing biases, to applying differential privacy techniques to permit statisticians to adaptively explore data sets without corrupting the validity of subsequent inferences (Dwork et al., 2015), to automating the entire process of statistical modeling and analysis so that data sets can be analyzed without human intervention to generate all warranted data-driven conclusions whether or not they support the preconceptions of the investigators. This session will review progress in methods for protecting against p-hacking by automating causal inference and will illustrate their use in scientific inference, hypothesis testing, and causal effect size estimation.
