Randomization-Based Inference for Industrial Experiments

• Activity Number: 321
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Section on Physical and Engineering Sciences
• Author(s): Tirthankar Dasgupta* and Joseph Lee and Donald B. Rubin
• Companies: Harvard University and Harvard University and Harvard University
• Keywords: Experimental design ; Causal Inference ; Null hypothesis ; Re-randomization ; Restricted randomization ; Potential Outcomes

Abstract

Randomization tests, conceived by Fisher, are useful tools because they assess the statistical significance of estimated treatment effects without making any assumptions about the underlying distribution of the data. Other attractive features of randomization tests include flexibility in the choice of test statistic and adaptability to experiments with complex randomization schemes and non-standard (e.g., ordinal) data. In the past, these tests' major drawback was their possibly prohibitive computational requirements, even under null hypotheses, which statisticians circumvented by deriving asymptotic distributions of test statistics. Modern computing resources make randomization tests pragmatic, useful tools driven primarily by intuition. In hopes of encouraging a renewed interest in such tests among engineers and practitioners, we outline a principled approach to conducting randomization-based inference in a wide array of industrial settings. We also provide an R package that helps perform randomization tests in various experimental settings.