DOE: A Critical Component in the Data Scientist’s Toolbox

• Activity Number: 250 - The Future of Designed Experiments in the Era of Big Data
• Type: Invited
• Date/Time: Tuesday, August 4, 2020 : 1:00 PM to 2:50 PM
• Sponsor: Quality and Productivity Section
• Author(s): Abigael Nachtsheim*
• Companies: Arizona State University
• Keywords: A/B testing; Causal inference; Hyperparameter optimization; Machine learning; Observational studies; Transfer learning

Abstract

We consider contributions that the field of experimental design can make to modern model development and data analysis in the context of big data. We provide an overview of the potential pitfalls of relying on purely observational studies, regardless of sample size, especially when questions of causality are of interest to the researcher or practitioner. We explore applications where experimental design approaches have been successfully coupled with machine learning methods to improve the performance of machine learning algorithms. We conclude by considering the importance for continued cross-collaboration and discussion among statisticians and data scientists, with the goal of combining areas of expertise to develop better ideas, higher quality research, and innovative solutions to real-world problems.