Conference Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 241 - SPEED: Statistics in Social Sciences and Survey Research Part 1
Type: Contributed
Date/Time: Tuesday, August 9, 2022 : 8:30 AM to 10:20 AM
Sponsor: Quality and Productivity Section
Abstract #323391
Title: Model-Robust Experimentation Strategy for Estimation of Expensive Black-Box Functions with Mixed Quantitative and Qualitative Factors
Author(s): Gautham Sunder* and Christopher Nachtsheim
Companies: Carlson School of Management and Carlson School of Management
Keywords: Black-box Optimization; Model Robust Designs; Bayesian Optimization; Sequential Experiments; Goodness-of-fit test
Abstract:

Motivated by our joint work with a medical device manufacturer on hyperparameter tuning of deep neural networks, in this study, we propose a model-robust experimentation strategy for estimating stochastic black-box functions (BBF) with mixed quantitative and qualitative (QQ) factors. The complexity of the BBF determines the appropriate strategy to be adopted, for example, when locally second-order, Response Surface Optimization (RSO) methods are highly efficient and when complex, active learning strategies for Gaussian Process surrogate models (AL-GP) are efficient. For initializing our model-robust experimentation strategy, we propose compromise designs between space-filling and Bayesian D-optimal designs that are supersaturated for a full second-order model. And for dynamically selecting the appropriate model we propose a bootstrap goodness-of-fit test for validating the second-order approximation. Our simulation study illustrates that the proposed strategy is highly efficient for estimating second-order BBFs and has high power to detect the inadequacy of the second-order approximation. Additionally, for complex BBFs, the proposed strategy has comparable performance to AL-GP.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2022 program