Abstract:
|
Motivated by our joint work with a medical device manufacturer on hyperparameter tuning of deep neural networks, in this study, we propose a model-robust experimentation strategy for estimating stochastic black-box functions (BBF) with mixed quantitative and qualitative (QQ) factors. The complexity of the BBF determines the appropriate strategy to be adopted, for example, when locally second-order, Response Surface Optimization (RSO) methods are highly efficient and when complex, active learning strategies for Gaussian Process surrogate models (AL-GP) are efficient. For initializing our model-robust experimentation strategy, we propose compromise designs between space-filling and Bayesian D-optimal designs that are supersaturated for a full second-order model. And for dynamically selecting the appropriate model we propose a bootstrap goodness-of-fit test for validating the second-order approximation. Our simulation study illustrates that the proposed strategy is highly efficient for estimating second-order BBFs and has high power to detect the inadequacy of the second-order approximation. Additionally, for complex BBFs, the proposed strategy has comparable performance to AL-GP.
|