|
Activity Number:
|
139
- Competing Effectively: Hosting, Designing, and Participating in Kaggle-Style Competitions
|
|
Type:
|
Invited
|
|
Date/Time:
|
Monday, July 30, 2018 : 10:30 AM to 12:20 PM
|
|
Sponsor:
|
Section on Statistics in Defense and National Security
|
|
Abstract #326778
|
Presentation
|
|
Title:
|
Bayesian Design of Experiments with Multiple Priors for Kaggle Competition Design
|
|
Author(s):
|
Kevin Randal Quinlan* and Christine M Anderson-Cook
|
|
Companies:
|
The Pennsylvania State University and Los Alamos National Laboratory
|
|
Keywords:
|
Design of Experiments;
Bayesian D-optimal Designs;
Logistic Regression;
Kaggle Competition
|
|
Abstract:
|
When assessing the performance of multiple algorithms in a competition setting, it is desirable to construct a design that considers a variety of performance levels across the algorithms. We describe a strategy to use Bayesian design of experiments with multiple prior estimates to capture anticipated performance. Our goal is to characterize results from the different algorithms as a function of different explanatory variables and use this to help choose a design about which units to test. We use this approach to develop methodology for the case where there are several potentially non-overlapping priors under consideration. While multiple priors have been used for analysis in the past, they have not been used in a design context. The Weighted Priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other reasonable design choices. We illustrate the method as well through multiple scenarios and a motivating example. Additionally, we propose multiple new plots which are useful for evaluating design performance in higher dimensional problems.
|
Authors who are presenting talks have a * after their name.
