Balancing Competing Objectives for a Good Designed Experiment

• Activity Number: 370
• Type: Roundtables
• Date/Time: Tuesday, August 3, 2010 : 12:30 PM to 1:50 PM
• Sponsor: Section on Physical and Engineering Sciences
• Author(s): Christine M. Anderson-Cook*+
• Companies: Los Alamos National Laboratory
• Keywords: Bias-variance trade-off ; model uncertainty ; missing data ; alphabetic optimality

Abstract

In our daily lives we constantly balance trusting that things will work and protecting against things that might go wrong. When we drive around town, we trust that the other cars will stop at the red light, but also drive defensively enough to anticipate someone running the light. Designing a data collection plan for an experiment should involve the same balancing between trusting our knowledge of the system and having a healthy skepticism that we have correctly described the features of the system. Knowing how to evaluate our choices is a function of being able to articulate both our assumptions about "how things work" and also what potential problems we want to protect against. During this roundtable, we can discuss when "optimal designs" might not be optimal, and how to balance competing objectives to obtain a good designed experiment for practical situations.