Designing Test Information and Test Information in Design

• Activity Number: 255
• Type: Contributed
• Date/Time: Monday, August 10, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Section on Physical and Engineering Sciences
• Author(s): David Jones* and Xiao-Li Meng
• Companies: Harvard University and Harvard University
• Keywords: optimal design ; hypothesis testing ; model selection ; entropy ; power ; astronomy

Abstract

Degroot (1962) developed a general framework for constructing Bayesian measures of the expected information that an experiment will provide for estimation. We propose an analogous framework for frequentist and Bayesian measures of information for hypothesis testing, and illustrate how such measures can be applied in experimental design. In contrast to estimation information measures that are typically used in experimental design for surface estimation, test information measures are most useful in experimental design for model selection and classification problems. Indeed, our framework suggests a probability based measure of test information, which in hypothesis test applications has more appealing properties than variance based measures. Nicolae et al. (2008) give an asymptotic equivalence between their test information measures and Fisher information, and we extend this result to all test information measures under our framework, and also to Bayesian estimation information measures. Thus, estimation and test information measures can be partially reconciled. We illustrate the use of our test information measures in experimental design with an application from astronomy.