Information Criteria as Alternatives to Hypothesis Testing

• Activity Number: 133
• Type: Contributed
• Date/Time: Monday, August 5, 2013 : 8:30 AM to 10:20 AM
• Sponsor: Biopharmaceutical Section
• Author(s): Charles Tan*+
• Companies: Pfizer Inc.
• Keywords: information criteria ; hypothesis testing ; assay validation ; clinical trial

Abstract

Even for well planned experiments, such as clinical trials or assay validation studies, we only have resources and information to properly power a few primary hypotheses. However, many statistical decisions have to be made along the way, such as whether treatment by center interaction is small enough to be ignored, or whether variability estimates are consistent enough to be pooled, or whether simple linear regression is a good enough model for standard curves within a given range. Compared to the primary goals of the experiments, these questions are stepping stones and are tactical in nature. In current practice, these questions are answered using significance tests, with the simpler and more desirable model as the null hypothesis. There are serious flaws with this approach: it rewards noisier and smaller experiments and punishes well-controlled and larger experiments. Information criteria, e.g. AIC, is contrasted with significance tests as an alternative paradigm to make statistical decisions. It is argued that information criteria are the more appropriate and sensible way to answer those tactical questions. Examples will be used to illustrate the information criteria approach.