Using Clustering To Enhance Hypothesis Testing

• Activity Number: 316
• Type: Topic Contributed
• Date/Time: Tuesday, August 9, 2005 : 2:00 PM to 3:50 PM
• Sponsor: ENAR
• Author(s): David Dahl*+ and Michael Newton
• Companies: Texas A&M University and University of Wisconsin, Madison
• Keywords: Correlated hypothesis tests ; Model-based clustering ; DNA microarrays ; Bayesian nonparametrics ; Conjugate Dirichlet process mixture model

Abstract

Both multiple hypothesis testing and clustering have been the subject of extensive research in high-dimensional inference, yet these problems traditionally have been treated separately. We propose a hybrid statistical methodology that uses clustering information to increase testing sensitivity. A test for object i that uses data from all objects clustered with it will be more sensitive than one that uses data from object i in isolation. While the true clustering is unknown, there is increased power if the clustering can be estimated relatively well. We first consider a simplified setting that compares the power of the standard Z-test to the power of a test using an estimated cluster. Theoretical results show that if the cluster is estimated sufficiently, the new procedure is more powerful. In the setting of gene expression data, we develop a model-based analysis using a carefully formulated conjugate Dirichlet process mixture model. The model is able to borrow strength from objects likely to be clustered. Simulations reveal this new method performs substantially better than its peers. The proposed model is illustrated on a large microarray dataset.