Two-Way Gaussian Mixture Models for High-Dimensional Classification

• Activity Number: 112
• Type: Topic Contributed
• Date/Time: Monday, August 2, 2010 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Mu Qiao*+ and Jia Li
• Companies: Penn State and Penn State
• Keywords: Two-way mixture model ; Mixture of Gaussian distributions ; High dimensional classification ; Variable grouping ; Dimension reduction

Abstract

Mixture discriminant analysis (MDA) has gained applications in a wide range of engineering and scientific fields. In this paper, under the paradigm of MDA, we propose a two-way Gaussian mixture model for classifying high dimensional data. This model regularizes the mixture component means by dividing variables into groups and then constraining the parameters for the variables in the same group to be identical. The grouping of the variables is optimized as part of model estimation. A dimension reduction property for a two-way mixture of distributions from a general exponential family is proved. Estimation methods for the two-way Gaussian mixture with or without missing data are derived. Experiments on several data sets show that the parsimonious two-way mixture often outperforms a mixture model without variable grouping; and as a byproduct, significant dimension reduction is achieved.