On Lassoing Mixtures

• Activity Number: 70
• Type: Contributed
• Date/Time: Sunday, August 7, 2005 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Guan Xing*+ and J. Sunil Rao
• Companies: Case Western Reserve University and Case Western Reserve University
• Keywords: Mixture ; Lasso ; EM

Abstract

We consider the problem of selecting the number of components in a density estimate estimated by a finite mixture of Gaussian components from a sample of size n. We develop a novel characterization of the mixture by casting it as a special type of linear regression model where created responses based on cumulative probabilities from the full n component mixture are regressed upon the sum of the individual mixture components. The mixture weights are now regression parameters to be estimated. Sparse solutions are found by using L1 constrained lasso estimates with sum-to-one and positivity constraints using a variation of the positive-lars algorithm of Efron et al. (2004). Individual component parameters are estimated by an EM step. Iteration between the two estimation steps till convergence leads to a sparse characterization of the final mixture without the need to do formal model selection and choose between all 2n -1 possible configurations. We derive conditions for consistency of the component selection process and illustrate the methodology on well-known datasets and simulations.