Discriminant Analysis for High-Dimensional Problems

• Activity Number: 600
• Type: Invited
• Date/Time: Thursday, August 4, 2011 : 8:30 AM to 10:20 AM
• Sponsor: Biometrics Section
• Author(s): Daniela Witten*+ and Robert Tibshirani
• Companies: University of Washington and Stanford University
• Keywords: classification ; lda ; regularization ; lasso ; sparsity

Abstract

We consider the classification setting, in which the data consist of p features measured on n observations, each of which belongs to one of K classes. Linear discriminant analysis (LDA) is a classical method for this problem, and it follows from three distinct viewpoints: maximum likelihood, optimal scoring, and Fisher's discriminant problem. In the high-dimensional setting where p>>n, LDA is not appropriate for two reasons. First, the standard estimate for the within-class covariance matrix is singular, and so the usual discriminant rule cannot be applied. Second, when p is large, it is difficult to interpret the classification rule obtained from LDA, since it involves all p features. A number of proposals have been made in the literature for solving this problem, and have centered around adapting the maximum likelihood and optimal scoring problems to the high-dimensional setting using regularization approaches. Fisher's discriminant problem has been largely overlooked because when penalties are applied, the problem is highly non-convex. We use a minorization algorithm to overcome this obstacle, and show that the resulting classifier is interpretable and accurate.