L1 Regularized Linear Discriminant Analysis

• Activity Number: 81
• Type: Contributed
• Date/Time: Sunday, August 1, 2010 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Qing Mai*+ and Hui Zou and Ming Yuan
• Companies: University of Minnesota and University of Minnesota and Georgia Institute of Technology
• Keywords: Linear discriminant analysis ; Feature selection ; L1 penalty ; Consistency

Abstract

Linear discriminant analysis (LDA) has long been a popular classification tool. We study the feature selection problem in LDA with high dimensions. We first carefully distinguish interesting features and informative features. In general, interesting features often discovered by multiple testing procedures are not necessarily the informative features that define the Bayes rule. In order to eliminate uninformative features and produce an accurate classifier simultaneously, we propose a novel L1 regularized LDA classifier which is shown to be very efficient computationally. We establish asymptotic properties of the L1 regularized LDA. The algorithm proposed is examined by simulations and on real data.