Supervised Classification Using Sparse Fisher's LDA

• Activity Number: 338
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Irina Gaynanova*+
• Companies: Cornell University
• Keywords: supervised classification ; discriminant analysis ; gradient descent ; methylation

Abstract

It is well known that in a supervised classification setting when the number of features, p, is smaller than the number of observations, n, Fisher's linear discriminant rule asymptotically achieves Bayes rule. However, there are numerous applications where classification is needed in p>>n setting. Naive implementation of Fisher's rule in this setting fails to provide good results because the sample covariance matrix is singular. Moreover, by constructing a classifier that relies on all p features the interpretation of the results is challenging. Our goal is to provide robust classification that relies only on a small subset of features and accounts for the complex correlation structure. We propose to apply L1 penalty to the discriminant vector to insure sparsity of the solution and use a shrinkage type estimator for the covariance matrix. The resulted optimization problem is solved using an iterative coordinate descent algorithm. The simulation results show that the proposed method performs favorably in comparison to alternatives. The method is used to classify 344 leukemia patients based on 19000 DNA methylation features.