Simultaneous Sparse Estimation of Canonical Vectors in the P>>N Setting

• Activity Number: 368
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Irina Gaynanova*+ and James Booth and Martin T Wells
• Companies: Cornell University and Cornell University and Cornell University
• Keywords: Block-coordinate descent ; Canonical vectors ; Discriminant analysis ; Group Penalization

Abstract

This work considers the problem of sparse estimation of canonical vectors in linear discriminant analysis when p>>n. Several methods have been proposed in the literature that estimate one canonical vector in the two group case. However, G-1 canonical vectors can be considered if the number of groups is G. In multi-group context, it is common to estimate canonical vectors in a sequential fashion. Moreover, separate prior estimation of covariance structure is often required. In contrast, the proposed method estimates all canonical vectors directly. First, we show that in the n>p setting the canonical vectors can be expressed in a closed form up to an orthogonal transformation. Secondly, we extend this form to the p>>n setting and propose to achieve feature selection using a group penalty. The resulting optimization problem is convex and can be solved using block-coordinate descent. The performance of the method is evaluated through simulation studies as well as real data applications. The results suggest that the proposed classifier performs favorably in comparison to alternative methods.