High-Dimensional Quadratic Discriminant Analysis: A Convex Optimization Approach

• Activity Number: 367
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Lucy Xia*+ and Tracy Ke and Jianqing Fan
• Companies: Princeton University and Princeton University and Princeton University
• Keywords: Quadratic Discriminant Analysis ; High Dimension ; Classification ; Convex Optimization

Abstract

High dimensional quadratic discriminant analysis (QDA) is challenging mainly because the estimations of inverses of covariance matrices are usually unstable and inaccurate. Therefore, regularized quadratic rules were proposed in literature, for example, QDA with plug-in sparse estimates of covariance matrices, or regularized discriminant rules (RDA). However, these methods either require special structures on covariance matrices to get good estimates, or are computationally prohibitive in high dimensional settings.

We propose a new method for computing quadratic rules through convex programming. Our method does not need estimates of inverses of covariance matrices, and hence is stable. Computationally, the method can be equivalently formulated as a Lasso problem with linear equality constraints, which can be efficiently solved. Under the two-sample elliptical distribution settings, we show that our method achieves nearly optimal rates on classification errors.

Numerical studies support our methods. Under a variety of settings, it is highly competitive and sometimes outperforms RDA, plug-in QDA and various LDA methods.