Variable Selection for Discriminant Analysis with Quadratic Multinomial Regression

• Activity Number: 609
• Type: Contributed
• Date/Time: Wednesday, August 12, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Biometrics Section
• Author(s): Yang Li* and Jun S. Liu
• Companies: Harvard University and Harvard University
• Keywords: Discriminant analysis ; Bayesian information criterion ; Variable selection ; Classification

Abstract

Variable selection for classification methods is necessary when there are many predictor variables. Recently three articles proposed discriminant analysis variable selection criteria based on the Bayesian information criterion (BIC) of full likelihood function. The proposed criteria are proved to be consistent under assumption that all predictors including the irrelevant ones follow multivariate normal distribution in each class. However, when this assumption does not hold, there will easily be false positives selected and the classification accuracy will be compromised. In this article, we propose a BIC score based on the conditional likelihood of discriminant analysis. We show that no distribution assumption for the irrelevant variable is required for the its criterion consistency. We propose a forward-backward procedure based on this new BIC score, and studied the asymptotic properties of the criterion and procedure under high-dimensional setting. We compare the proposed procedure with existing methods on simulated and real data sets. It is demonstrated that our method usually selects less predictors but achieves higher classification accuracy.