A Classification Rule of Feature Augmentation via Nonparametrics and Selection (FANS) in High-Dimensional Space

• Activity Number: 209
• Type: Invited
• Date/Time: Monday, August 5, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Jianqing Fan and Yang Feng*+ and Xin Tong
• Companies: Princeton University and Columbia University and MIT
• Keywords: nonparametric ; classification ; feature augmentation ; parallel computing ; high-dimensional

Abstract

In this paper, we propose a new classification rule that involves feature augmentation via nonparametrics and selection (FANS) for high dimensional problems, where the number of features $p$ is comparable to or larger than the sample size $n$. FANS follows a two-step procedure. In the first step, marginal class conditional densities are estimated nonparametrically. In the second step, we invoke penalized logistic regression, taking as input features the estimated log ratios of the marginal class conditional densities. We motivate FANS by generalizing the Naive Bayes model, writing the log ratios of class conditional densities as a linear combination of log ratios of marginal class conditional densities. An oracle inequality regarding the risk is developed for FANS. In numerical analysis, we compare FANS to other competing methods, so as to provide a guideline about its best application domain. Real data analysis demonstrates that FANS performs very competitively on benchmark email spam and gene expression data sets. Moreover, FANS is implemented by an extremely fast algorithm through parallel computing.