Discrepancy Pursuit: A Nonparametric Framework for High-Dimensional Variable Selection

• Activity Number: 166
• Type: Topic Contributed
• Date/Time: Monday, August 5, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Li Liu*+ and Kathryn Roeder and Han Liu
• Companies: Carnegie Mellon University and CMU and Princeton University
• Keywords: nonparametric inference ; variable selection ; homogeneity test ; reproducing kernel Hilbert space ; maximum mean discrepancy ;

Abstract

Existing high dimensional variable selection methods are either based on parametric linear model or semiparametric additive model assumptions. In this paper we propose a completely nonparametric framework for high dimensional variable selection. The key idea is to reduce the variable selection problem into a subset pursuit problem which aims to find the subset of variables that maximizes the discrepancy criterion of their corresponding distributions across different groups. By embedding the distribution into a reproducing kernel Hilbert space (RKHS), such a framework can be implemented using the maximum mean discrepancy (MMD) criterion combined with a backward greedy pursuit procedure. The main advantages of such a framework are: (i) easily incorporating structural information such that different types of dependence can be captured; (ii) naturally handling non-numeric data; and (iii) avoiding strong distributional assumptions. Theoretically, we can show that even though the model is nonparametric, the estimation possesses a parametric rate.