Nonlinear Variable Selection via Deep Neural Networks

• Activity Number: 329 - SLDS Student Paper Awards
• Type: Topic Contributed
• Date/Time: Tuesday, July 30, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Yao Chen* and Qingyi Gao and Faming Liang and Xiao Wang
• Companies: Purdue University and Purdue University and Purdue University and Purdue University
• Keywords: Variable Selection; Neural Network

Abstract

This paper presents a general framework for high-dimensional nonlinear variable selection using deep neural networks under the framework of supervised learning. The network architecture includes both a selection layer and approximation layers. The problem can be cast as a sparsity-constrained optimization with a sparse parameter in the selection layer and other parameters in the approximation layers. This problem is challenging due to the sparse constraint and the nonconvex optimization. We propose a novel algorithm, called Deep Feature Selection, to estimate both the sparse parameter and the other parameters. Theoretically, we establish the algorithm convergence and the selection consistency when the objective function has a Generalized Stable Restricted Hessian. This result provides theoretical justifications of our method and generalizes known results for high-dimensional linear variable selection. Simulations and real data analysis are conducted to demonstrate the superior performance of our method.