Sparse Machine Learning Methods for Regression with Regularized Tensor Product Kernel

• Activity Number: 173 - Recent Advances in Statistical Learning and Missing Data Handling
• Type: Contributed
• Date/Time: Tuesday, August 4, 2020 : 10:00 AM to 2:00 PM
• Sponsor: Korean International Statistical Society
• Author(s): Hang Yu* and Yuanjia Wang and Donglin Zeng
• Companies: University of North Carolina at Chapel Hill and Columbia University and University of North Carolina at Chapel Hill
• Keywords: Alternating direction method of multipliers; Fisher consistency; Reproducing kernel Hilbert space; Oracle property; Tensor product

Abstract

Feature selection continues to be an important and challenging problem in the modern era of large scale data. Most of existing methods for feature selection are based on either parametric or semiparametric models, so the resulting performance can severely suffer from model misspecification when high-order nonlinear interactions among the features are present. A limited number of approaches for nonparametric feature selection were proposed, but they are computationally intensive and may not even converge. In this paper, we propose a novel and computationally efficient approach for nonparametric feature selection in regression field based on a tensor-product kernel function over the feature space. The importance of each feature is governed by a parameter in the kernel function which can be efficiently computed iteratively from a modified alternating direction method of multipliers (ADMM) algorithm. We prove the oracle selection property of the proposed method. Finally, we demonstrate the superior performance of our approach compared to existing methods via simulation studies and an application to prediction of Alzheimer's disease.