Naturally Efficient Sparsity Tuner for Kernel Regression

• Activity Number: 341
• Type: Contributed
• Date/Time: Tuesday, August 2, 2011 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Ernest Fokoue*+
• Companies: Rochester Institute of Technology
• Keywords: Regression ; Information Matrix ; Sparsity ; Structured Prior Matrix ; Kernel ; Support Points

Abstract

We propose a novel approach to achieving sparse representation in kernel regression through a straightforward algorithm that consists in a refinement of the maximum a posteriori (MAP) estimator of the weights of the kernel expansion. Our proposed method combines structured prior matrices and functions of the information matrix to zero in on a very sparse representation. We show computationally that our naturally efficient sparsity tuner (NEST) achieves a very sparse and predictively accurate estimator of the underlying function, for a variety of choices of the covariance matrix of our Gaussian prior over the weights of the kernel expansion. Our computational comparisons on both artificial and real examples show that our method compete very well - usually favorably - with the Support Vector Machine, the Relevance Vector Machine and Gaussian Process regressors.