Variable Selection in Convex Regression

• Activity Number: 202
• Type: Invited
• Date/Time: Monday, August 4, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Section on Nonparametric Statistics
• Author(s): John Lafferty*+ and Min Xu and Minhua Chen
• Companies: University of Chicago and Carnegie Mellon and University of Chicago
• Keywords: convex regression ; shape-constrained inference ; variable selection ; sparsity

Abstract

We consider the problem of estimating a convex function of several variables from noisy values of the function at a finite sample of input points. Recent work of Guntuboyina (2012) shows that the minimax rate for estimating the support function of a convex set in $d$ dimensions is $n^{-4/(3+d)}$, where $n$ is the number of noisy measurements. The equivalent rate of $n^{-4/(4+d)}$ is conjectured, but not yet fully proved, for convex regression. The geometric convexity constraint is thus statistically equivalent to requiring two derivatives of the function, and is subject to the same curse of dimensionality. However, if the function is sparse, with $s\ll d$ relevant variables, then the faster rate $n^{-4/(4+s)}$ may be achievable if the $s$ variables can be identified. We develop a screening procedure to identify irrelevant variables. Our approach adopts on a two-stage method that estimates a sum of $p$ one-dimensional convex functions, followed by one-dimensional concave regression fits on the residuals. The method is based on quadratic programming, and in contrast to standard sparse additive models, requires no tuning parameters for smoothness.