294 – High-Dimensional Regression
A Focused Mean Squared Error Approach for Selecting Tuning Parameters in Penalized Regression
Kristoffer H. Hellton
University of Oslo
Penalized regression methods depending on one or more tuning parameters require fine-tuning to achieve optimal prediction performance. For ridge regression, which introduces an L2 penalty on the regression coefficients, a range of tuning procedures have been developed, but in practice K-fold cross-validation has become the standard procedure. This paper explores a focused tuning approach for ridge regression where the tuning parameter is made dependent on the covariates of the specific observation to be predicted. The observation-specific tuning parameter is defined as the minimand of the empirical mean square prediction error, obtained by plugging in pilot estimates of the regression coefficients and error variance in the theoretical mean squared error expressions. Several pilot estimates are proposed, and we present risk expressions for the case of an OLS pilot. The focused ridge estimator is compared to standard ridge regression fine-tuned by cross-validation in simulations and a real data set.