|
Abstract:
|
In high dimensional (p>n) regressions where the predictors are correlated, shrinkage regression techniques such as ridge regression (RR) and principal component regression (PCR) are widely used and the LASSO estimate is known to be unstable. Let X=UDW' denote the singular value decomposition of the design matrix where the singular values are in decreasing order, RR and PCR shrink the least squared estimates of the regression coefficients according to their corresponding singular values. When components with larger singular values are highly correlated with the responses, RR and PCR reduce prediction risk compared to the least squared estimates. On the other hand, when components with smaller singular values are highly correlated with the responses, RR and PCR do not reduce prediction risk effectively . We show that the horseshoe regression, recently introduced by Polson and Scott (2012), can solve these problems with shrinkage regression. We formally demonstrate this by computing Stein's estimators of prediction risk for HS, RR and PCR. HS performs better in both Stein's risk and actual out of sample prediction error in our simulations.
|
