The Residual Bootstrap for High-Dimensional Linear Models with Low-Rank Designs

• Activity Number: 463
• Type: Contributed
• Date/Time: Wednesday, August 6, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Nonparametric Statistics
• Author(s): Miles Lopes*+
• Companies: University of California, Berkeley
• Keywords: bootstrap ; high-dimensional ; linear model ; regularization ; ridge regression ; resampling

Abstract

We study the residual bootstrap method in the context of the high-dimensional linear model, with fixed design. Specifically, we analyze the distributional approximation of linear contrasts of regression coefficients. When the contrasts are obtained from least squares or other M-estimation procedures, classical results show that the residual bootstrap is consistent, provided that p/n tends to 0. Up to now, relatively little work has considered how additional structure in the design matrix may extend the validity of the bootstrap to situations where p/n is of order 1. In this regime, we study contrast estimators obtained from ridge regression. Our main structural assumption on the design matrix is that it is nearly low rank -- in the sense that its singular values decay fairly quickly. Under a few extra technical assumptions, we show the residual bootstrap works for suitable contrasts. Our consistency results are non-asymptotic and are stated in terms of the Mallows (Wasserstein) metric. This approach allows us to prove consistency without assuming that contrasts approach limiting distribution.