Statistical Inference When Fitting Simple Models to High-Dimensional Data

• Activity Number: 600
• Type: Contributed
• Date/Time: Wednesday, August 12, 2015 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): Lukas Steinberger* and Hannes Leeb
• Companies: University of Vienna and University of Vienna
• Keywords: High-dimensional models ; small sample size ; misspecified model ; regression analysis ; prediction ; F-test

Abstract

We study linear subset regression in the context of the high-dimensional overall model $y = \theta'z + u$ with univariate response $y$ and a $d$-vector of random regressors $z$, independent of $u$, where the sample size $n$ may be much less than $d$. We consider simple linear submodels where $y$ is regressed on a set of $p$ regressors given by $x = B'z$, for some $d \times p$ matrix $B$ with $p \leq n$. The corresponding simple model $y = \gamma' x + v$, can be justified by imposing restrictions on the unknown parameter $\theta$; otherwise, this simple model can be grossly misspecified. In this talk, we show that the OLS predictor obtained by fitting the simple linear model is typically close to the Bayes predictor $E[y|x]$, uniformly in $\theta\in{\mathbb R}^d$, provided only that $d$ is large. Moreover, we establish the asymptotic validity of the standard $F$-test on the surrogate parameter $\gamma$ which realizes the best linear fit of $x$ on $y$. On a technical level, we rely on recent results from Steinberger and Leeb (arXiv, 2014) on conditional moments of high dimensional random vectors given low dimensional projections; see also Leeb (AoS, 2013) and Hall and Li (AoS, 1993).