This is the program for the 2010 Joint Statistical Meetings in Vancouver, British Columbia.
Abstract Details
Activity Number:
|
298
|
Type:
|
Contributed
|
Date/Time:
|
Tuesday, August 3, 2010 : 8:30 AM to 10:20 AM
|
Sponsor:
|
Business and Economic Statistics Section
|
Abstract - #307470 |
Title:
|
Limit Theory for Comparing Overfit Models Out-of-Sample
|
Author(s):
|
Gray Calhoun*+
|
Companies:
|
Iowa State University
|
Address:
|
, , ,
|
Keywords:
|
forecasting ;
dimension asymptotics ;
forecast evaluation ;
model comparison ;
hypothesis test
|
Abstract:
|
This paper studies the rationale for comparing overfit forecasts out-of-sample instead of in-sample. The two models to be compared are nested linear regressions and the number of predictors used by the larger model, k, increases with the number of observations, T , so that the ratio k/T remains uniformly positive. Under this limit theory, tests that are designed to reject if the larger model is true, such as the usual in-sample Wald and LM test and also popular out-of-sample statistics, will choose the larger model too often when the smaller model is more accurate. We show that the usual out-of-sample test using Gaussian critical values is valid in the following sense: it will incorrectly choose the larger model with probability equal to the test's nominal size. We also show that the out-of-sample period must be small relative to the sample size when the models are overfit.
|
The address information is for the authors that have a + after their name.
Authors who are presenting talks have a * after their name.
Back to the full JSM 2010 program
|
2010 JSM Online Program Home
For information, contact jsm@amstat.org or phone (888) 231-3473.
If you have questions about the Continuing Education program, please contact the Education Department.