Abstract Details
Activity Number:
|
541
|
Type:
|
Contributed
|
Date/Time:
|
Wednesday, August 7, 2013 : 10:30 AM to 12:20 PM
|
Sponsor:
|
Section on Bayesian Statistical Science
|
Abstract - #308537 |
Title:
|
Challenges with the Use of Cross-Validation for Comparing Structured Models
|
Author(s):
|
Wei Wang*+ and Andrew Gelman
|
Companies:
|
and Columbia University
|
Keywords:
|
model comparison ;
cross-validation ;
Bayesian hierarchical models ;
survey methodology
|
Abstract:
|
As a simple and compelling approach for estimating out-of-sample predictive error, cross-validation naturally lends itself to the task of model comparison. However, we feel that the legitimacy of cross-validation methods in model comparison is often being taken for granted. In this work, we want to clarify what cross-validation methods are measuring when they are used for model comparison. Using a hierarchical model fit to large survey data with a battery of questions, we show that even though cross-validation might give good estimates of out-of-sample performance, it is not always a sensitive instrument for model comparison. In addition, we emphasize the importance of proper calibration in assisting us interpret the practical importance of the results when conducting model comparison.
|
Authors who are presenting talks have a * after their name.
Back to the full JSM 2013 program
|
2013 JSM Online Program Home
For information, contact jsm@amstat.org or phone (888) 231-3473.
If you have questions about the Continuing Education program, please contact the Education Department.
The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.
Copyright © American Statistical Association.