Online Program Home
My Program

Abstract Details

Activity Number: 622
Type: Contributed
Date/Time: Wednesday, August 3, 2016 : 2:00 PM to 3:50 PM
Sponsor: Section on Nonparametric Statistics
Abstract #320227 View Presentation
Title: Model Selection Probabilities
Author(s): Xin Lu Tan* and Andreas Buja and Lawrence D. Brown and Abba Krieger and Zongming Ma
Companies: and The Wharton School and University of Pennsylvania and University of Pennsylvania and University of Pennsylvania
Keywords: asymptotic normality ; bootstrap ; model selection ; stability ; U-statistic ; V-statistic

We consider the estimation of true model/variable selection probabilities in the context of regression. We show, through a simple slope test example, that bootstrap fails in estimating model selection probabilities. Indeed, we establish a rigorous impossibility result that no method is able to consistently estimate model selection probabilities for data of size n from a dataset of the same size. We then show that the m-out-of-n bootstrap can consistently estimate selection probabilities for data of size m = o(n) with a sample size n. We establish the asymptotic normality of the m-out-of-n bootstrap estimator, allowing m to grow with n, and provide a consistent estimator for its asymptotic variance. This leads to asymptotically valid confidence intervals for selection probabilities associated with data of size m. We examine how true model selection probabilities change with sample sizes for several popular model selection methods on simulated data examples. Some of these examples illustrate the impossibility of extrapolating from small values of m to the actual sample size n, which agrees with our impossibility result.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

Copyright © American Statistical Association