Online Program Home
My Program

Abstract Details

Activity Number: 601 - Recent Advances in Variable Selection for Linear and Nonlinear Models
Type: Topic Contributed
Date/Time: Thursday, August 1, 2019 : 8:30 AM to 10:20 AM
Sponsor: Biometrics Section
Abstract #306489 Presentation
Title: Model Selection Bias Invalidates Goodness of Fit Tests
Author(s): Joshua Loftus*
Companies: New York University
Keywords: Selective inference; Model selection; Goodness of fit; Unbiased tests

We study goodness of fit tests in a variety of model selection settings and find that selection bias generally makes such tests conservative. Since selection methods choose the "best" model, a goodness of fit test will usually fail to reject, even when the incorrect model has been chosen. This is troubling, as it implies these tests in practice do not actually provide evidence in favor of the chosen model. We also explore methods of post selection inference to obtain conditionally unbiased goodness of fit tests and show how these outperform the unadjusted tests.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program