Inference After Model Selection Using Restricted Permutation Methods

• Activity Number: 517
• Type: Invited
• Date/Time: Thursday, August 2, 2007 : 10:30 AM to 12:20 PM
• Sponsor: Biometrics Section
• Author(s): Rui Wang*+ and Stephen Lagakos
• Companies: Harvard School of Public Health and Harvard School of Public Health
• Keywords: automated variable selector ; covariates ; regression ; training and validation data sets

Abstract

Many statistical problems first apply a model-selection algorithm to identify a subset of candidate covariates, and then make inferences about the association between the selected covariates and the response. It is well-known that ignoring the fact that the covariates were selected on the basis of their apparent association with the response can lead to biased inference. We develop inference methods based on partitioning the response-covariate data matrix in specific ways for purposes of generating permutations, depending on the (arbitrary) model selection procedure and correlation structure of the response-covariate data matrix, and then basing inferences on a restricted set of permutations. We illustrated how the proposed methods correct biases from model selection, their efficiency relative to use of an independent validation set, and their robustness to violations of assumptions.