Two Novel Applications of Selective Inference

• Activity Number: 50
• Type: Invited
• Date/Time: Sunday, August 9, 2015 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Sam Gross and Stephen Reid and Rob Tibshirani*
• Companies: Stanford University and Stanford University and Stanford University
• Keywords: principal components ; hypothesis testing ; p-values ; selective inference

Abstract

I will discuss two applications of the recently developed theory of selective inference. The first application is to high dimensional regression and testing problems. The idea is pre-cluster the features and extract a prototype from each cluster. Then we run the lasso or a multiple testing procedure on the prototypes. Using the theory of selective inference we derive exact p-values that account for the prototype selection, and any further selection in the model building. The second application is for the problem of comparing a data-derived predictor to an external one. Tibshirani and Efron (2007) proposed the "pre-validation'' method for this problem. But this method does not guarantee proper type I error control. Using selective inference theory we derive an exact procedure for this problem. The first part is joint with Stephen Reid; the second part is joint with Sam Gross and Jonathan Taylor.