|
Abstract:
|
As datasets continue to grow in size, in many settings the focus of data collection has shifted away from testing pre-specified hypotheses, and towards hypothesis generation. Researchers are often interested in performing an exploratory data analysis in order to generate hypotheses, and then testing those hypotheses on the same data; I will refer to this as 'double dipping'. Unfortunately, double dipping can lead to highly-inflated Type 1 errors. In this talk, I will consider the special cases of hierarchical clustering and CART decision trees. First, I will show that sample-splitting does not solve the 'double dipping' problem for clustering. Then, I will propose a test for a difference in means between estimated clusters that accounts for the cluster estimation process, using a selective inference framework. Finally, I will show that a similar approach can be applied to test hypotheses related to a fitted CART decision tree. This is joint work with Lucy Gao (University of Waterloo), Anna Neufeld (University of Washington), and Jacob Bien (University of Southern California).
|
