Tree-based Extrapolation Diagnostics for Machine Learning

• Activity Number: 22
• Type: Contributed
• Date/Time: Sunday, August 8, 2004 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): Giles Hooker*+
• Companies: Stanford University
• Keywords: CART ; diagnostics ; extrapolation ; tree ; machine learning ; nonparametric

Abstract

Machine-learning applications typically occur in very high-dimensional contexts and usually involve datasets of highly correlated predictors. This results in the hypercube bounding the data containing large unpopulated regions. Moreover, many learning procedures are highly flexible, meaning that their behavior is uncontrolled in these regions. This is a problem when the distribution of predictor variables shifts. It is also a concern for functional diagnostics which often require the function to be evaluated on a product measure. We present a diagnostic for extrapolation as a test statistic for a point originating from the data distribution against a uniform null hypothesis. This allows us to employ general classification methods to estimate this statistic. Further, we observe that CART can be given an exact distribution as an argument to provide a more stable estimate. This is the basis of our extrapolation-detection procedure. We explore some advantages of this approach and present examples of it working in practice.