Bayesian Trees Are Spatially Adaptive

• Activity Number: 292 - Nonparametric and High-Dimensional Bayes: Uncertainty Quantification, Computation, and Posterior Contraction
• Type: Invited
• Date/Time: Wednesday, August 5, 2020 : 10:00 AM to 11:50 AM
• Sponsor: IMS
• Author(s): Veronika Rockova* and Judith Rousseau
• Companies: University of Chicago and Oxford University
• Keywords: Bayesian Trees; Spatial Inhomogeneity; Uncertainty Quantification

Abstract

This paper addresses the following question:'Can regression trees do what other machine learning methods cannot?' To answer this question, we consider the problem of estimating regression functions with spatial inhomogeneities. Many real life applications involve functions that exhibit a variety of shapes including jump discontinuities or high-frequency oscillations. Unfortunately, the overwhelming majority of existing asymptotic minimaxity theory (for density or regression function estimation) is predicated on homogeneous smoothness assumptions which are inadequate for such data. Focusing on locally Holder functions, we provide locally adaptive posterior concentration rate results under the supremum loss. These results certify that trees can adapt to local smoothness by uniformly achieving the point-wise (near) minimax rate. Going further, we construct locally adaptive credible bands whose width depends on local smoothness and which achieve uniform coverage under local self-similarity. To highlight the benefits of trees, we show that Gaussian processes cannot adapt to local smoothness by showing lower bound results under a global estimation loss.