Fast Computation for Bayesian Variable Selection

• Activity Number: 123
• Type: Topic Contributed
• Date/Time: Monday, August 10, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Jin Wang* and Feng Liang and Yuan Ji
• Companies: University of Illinois at Urbana-Champaign and University of Illinois at Urbana-Champaign and The University of Chicago
• Keywords: Bayesian Variable Selection ; EM agorithm ; Bayesian Bootstrap

Abstract

A limitation of commonly used Bayesian approaches for variable selection is scalability: posterior inference replies on MCMC, which is time consuming and can hardly scale with large data. Motivated by a recent work by Rockova and George (2013), we derive an efficient EM algorithm which can retrieve the variable subset which achieves the highest posterior probability (MAP). An appealing feature of our EM algorithm is that we do not need to calculate the inverse of a large matrix in each iteration, which is unavoidable in many other algorithms, due to a computation trick which utilizes a special structure of our algorithm. We further propose an ensemble approach for variable selection based on Bayesian bootstrap: the main idea is to repeatedly apply a stochastic version of our EM algorithm on a subset of the data (i.e., the Bayesian bootstrap samples), and then aggregate variable selection results across those bootstrap experiments. Empirical studies have shown that the bootstrap-EM method is much faster and more accurate than the original EM, especially in the large p small n scenario.