Online Program Home
My Program

Abstract Details

Activity Number: 248
Type: Contributed
Date/Time: Monday, August 1, 2016 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #320510 View Presentation
Title: A Variational Bayesian Algorithm for Variable Selection
Author(s): Xichen Huang* and Jin Wang and Feng Liang
Companies: and University of Illinois and University of Illinois at Urbana-Champaign
Keywords: variable selection ; variational inference ; spike-and-slab prior ; batch-wise updating ; selection consistency

There has been an intense development on the estimation of a sparse regression coeffcient,including penalized likelihood and the Bayesian methods. We focus on the the Bayesian methods by putting a spike-and-slab prior with a mixture of a normal distribution and a point mass at 0 to the coefficients. Carbonetto and Stephens (2012) proposed a variational Bayesian algorithm which sequentially updates the parameters associated with each coefficient given all others. We show that such a component-wise updating scheme is prone to error accumulation especially when the dimension is large and the predictors are correlated. We propose a new variational algorithm whose updating is simultaneously for all dimensions, namely the batch-wise updating scheme. We show that the new algorithm achieves both frequentist consistency and Bayesian consistency even when the feature dimension diverges with the sample size after we run the algorithm just once. The simulation results show that the batch-wise algorithm outperforms the component-wise algorithm, and LASSO and ridge regression in some situations.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

Copyright © American Statistical Association