|
Abstract:
|
There has been an intense development on the estimation of a sparse regression coeffcient,including penalized likelihood and the Bayesian methods. We focus on the the Bayesian methods by putting a spike-and-slab prior with a mixture of a normal distribution and a point mass at 0 to the coefficients. Carbonetto and Stephens (2012) proposed a variational Bayesian algorithm which sequentially updates the parameters associated with each coefficient given all others. We show that such a component-wise updating scheme is prone to error accumulation especially when the dimension is large and the predictors are correlated. We propose a new variational algorithm whose updating is simultaneously for all dimensions, namely the batch-wise updating scheme. We show that the new algorithm achieves both frequentist consistency and Bayesian consistency even when the feature dimension diverges with the sample size after we run the algorithm just once. The simulation results show that the batch-wise algorithm outperforms the component-wise algorithm, and LASSO and ridge regression in some situations.
|
