A Split-and-Merge Bayesian Variable Selection Approach for Ultra-High-Dimensional Regression

• Activity Number: 596
• Type: Topic Contributed
• Date/Time: Thursday, August 7, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Biometrics Section
• Author(s): Faming Liang*+ and Qifan Song
• Companies: Texas A&M and Texas A&M
• Keywords: Big Data ; Markov chain Monte Carlo ; Variable Selection

Abstract

This talk presents a Bayesian variable selection approach for ultra-high dimensional linear regression based on the strategy of split-and-merge. The proposed approach consists of two stages: (i) split the ultra-high dimensional dataset into a number of lower dimensional subsets and select relevant variables from each of the subsets, and (ii) aggregate the variables selected from each sub- set and then select relevant variables from the aggregated dataset. Since the proposed approach has an embarrassingly parallel structure, it can be easily implemented in a parallel architecture and applied to big data problems with millions or more of explanatory variables. Under mild conditions, we show that the proposed approach is consistent; that is, the true explanatory variables can be correctly identified by the proposed approach as the sample size becomes large. Extensive comparisons of the proposed approach have been made with the penalized likelihood approaches, such as Lasso, elastic net, SIS and ISIS. The numerical results show that the proposed approach generally outperforms the penalized likelihood approaches: The models selected by the proposed approach tend to be more sparse and cl