Regression Model Stochastic Search via Local Orthogonalization

• Activity Number: 526
• Type: Contributed
• Date/Time: Wednesday, August 3, 2011 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Computing
• Author(s): Ruoxi Xu*+ and Chris Hans
• Companies: The Ohio State University and The Ohio State University
• Keywords: Model Uncertainty ; Gibbs ; Metropolis Hastings ; Othonomral Rotation ; Multicollinearity ; Point-Mass Prior

Abstract

Bayesian model uncertainty problems are often challenged by high dimensional model spaces. When the number of predictors is large, it is often infeasible to enumerate the model space, which makes Markov chain Monte Carlo (MCMC) methods such as the Gibbs sampler popular among practitioners. A common problem with the Gibbs sampler is its potential to get stuck in local regions of the model space when predictors are highly correlated. Motivated by the need to explore the model space efficiently when high multicollinearity presents, we introduce a Metropolis-Hastings-Based algorithm with an orthonormal rotation on the regression coefficients. The orthonormal rotation is based on the spectrum decomposition of the covariance matrix and is shown to facilitate MCMC updates in directions along which the chain is less likely to stay in local regions. We demonstrate the effectiveness of the resulting sampling algorithm over other popular sampling methods on a real data set with a large number of highly correlated predictors.