Bayesian Regularization via the Graph Laplacian Prior

• Activity Number: 298
• Type: Contributed
• Date/Time: Tuesday, August 2, 2011 : 8:30 AM to 10:20 AM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Sounak Chakraborty*+ and Fei Liu and Fan Li and Yan Liu
• Companies: University of Missouri at Columbia and IBM Thomas J. Watson Research Center and Duke University and University of Southern California at Los Angles
• Keywords: Bayesian ; Elastic Net ; Graph Laplacian ; Grouping ; Ridge regression ; Lasso

Abstract

Regularization is an important approach to prevent overfitting in regression. Under the Bayesian paradigm, many regularization techniques correspond to imposing certain shrinkage prior distributions on the regression coefficients. Existing Bayesian methods usually assume independence between explanatory variables a priori. In this article, we propose a novel Bayesian approach, which explicitly models the dependence structure between variables through a graph Laplacian matrix. We generalize the graph Laplacian to allow both positive and negative correlations. A prior distribution for the graph Laplacian is then proposed, which allows conjugacy and thereby greatly simplifies the computation. We show that the proposed Bayesian model leads to proper posterior distribution. Connection is made between the proposed method and some existing regularization approaches, such as the Lasso, the Elastic Net, and ridge regression. An efficient MCMC method based on parameter augmentation is developed for posterior computation. Finally, we demonstrate the method through simulation studies and a real data analysis.