Applications of Bregman Divergence Measures in Sparse High-Dimensional Problems

• Activity Number: 414
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Gyuhyeong Goh*+ and Dipak Dey
• Companies: University of Connecticut and University of Connecticut
• Keywords: Bayesian Lasso ; Bregman divergence ; Lasso ; Total Bregman divergence

Abstract

For sparse high-dimensional data modeling, the Lasso (Tibshirani, 1996) is the most popular technique. Since the Lasso method induces sparse solutions of parameters, the estimation procedure automatically reduces the dimension of the parameter space. In similar spirit, Park and Casella (2008) developed Bayesian Lasso using hierarchical Bayesian modeling with the Laplace prior on the coefficient parameter. Although the aforementioned two methods are theoretically very attractive, they involve some drawbacks in practice. The Lasso requires a deterministic tuning parameter prior in implementing parameter estimation. The Bayesian Lasso is computationally intensive and slow. In this study, we introduce a new version of Lasso that is developed by using the Bregman divergence and the Total Bregman divergence with certain convex functions in a Bayesian framework. Since the Bregman divergence and the Total Bregman divergence induce smooth (differentiable) loss functions, all parameter estimates including the tuning parameter can be easily obtained by a simple algorithm derived from differentiating the full posterior density function.