Sparse Bayes Learning by Annealing Entropy

• Activity Number: 304
• Type: Topic Contributed
• Date/Time: Tuesday, August 4, 2009 : 10:30 AM to 12:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Ryo Yoshida*+ and Mike West
• Companies: Institute of Statistical Mathematics and Duke University
• Keywords: Latent variable models ; Annealing ; Graphical model ; Gene expression data ; Bayesian computation ; Bayesian computation

Abstract

We bring a novel Bayesian computing to find sparse estimates for a range of statistical models. The problem of sparsity identification requires substantial efforts necessary to solve a hard combinatorial optimization involved in a large configuration space of sparsity patterns. The essence of our approach is the maximum a posteriori (MAP) with regard to sparsity configurations and model parameters. To realize an efficient MAP computing, we impose an artificial regularizer on the posterior entropy of sparsity configurations where the degree of regularization is dynamically controlled by a meta parameter called temperature. Our algorithm prescribes a schedule for lowering temperature so as to decay slowly and reach to zero, and then proceeds with iterative-optimization over sparsity configurations and parameters while keeping on the cooling schedule.