Cloud Computing for Bayesian Hidden Markov Models

• Activity Number: 5
• Type: Invited
• Date/Time: Sunday, August 1, 2010 : 2:00 PM to 3:50 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Steven Lee Scott*+
• Companies: Google
• Keywords: Markov chain Monte Carlo ; cloud computing ; parallel computation

Abstract

Hidden Markov models (HMM's) are a useful way to model streams of data produced by groups of subjects. Bayesian inference for HMM's typically involves data augmentation implemented using a forward-backward sampling procedure. The output of each iteration is a set of complete data sufficient statistics that are used in a draw of model parameters. With large data set the data augmentation step is many orders of magnitude more expensive than the parameter draw, and so is a natural candidate for parallelization. This talk describes a parallel implementation of a Bayesian HMM in a "cloud" with many thousands of processors. We describe the gains in speed experienced when computing with very large data sets, as well as the software design and programming decisions faced when computing in a very large cluster.