Stan: HMC and Nuts for Hierarchical Modeling

• Activity Number: 586
• Type: Invited
• Date/Time: Thursday, August 7, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Computing
• Author(s): Bob Carpenter and Ben Goodrich*+
• Companies: Columbia University and Columbia University
• Keywords: Stan ; HMC ; MCMC ; hierarchical

Abstract

Stan is a general-purpose probabilistic programming language that was motivated in large part by hierarchical models. In this talk, I will discuss how Stan can be used for flexible hierarchical modeling and model fitting and inference with MCMC sampling or optimization-based point estimation.

Hierarchical models present posterior densities in which the scale of lower-level parameters varies with the hierarchical parameters. This presents challenges for Gibbs sampling, random-walk Metropolis, and Hamiltonian Monte Carlo. For Gibbs, the problems involve the posterior correlation; for Metropolis and HMC, the problem is the lack of a global scale to adapt to.

I will compare the efficiency of HMC (as implemented in Stan) with Gibbs (as implemented in JAGS and BUGS) on a range of hierarchical modeling problems, in terms of both theory and practical inference. I will also discuss strategies to ensure the posterior samples are not biased by the hierarchical structure of the problem and its varying posterior scale.

(Work on Stan was done jointly by the Stan development team; an extensive study of hierarchical models was carried out by Michael Betancourt and Peter Li.)