Incompatibility in Gibbs Samplers

• Activity Number: 140
• Type: Topic Contributed
• Date/Time: Monday, August 12, 2002 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing*
• Author(s): David van Dyk*+ and Xiao-Li Meng and Hosung Kang
• Affiliation(s): Harvard University and Harvard University and Harvard University
• Keywords: Generalized Linear Mixed Model ; Gibbs Sampler ; Incompatibilty ; Marginal Augmentation ; Rate of Convergence

Abstract

The conditional distributions $p(x|y)$ and $p(y|x)$ are said to be compatible if there exists a joint distribution, $p(x,y)$, such that $p(x|y)=p(x,y)/\int p(x,y)dx$ and $p(y|x)=p(x,y)/\int p(x,y)dy$. In this case, a Gibbs sampler can be constructed by iteratively sampling from the two conditional distributions, and under certain regularity conditions, the stationary distribution of the resulting Markov chain is $p(x,y)$. Much less is known about the behavior of a Gibbs sampler that is constructed using incompatible conditional distributions. In this talk, we explore this possibility and show that careful choice of such incompatible distribution can lead to a Gibbs sampler with known stationary distribution and with a better geometric rate of convergence than the standard sampler. The method of marginal data augmentation gives a prescription for such a choice. We illustrate our methods using the Generalized Linear Mixed Model and demonstrate the improved rate of convergence of the sampler, some of its unexpected behavior, and warnings to potential users.