Inferring Parameters and Structure of Markov Random Field Models

• Activity Number: 38
• Type: Invited
• Date/Time: Sunday, August 7, 2005 : 4:00 PM to 5:50 PM
• Sponsor: IMS
• Author(s): Max Welling*+
• Companies: University of California, Irvine
• Keywords: Model Inference ; Markov Random Field ; Bayesian Inference

Abstract

Model estimation for Markov Random Fields is typically intractable due to the presence of a global normalization constant. A number of methods are available that optimize approximate objective measures such as the pseudolikelihood, the mean field free energy, and the contrastive divergence. To compare these methods, we have drawn perfect samples from binary square lattices using "forward filtering backward sampling" on junction trees and binary fully connected MRFs with positive interactions using "coupling from the past." We investigate properties such as asymptotic normality, variance, and bias as a function of interaction strengths. We also test how the methods perform for small sample sizes. As a second contribution, we propose a novel approximate method to infer the structure of MRF models. Our approach is based on a second-order (Laplace) approximation of the marginal likelihood. The Hessian of the log-normalization constant is equal to the covariance matrix between all vertices in the MRF. To approximate this quantity, we use a technique based on "belief propagation" on cyclic graphs.