Large-Scale Kernel Belief Propagation for Nonparametric Graphical Model

• Activity Number: 288
• Type: Topic Contributed
• Date/Time: Tuesday, August 2, 2011 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Computing
• Author(s): Le Song*+
• Companies: Carnegie Mellon University

Abstract

Belief propagation is an inference algorithm for graphical models that has been widely and successfully applied in a great variety of domains. We propose a nonparametric generalization of belief propagation, Kernel Belief Propagation (KBP), for pairwise Markov random fields: messages are represented as functions in a reproducing kernel Hilbert space (RKHS), and message updates are simple linear operations in the RKHS. KBP makes none of the assumptions commonly required in classical BP algorithms: the variables need not arise from a finite domain or a Gaussian distribution, nor must their relations take any particular parametric form. Rather, the relations between variables are represented implicitly, and are learned nonparametrically from training data. KBP has the advantage that it may be used on any domain where kernels are defined (R^d, strings, groups), even where explicit parametric models are not known, or closed form expressions for the BP updates do not exist. The computational cost of message updates in KBP is polynomial in the training data size. We also propose a constant time approximate message update procedure by representing messages using a small number of basis functions. We experiment with a parallel implementation of KBP for image denoising, image to depth prediction, and protein structure prediction problem: KBP is faster than competing classical and nonparametric approaches (by orders of magnitude, in some cases), while providing significantly more accurate results.