Online Program Home
My Program

Abstract Details

Activity Number: 554 - Deep Learning and Statistical Modeling with Applications
Type: Invited
Date/Time: Wednesday, August 1, 2018 : 2:00 PM to 3:50 PM
Sponsor: Biometrics Section
Abstract #326633 Presentation
Title: Cooperative Learning of Deep Energy-Based Model and Latent Variable Model via MCMC Teaching
Author(s): Ying Nian Wu*
Companies: UCLA
Keywords: Deep generative models; Unsupervised learning; Convolutional neural networks

We propose a cooperative learning algorithm to train both the deep energy-based model and the deep latent variable models jointly. The learning algorithm interweaves the maximum likelihood algorithms for learning the two models, and each iteration consists of the following two steps: (1) Modified contrastive divergence for energy-based model: The learning of the energy-based model is based on the contrastive divergence, but the finite-step MCMC sampling of the model is initialized from the synthesized examples generated by the latent variable model instead of being initialized from the observed examples. (2) MCMC teaching of the latent variable model: The learning of the latent variable model is based on how the MCMC in (1) changes the initial synthesized examples generated by the latent variable model, where the latent variables that generate the initial synthesized examples are virtually known so that the learning is essentially supervised. Our experiments show that the cooperative learning algorithm can learn highly realistic models of images. Based on joint work with Jianwen Xie, Yang Lu, Ruiqi Gao, and Song-Chun Zhu.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program