An Exposition on the Propriety of Restricted Boltzmann Machines

• Activity Number: 74
• Type: Contributed
• Date/Time: Sunday, July 31, 2016 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Andrea Kaplan* and Daniel Nordman and Stephen Vardeman
• Companies: Iowa State University and Iowa State University and Iowa State University
• Keywords: undirected graphical models ; model degeneracy ; deep learning ; Boltzmann machines ; fitting highly flexible models

Abstract

A restricted Boltzmann machine (RBM) is an undirected graphical model constructed for discrete random variables, specified as having two layers, a hidden and a visible layer with no conditional dependency within a layer. In recent years, RBMs have risen to prominence due to their connection to deep learning - by treating a hidden layer of one RBM as the visible layer in a second RBM a deep architecture can be created. The method is claimed to have the ability to learn very complex and rich structures in data, making these models attractive for supervised learning. However, the generative behavior of RBMs has largely been unexplored. In this presentation, we discuss the relationship between parameter specification and the prevalence of degenerate models as well as an exposition of the difficulties in fitting such highly flexible models.