On the Relationship Between Non-Negative Matrix Factorization and Generalized Linear Modeling

• Activity Number: 295
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Karthik Devarajan*+ and Vincent Chi Kwan Cheung
• Companies: Fox Chase Cancer Center and Massachusetts Institute of Technology
• Keywords: non-negative matrix factorization ; generalized linear models ; EM algorithm ; alternating constrained minimization ; high-dimensional data ; signal-dependent noise

Abstract

Nonnegative matrix factorization (NMF) by the multiplicative updates algorithm is a powerful machine learning method for decomposing a high-dimensional nonnegative matrix V into the product of two nonnegative matrices, W and H where V ~ WH. The non-negativity constraints in NMF allow only additive linear combinations of the data which enables it to learn parts that have distinct physical representations in reality. NMF has found successful applications in the analysis and interpretation of large scale data arising in areas such as computational biology and natural language processing, among others. In this paper, we demonstrate a link between NMF and generalized linear models (GLM). We show that known formulations of NMF based on the Poisson and Gaussian likelihoods can be simply re-formulated as an alternating constrained minimization problem using GLM. Furthermore, we extend NMF by generalizing this approach to include other members of the exponential family of distributions such as gamma and inverse Gaussian that are commonly used within the GLM framework. We illustrate our methods using an application to biomedical data containing signal-dependent noise.