A Unified Statistical Approach to Non-Negative Matrix Factorization and Probabilistic Latent Semantic Indexing

• Activity Number: 120
• Type: Topic Contributed
• Date/Time: Monday, August 5, 2013 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Karthik Devarajan*+ and Guoli Wang and Nader Ebrahimi
• Companies: Fox Chase Cancer Center and SRA International Inc. and Northern Illinois University
• Keywords: Renyi divergence ; non-negative matrix factorization ; probabilistic latent semantic indexing ; Poisson likelihood ; biomedical informatics ; text mining

Abstract

Non-negative matrix factorization (NMF) by the multiplicative updates algorithm is a powerful machine learning method for decomposing a high-dimensional nonnegative matrix into the product of two nonnegative matrices. NMF has been shown to have a unique parts-based, sparse representation of the data. NMF has been successfully applied in diverse areas such as natural language processing, information retrieval, signal processing and computational biology for analyzing large-scale data. There has also been simultaneous development of a related statistical latent class modeling approach, namely probabilistic latent semantic indexing (PLSI), for analyzing co-occurrence count data arising in natural language processing. In this talk, we describe a generalized statistical approach to NMF and PLSI based on Renyi's divergence between two non-negative matrices related to the Poisson likelihood. Our approach unifies various competing models and provides a unique theoretical framework for these methods by generalizing the relationship between them. We demonstrate the applicability of our approach using document clustering and text mining data arising in biomedical informatics.