Simplex Factor Models for High-Dimensional Categorical Data and Data Fusion

• Activity Number: 2
• Type: Invited
• Date/Time: Sunday, July 31, 2011 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): David Dunson*+ and David Dunson
• Companies: Duke University and Duke University
• Keywords: tensor decomposition ; factor analysis ; categorical data ; SVD ; nonparametric Bayes ; data fusion

Abstract

Gaussian latent factor models are routinely used for modeling of dependence in continuous, binary and ordered categorical data. For unordered categorical variables, Euclidean latent factor models lead to challenging computation and overly complex modeling structures. With this motivation, we propose a novel class of simplex factor models. In the single factor case, the model treats the different categorical outcomes as independent with unknown marginals. The model can characterize highly flexible dependence structures parsimoniously with few factors, and as factors are added, any multivariate categorical data distribution can be accurately approximated. Using a Bayesian approach for computation and inferences, a highly efficient MCMC algorithm is proposed that scales well with increasing dimension, with an adaptive Gibbs step enabling selection of the number of factors. Relationships with mixed membership models and tensor decompositions are described, and we evaluate the approach through simulation examples and applications to modeling dependence and classification from nucleotide sequences. The framework is natural for sparsely characterizing higher order interactions in categ