Fast Bayesian Factor Analysis via Automatic Rotations to Sparsity

• Activity Number: 235
• Type: Topic Contributed
• Date/Time: Monday, August 10, 2015 : 2:00 PM to 3:50 PM
• Sponsor: International Society for Bayesian Analysis (ISBA)
• Author(s): Veronika Rockova* and Edward I. George
• Companies: University of Pennsylvania and The Wharton School
• Keywords: Indian Buffet Process ; Factor Rotation ; LASSO ; PX-EM ; Spike-and-Slab

Abstract

Rotational transformations have traditionally played a key role in enhancing the interpretability of factor analysis via post-hoc modifications of the factor model orientation. Regularization methods also serve to achieve this goal by prioritizing sparse loading matrices. In this work, we cross-fertilize these two paradigms within a unifying Bayesian framework. Our approach deploys intermediate factor rotations throughout the learning process, greatly enhancing the effectiveness of sparsity inducing priors. These automatic rotations to sparsity are embedded within a PXL-EM algorithm, a Bayesian variant of parameter-expanded EM for posterior mode detection. By iterating between soft-thresholding of small factor loadings and transformations of the factor basis, we obtain (a) dramatic accelerations, (b) robustness against poor initializations and (c) better oriented sparse solutions. For accurate recovery of factor loadings, we deploy a two-component refinement of the Laplace prior, the spike-and-slab LASSO prior. The potential of the proposed procedure is demonstrated on both simulated and real high-dimensional data, which would render posterior simulation impractical.