Recover Low-Rank Matrices with Heteroscedastic Noise

• Activity Number: 137
• Type: Contributed
• Date/Time: Monday, August 4, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Jingshu Wang*+ and Art Owen
• Companies: Stanford University and Stanford University
• Keywords: low-rank matrices recovery ; heteroscedastic noise ; BCV ; Factor Analysis ; rank selection

Abstract

We consider recovery of low-rank matrices from noisy data with heteroscedastic noise. We use an early stopping alternating method (ESAM) which iteratively alters the estimate of the noise variance and the low-rank matrix and corrects over-fitting by an early-stopping rule. Various simulations in our study suggest stopping after just 3 iterations and we have seen that ESAM gives better recovery than the SVD on either the original data or the standardized data with the optimal rank given. To select a rank, we use an early-stopping bi-cross-validation (BCV) technique modified from BCV for the white noise model. Our method leaves out half the rows and half the columns as in BCV, but uses low rank operations involving ESAM instead of the SVD on the retained data to predict the held out entries. Simulations considering both strong and weak signal cases show that our method is the most accurate overall, compared to some BCV strategies and two versions of Parallel Analysis (PA). PA is a state-of-the art method for choosing the number of factors in Factor Analysis.