Thresholded Reduced-Rank Multivariate Regression

• Activity Number: 190
• Type: Contributed
• Date/Time: Monday, August 5, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Ranye Sun*+ and Mohsen Pourahmadi
• Companies: Texas A&M University and Texas A&M University
• Keywords: Iterative subspace projections ; Low-rank matrix approximation ; Multivariate linear regression ; Regularization ; Singular value decomposition

Abstract

In large multivariate linear regression models it is desirable to approximate the regression coefficient matrix by low-rank, sparse matrices constructed from its singular value decomposition (SVD). We reduce this regression problem to a sparse SVD (SSVD) problem for a correlated data matrix. Its solution requires generalizing the fast iterative thresholding (FIT-SSVD) algorithm in Yang, Ma and Buja (2011) to the correlated data situation. We provide such a generalization which inherits all the computational and statistical advantages of FIT-SSVD including its sparse initialization, novel ways of estimating the thresholding parameters without relying on the computationally expensive cross-validation and the thresholded subspace iterations. The latter guarantees the orthogonality of the singular vectors and computes these simultaneously and not sequentially. The methodology and potential adverse impact of dependence on the algorithms are illustrated using simulation and real data.