Alternatives to Penalization for Sparse Models

• Activity Number: 319
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 8:30 AM to 10:20 AM
• Sponsor: WNAR
• Author(s): Sarah Emerson*+
• Companies: Oregon State University
• Keywords: Lasso ; Sparse methods ; High-dimension ; Clustering ; Matrix Decomposition ; Penalized methods

Abstract

Penalized methods, such as the lasso and adaptive lasso, are employed to obtain sparse solutions in many high-dimensional problems including regression modeling, covariance matrix estimation, and sparse clustering. These methods are often applied in analysis of genomic data, or more generally in any setting where a large number of predictors are available, with the goal of identifying or discriminating between phenotypes or sub-populations. In some of these settings, it is not clear that this penalization approach is an efficient or optimal solution. While the lasso penalty does produce a sparse solution for most problems, it does not necessarily produce the best sparse solution, and involves the inconvenient choice of tuning parameter value. We explore computationally simpler, faster, and more direct approaches for sparse matrix decompositions and variable selection for clustering, and demonstrate that the resulting solutions are generally superior to the lasso penalty approach. For a given degree of sparsity, our solutions recover a higher proportion of the signal present. Furthermore, the proposed approach makes it easier to obtain a solution with a desired sparsity.