Fast Exact Path Algorithm for Regularized Statistical Estimation with Applications to Genomics

• Activity Number: 57
• Type: Invited
• Date/Time: Sunday, July 31, 2011 : 4:00 PM to 5:50 PM
• Sponsor: IMS
• Author(s): Hua Zhou*+ and Kenneth Lange
• Companies: North Carolina State University and University of California at Los Angeles
• Keywords: penalized regression ; regularization ; solution path

Abstract

Regularization is widely used in statistics and machine learning to prevent overfitting and gear solution towards prior information. In general a regularized problem minimizes the sum of a loss function and a penalty term that enforces certain constraints on the parameters. Particular choices of constraints lead to the popular lasso, fused-lasso, and other generalized lasso regression methods. Developing efficient optimization methods for many nonseparable penalties remains a challenge. We propose an efficient, exact path algorithm that works for any convex loss function with $l_1$ regularizations and/or more complicated regularizations such as inequality constraints among the parameters encountered in shape-restricted regressions and nonparametric density estimation. In the path following process, the solution path hits, exits, and slides along the various constraints. Inspection of the entire solution path is revealing as it vividly illustrates the tradeoffs between goodness of fit and model parsimony. The potential of the new path algorithm is illustrated by various applications in modern genetics.