Convexlars: A Solution Path Algorithm for General Convex Loss Function

• Activity Number: 401
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Wei Xiao*+ and Yichao Wu and Hua Zhou
• Companies: North Carolina State University and North Carolina State University and North Carolina State University
• Keywords: LARS ; solution path algorithm ; LASSO ; convex loss ; ordinary differential equation

Abstract

Regularization is a statistical tool to avoid overfitting and obtain parsimonious and interpretable models. In general, a regularization method minimizes the sum of a loss function and a penalty term. The simplest penalty term is the L1 penalty and it leads to the popular L1-penalized methods. Efron, Hastie, Johnstone & Tibshirani (2004) proposed a solution path algorithm, the least angel regression (LAR), for the least square case, where the LAR solution path is piecewise linear. The authors also demonstrated that a slight modification of the LAR solution path leads to the LASSO solution path. In this article we extend this elegant path algorithm to general convex loss functions, where the solution path is exact and piecewise given by ordinary differential equations. We also show that a slight modification of this LAR solution path leads to the LASSO solution path. Last, we illustrate the capability of our algorithm with four real world examples, i.e., recurrent event data, panel count data, ada-boost and gaussian graphical model.