Fast Stagewise Algorithms for Approximate Regularization Paths

• Activity Number: 651
• Type: Contributed
• Date/Time: Thursday, August 8, 2013 : 8:30 AM to 10:20 AM
• Sponsor: IMS
• Author(s): Ryan Tibshirani*+
• Keywords: stagewise ; boosting ; regularization path

Abstract

Forward stagewise regression enjoys an interesting connection to the lasso: under some conditions, the path of estimates constructed by forward stagewise exactly coincides with the lasso path, as the step size goes to zero. Essentially the same equivalence holds outside of regression, for the minimization of arbitrary differentiable convex loss functions subject to an $\ell_1$ norm constraint. Stagewise estimates provide a useful approximation even when they do not match their $\ell_1$-constrained analogues, and are computationally appealing. In general, regularization can take many forms, beyond the $\ell_1$ norm and sparsity; in this talk, we extend the stagewise idea to general convex regularization problems.