Structured Learning via Alternating Linearization

• Activity Number: 544
• Type: Contributed
• Date/Time: Wednesday, August 7, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Xiaodong Lin*+ and Minh Pham and Andrzej Ruszczynski
• Companies: Rutgers University and Rutgers University and Rutgers University
• Keywords: Generalized lasso ; Non-smooth optimization ; Operator splitting

Abstract

We adapt the alternating linearization method for proximal decomposition to structured regularization problems, in particular, to the generalized lasso problems. The method is related to two well-known operator splitting methods, the Douglas-Rachford and the Peaceman-Rachford method, but it has descent properties with respect to the objective function. Its convergence mechanism is related to that of bundle methods of nonsmooth optimization. We present implementation for very large problems with the use of specialized algorithms and sparse data structures. Extensions to non-convex situations will also be discussed. Finally, we present numerical results for several synthetic and real-world examples including a three-dimensional fused lasso problem, which illustrate the scalability, efficacy, and accuracy of the method.