Grouped and Hierarchical Model Selection through Composite Absolute Penalties

• Activity Number: 446
• Type: Invited
• Date/Time: Wednesday, August 1, 2007 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Peng Zhao and Guilherme V. Rocha*+ and Bin Yu
• Companies: University of California, Berkeley and University of California, Berkeley and University of California, Berkeley
• Keywords: regularization ; penalization ; regression ; classification ; variable selection

Abstract

For datasets with many predictors and few samples, side information often must be added to fitting. We introduce Composite Absolute Penalties (CAP) to blend predefined grouping and hierarchical information among the predictors into regression and classification. Special cases include Zou & Hastie(2005)'s elastic net, Kim et. al(2005)'s Blockwise Sparse Regression and Yuan & Lin(2006)'s GLASSO. CAPs are built by combining norm penalties at the across and within group levels. For disjoint groups, a Bayesian interpretation lays bare the role of the norms used to construct CAP. Hierarchical selection is reached by defining nested groups. For general CAPs, we use the BLASSO and cross-validation to compute CAP estimates. For CAPs built from L_1 and L_\infty norms, we give efficient algorithms and regularization selection criteria. The feasibility of CAP is shown through simulated experiments.