Grouped and Hierarchical Model Selection through Composite Absolute Penalties (CAPs)

• Activity Number: 326
• Type: Topic Contributed
• Date/Time: Tuesday, August 8, 2006 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Guilherme Rocha*+ and Peng Zhao and Bin Yu
• Companies: University of California, Berkeley and University of California, Berkeley and University of California, Berkeley
• Keywords: variable selection ; LARS ; penalized regression ; penalized classification

Abstract

Recently, much attention has been devoted to model selection in regression and classification by use of a penalty function (Tibshirani 1996). In some cases, one may want to incorporate further natural groupings or hierarchical structures present within the regressors into the selected model. Our goal is to obtain model estimates that approximate the true model while preserving such structures. Letting (Y_i,X_i) be a set of observations of a response Y_i and corresponding explanatory variables X_i, we obtain our model estimates by modeling EY=g(Xb) and jointly minimizing a convex loss function L(b, Y, X) and penalty (CAP) function formed according to the desired structure. The CAP penalty is formed by suitably defining groups G_i, collecting the L_{\gamma_{i}} norm of the coefficients b_{G_i} into a new vector, and computing the norm of this new vector.