Model Selection and Estimation in Regression with Grouped Variables

• Activity Number: 112
• Type: Invited
• Date/Time: Monday, August 8, 2005 : 10:30 AM to 12:20 PM
• Sponsor: Section on Nonparametric Statistics
• Author(s): Ming Yuan*+ and Yi Lin
• Companies: Georgia Institute of Technology and University of Wisconsin, Madison
• Keywords: ANOVA ; LASSO ; LARS ; Nonnegative Garrote ; Piecewise linear solution path

Abstract

In this paper, we consider the problem of selecting grouped variables (factors) for accurate prediction in regression. Such a problem arises naturally in many practical situations with the multifactor ANOVA problem as the most important and well-known example. Instead of selecting factors by stepwise backward elimination, we focus on estimation accuracy and consider extensions of the LASSO, the LARS, and the nonnegative garrote for factor selection. The LASSO, the LARS, and the nonnegative garrote are recently proposed regression methods that can be used to select individual variables. We study and propose efficient algorithms for the extensions of these methods for factor selection and show that these extensions give superior performance to the traditional stepwise backward elimination method in factor selection problems. We also study the similarities and differences among these methods. Simulations and real examples are used.