Asymptotically Efficient Regularization Parameter Selection in Penalized Regression and Small Sample Corrections

• Activity Number: 40
• Type: Contributed
• Date/Time: Sunday, July 31, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Cheryl J. Flynn*+ and Clifford M. Hurvich and Jeffrey S. Simonoff
• Companies: New York University and New York University and New York University
• Keywords: AIC ; Asymptotic Efficiency ; BIC ; Model Selection ; Penalized Regression ; SCAD

Abstract

Zhang, Li and Tsai (2010) proved that AIC-type criteria are asymptotically efficient selectors of the tuning parameter in non-concave (SCAD-)penalized regression methods under the assumption that the population variance is known or that a consistent estimate is available. We relax this assumption to prove that AIC itself is asymptotically efficient and we study its performance in finite samples. In classical regression, Hurvich and Tsai (1989) showed that AIC tends to select overly complex models when the dimension of the maximum candidate model is large relative to the sample size. Simulation studies suggest that AIC suffers from the same shortcomings when used in SCAD-penalized regression. We therefore propose the use of the classical AICc as an alternative. A similar investigation into the finite sample properties of BIC reveals analogous overfitting tendencies and leads us to further propose the use of a corrected BIC, BICc. In their respective worlds, AICc and BICc have the desired asymptotic properties and we use simulations to assess their performance, as well as that of other selectors, in finite samples.