Algorithm for Constructing Model Selection Criteria in Sparse Regression Modeling

• Activity Number: 356
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Ibuki Hoshina*+ and Sadanori Konishi
• Companies: Chuo University and Chuo University
• Keywords: Degrees of freedom ; LARS ; L1 type of regularization ; Tuning parameter selection

Abstract

In recent years, sparse regression modeling has emerged as powerful tools for analyzing high-dimensional data. Regression coefficients are estimated by optimizing the penalized least square loss function with various L1 type norms such as the lasso and elastic net. The desirable performances of these regularization methods heavily depend on an appropriate choice of the tuning parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. The information criteria AIC, BIC and Mallows' Cp type of criteria may be used as a tuning parameter selection tool, for which the concept of model complexity plays a key role. We propose an algorithm for calculating the measure of model complexity, and introduce model selection criteria for evaluating models estimated by the lasso type regularization methods. We present simulated and real data examples to examine the performance of the proposed procedures.