Adjusted Information Criteria for High-Dimensional Sparse Variable Selection

• Activity Number: 131
• Type: Contributed
• Date/Time: Monday, August 4, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Nonparametric Statistics
• Author(s): Maarten Jansen*+
• Companies: Universite Libre de Bruxelles
• Keywords: high-dimensional ; sparsity ; model selection ; Mallows' Cp ; variable selection

Abstract

Information criteria such as those by Mallows or Akaike can be interpreted as (nearly) unbiased estimators of a distance between the proposed and the true model. When the criterion is optimized in a model selection routine, an accumulation of false positives may induce a groundless reduction of the criterion value for models larger than the true model. The effect of false positives on the distance to the true model can be reduced by the use of shrinkage estimators within the selected model, but shrinkage cannot undo the overestimation of the model size, which is an important issue in sparse model selection. This talk quantifies the optimizattion bias for least squares estimators without shrinkage. The effect of the false positives is described as a mirror: false positives present themselves as better than insignificant variables, but in reality they only carry above-average noise. The mirror that reflects the real onto the apparant behavior and vice versa is an oracular procedure that knows the noise-free observations for variable selection but not for estimation within the selection. Based on this formalism, we propose mirror-adjusted expressions for information criteria.