A General Framework for Identification of Permissible Variable Subsets in Structured Model Selection

• Activity Number: 97 - New Methods for Structured Variable Selection
• Type: Topic Contributed
• Date/Time: Monday, August 8, 2022 : 8:30 AM to 10:20 AM
• Sponsor: SSC (Statistical Society of Canada)
• Author(s): Guanbo Wang* and Mireille Elisa Schnitzer and Tom Chen and Rui Wang and Robert William Platt
• Companies: McGill University and Universite de Montreal and Harvard Pilgrim Health Care Institute and Harvard Medical School and Harvard T.H. Chan School of Public Health and McGill University
• Keywords: covariate structure; dictionary; group Lasso; latent overlapping group Lasso; selection rule; variable selection

Abstract

Variable selection is commonly used to arrive at a parsimonious model. Oftentimes a selection rule that prescribes the permissible variable combinations in the final model is desirable due to the inherent structural constraints among the candidate variables. Penalized regression methods can integrate these restrictions ("selection rules") by assigning the covariates to different groups and then applying different penalties to the groups of variables. However, no general framework has yet been proposed to formalize selection rules and their application. In this work, we develop a mathematical language for constructing selection rules in variable selection, where the resulting combination of permissible sets of selected covariates, called a "selection dictionary", is formally defined. We show that all selection rules can be represented as a combination of operations on constructs, and these can be used to identify the related selection dictionary. One may then apply some criteria to select the best model. We also present a necessary and sufficient condition for a grouping structure used with overlapping group Lasso to carry out variable selection under an arbitrary selection rule.