For High-Dimensional Hierarchical Models, Consider Exchangeability of Effects Across Covariates Instead of Across Data Sets

• Activity Number: 424 - Priors and Model Specifications for Variable and Feature Selection
• Type: Contributed
• Date/Time: Wednesday, August 10, 2022 : 10:30 AM to 12:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Brian Trippe* and Hilary Finucane and Tamara Broderick
• Companies: Massachusetts Institute of Technology and Broad Institute and Massachusetts Institute of Technology
• Keywords: linear mixed modeling; exchangeability; hierarchical modeling; high dimensional

Abstract

Hierarchical Bayesian methods enable information sharing across regression problems on multiple groups of data. While standard practice is to model regression parameters (effects) as (1) exchangeable across the groups and (2) correlated to differing degrees across covariates, we show that this approach exhibits poor statistical performance when the number of covariates exceeds the number of groups. For instance, in statistical genetics, we might regress dozens of traits (defining groups) for thousands of individuals (responses) on up to millions of genetic variants (covariates). When an analyst has more covariates than groups, we argue it is preferable to instead model effects as (1) exchangeable across covariates and (2) correlated to differing degrees across groups. We propose a hierarchical model expressing our alternative perspective. We devise an empirical Bayes estimator for learning the degree of correlation between groups. We develop theory that demonstrates our method outperforms the classic one when the number of covariates dominates the number of groups, and corroborate this result empirically on several high-dimensional multiple regression and classification problems.