Random Weighting in LASSO Regression

• Activity Number: 58 - Advanced Bayesian Topics (Part 1)
• Type: Contributed
• Date/Time: Sunday, August 8, 2021 : 3:30 PM to 5:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Tun Lee Ng* and Michael Newton
• Companies: University of Wisconsin, Madison and University of Wisconsin, Madison
• Keywords: weighted Bayesian bootstrap; random-weighting; LASSO; weighted likelihood bootstrap; perturbation bootstrap

Abstract

We establish statistical properties of the random weighting method in LASSO regression under different regularization parameters and suitable regularity conditions. The random weighting methods in view concern repeated optimization of a randomized objective function, motivated by the need for computational approximations to Bayesian posterior sampling. In the context of LASSO regression, we repeatedly assign analyst-drawn random weights to terms in the objective function (including the penalty terms), and optimize to obtain random weighting samples. We show that existing approaches have conditional model selection consistency and conditional asymptotic normality at different growth rates of regularization parameters. We propose an extension to the available random weighting methods and establish that the resulting samples attain conditional sparse normality and conditional consistency in a growing-dimension setting. We find that random weighting has both approximate-Bayesian and sampling-theory interpretations, and we illustrate the proposed methodology via simulation studies and a benchmark data example.