High-Dimensional Variable Selection with Reciprocal L_1 Regularization

• Activity Number: 240
• Type: Contributed
• Date/Time: Monday, August 4, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Qifan Song*+ and Faming Liang
• Companies: Texas A&M and Texas A&M
• Keywords: Lasso ; Variable selection ; High dimensional ; Penalized Likelihood ; Reciprocal Lasso

Abstract

Penalized likelihood methods have been widely used in variable selection problems, where the penalty functions are typically symmetric about 0, continuous and nondecreasing. We propose a new penalized likelihood method, reciprocal Lasso (or in short, rLasso), based on a new class of penalty functions which are decreasing, discontinuous at 0, and converge to infinity when the coefficients approach zero. The new penalty functions give nearly zero coefficients infinity penalties; in contrast, the conventional penalty functions give nearly zero coefficients nearly zero penalties (e.g., Lasso or SCAD) or constant penalties (e.g., L_0 penalty). This distinguishing feature makes rLasso very attractive for variable selection: It can effectively avoid to select overly dense models. We establish the consistency of the rLasso for variable selection and coefficient estimation under both the low and high dimensional settings. Since the rLasso penalty functions induce an objective function with multiple local minima, we also propose an efficient Monte Carlo optimization algorithm to solve the minimization problem. Numerical examples are demonstrated to illustrate the proposed rLasso method.