Revisiting Marginal Regression

• Activity Number: 49
• Type: Invited
• Date/Time: Sunday, August 1, 2010 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Jiashun Jin*+ and Christopher Genovese and Larry Wasserman
• Companies: Carnegie Mellon University and Carnegie Mellon University and Carnegie Mellon University
• Keywords: lasso ; marginal regression ; variable selection ; faithfulness ; irrepresentable ; phase diagram

Abstract

The lasso has become an important practical tool for high dimensional regression. But despite the availability of efficient algorithms, the lasso remains computationally demanding in regression problems where the number of variables vastly exceeds the number of data points. A much older method, marginal regression, largely displaced by the lasso, offers a promising alternative in this case. Computation for marginal regression is practical even when the dimension is very high. In this paper, we compare the conditions for exact reconstruction of the two procedures. Also, we derive conditions under which the marginal regression will provide exact reconstruction with high probability. Last, we derive rates of convergence for the procedures and offer a new partitioning of the ``phase diagram,'' that shows when exact or Hamming reconstruction is effective.