Ensemble Learning: Variable Selection and Collective Decisionmaking

• Activity Number: 485
• Type: Invited
• Date/Time: Wednesday, August 1, 2012 : 10:30 AM to 12:20 PM
• Sponsor: SSC
• Author(s): Mu Zhu*+
• Companies: University of Waterloo
• Keywords: parallel evolutionary algorithm ; neutrality ; ranking ; ROC curve ; stochastic search ; synergy

Abstract

Ensemble classifiers are widely known to be extremely powerful. In this talk, I will first present some of our recent work in applying the ensemble approach to the variable selection problem, e.g., by simulating Darwinian evolution in parallel universes or by repeatedly performing stochastic stepwise search. I will advocate that variable ranking may be a more useful objective in practice than variable selection per se, and that the ensemble approach is more naturally suited for such an objective. I will then present some results that we inadvertently "discovered" while studying the problem of ranking. Our results imply that two decision-making entities (e.g., a husband and wife) can work better together only if at least one of them is occasionally willing to stay neutral, thereby "justifying" an age-old practice among marriage counselors. I will end by highlighting some open problems. (Based on joint work with Hugh Chipman, Lu Xin, and Shangsi Wang, in chronological order.)