Statisticians: 3, Computer Scientists: 35

• Activity Number: 211
• Type: Invited
• Date/Time: Monday, August 2, 2010 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): David Purdy*+
• Companies: University of California, Berkeley
• Keywords: Machine Learning ; Future of statistics ; Ensemble methods ; Sparse data

Abstract

John Tukey and Leo Breiman warned us that a day would come when statistics would need to focus more on computing, or risk losing good students to computer science. The Netflix Prize provides many examples of how our field needs to do more.

In the top 2 teams, participants with a computer science background vastly outnumbered those with a statistics background. There are a number of lessons that the field of statistics can learn from the fact that undergraduates in CS were well equipped to compete, while statisticians at all levels were not well prepared to implement advanced algorithms.

In this talk, I will address methodological issues arising with such a large, sparse dataset, how it demands serious computational talents, and where there is ample room for statistics to make contributions.