Algorithms For Machine Learning On Massive Datasets

• Activity Number: 288
• Type: Topic Contributed
• Date/Time: Tuesday, August 2, 2011 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Computing
• Author(s): Alexander Gray*+
• Companies: Georgia Institute of Technology
• Keywords: massive data ; algorithms ; computational methods ; machine learning

Abstract

I'll describe algorithms and data structures for allowing the most powerful machine learning methods, which often scale quadratically or even cubically with the number of data points, to be performed many orders of magnitude faster than naive implementations. Such techniques can make previously impossible statistical analyses tractable on the scale of entire sky surveys. I will touch on scalable algorithms we have developed for nearest-neighbors, kernel density estimation, nonparametric Bayes classification, principal component analysis, local linear regression, hidden Markov models, k-means, manifold learning, support vector machines, and n-point correlation functions, among others. In addition to techniques inspired by computational geometry, fast multipole methods, and Monte Carlo integration, we employ a distributed framework which can be thought of as a higher-order version of Google's MapReduce. Our algorithms have enabled several first-of-a-kind large-scale analyses of astronomy data, networking data, biomedical data, and others.