Statistical Learning in the Cloud with Graphlab

• Activity Number: 288
• Type: Topic Contributed
• Date/Time: Tuesday, August 2, 2011 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Computing
• Author(s): Carlos Guestrin*+
• Companies: Carnegie Mellon University
• Keywords: machine learning ; statistical learning ; parallel algorithms ; distributed algorithms ; cloud computing ; large-scale data

Abstract

Exponentially increasing dataset sizes have driven Statistical Learning experts to explore parallel and distributed computing. Furthermore, cloud computing resources such as Amazon EC2 have become available, providing cheap and scalable platforms for large scale computation. However, due to the complexities involved in distributed design, it can be difficult for researchers to take full advantage of cloud resources. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave learning experts repeatedly solving the same design challenges.

Targeting common patterns in learning , we developed GraphLab, which compactly expresses asynchronous iterative algorithms with sparse computational dependencies, while ensuring data consistency and achieving a high degree of parallel performance. We demonstrate the expressiveness of the framework by designing and implementing parallel versions for a variety of real-world tasks, including learning graphical models with approximate inference, Gibbs sampling, tensor factorization, Co-EM, Lasso and Compressed Sensing, evaluating on clouds of up to 256 processors.