JSM 2013 Home
Online Program Home
My Program

Abstract Details

Activity Number: 51
Type: Invited
Date/Time: Sunday, August 4, 2013 : 4:00 PM to 5:50 PM
Sponsor: Section on Statistical Computing
Abstract - #307159
Title: Programming with Big Data in R
Author(s): George Ostrouchov*+ and Wei-Chen Chen and Drew Schmidt and Pragneshkumar Patel
Companies: Oak Ridge National Laboratory and Oak Ridge National Laboratory and University of Tennessee and University of Tennessee
Keywords: Big Data ; Parallel ; R
Abstract:

Current practice in R focuses mostly on multicore desktop platforms and on small clusters with a manager/workers parallel programming style. Much of this focus comes from R's emphasis on interactive data analysis, which is also one of R's great strengths and a source of its popularity. The High Performance Computing (HPC) community, on the other hand, has a focus on batch processing and a Single Program Multiple Data (SPMD) parallel programming style. We take note of this and conduct research to enable SPMD style programming in R for truly big data distributed across a large computing platform. Resulting R syntax is largely unchanged from serial programming and distributed parallel support is provided through S4 classes and methods (see packages at r-pbd.org). We tightly couple R with highly scalable SPMD HPC libraries (PBLAS, ScaLAPACK, and the BLACS). This work is also compelling from an HPC viewpoint, where R is viewed as a high-level domain specific language for data and where interest lies in high-level management of multiple levels of parallelism. Two levels of parallelism are available in R when SPMD libraries and multithreaded libraries, such as MKL or ACML, are combined.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2013 program




2013 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Continuing Education program, please contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

ASA Meetings Department  •  732 North Washington Street, Alexandria, VA 22314  •  (703) 684-1221  •  meetings@amstat.org
Copyright © American Statistical Association.