Online Program Home
  My Program

Abstract Details

Activity Number: 24 - Statistical Computing and Graphics Student Awards
Type: Topic Contributed
Date/Time: Sunday, July 30, 2017 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistical Computing
Abstract #323287
Title: The Biglasso Package: a Memory- and Computation-Efficient Solver for Lasso Model Fitting with Big Data in R
Author(s): Yaohui Zeng* and Patrick Breheny
Companies: University of Iowa and University of Iowa
Keywords: Lasso ; Big data ; Memory-mapping ; Feature screening ; Out-of-core computing ; Parallel computing
Abstract:

Penalized regression models such as the lasso have been extensively applied to analyzing high-dimensional data sets. However, due to memory limitations, existing R packages like glmnet and ncvreg are not capable of fitting lasso-type models for ultrahigh-dimensional, multi-gigabyte data sets that are seen in many areas such as genetics, genomics, biomedical imaging, and high-frequency finance. In this research, we implement an R package called biglasso that tackles this challenge. biglasso utilizes memory-mapped files to store the massive data on the disk, only reading data into memory when necessary during model fitting, and is thus able to handle out-of-core computation seamlessly. Moreover, it's equipped with newly proposed, more efficient feature screening rules, which substantially accelerate the computation. Benchmarking experiments show that our biglasso package, as compared to existing ones like glmnet, is much more memory- and computation-efficient. We further analyze a 31 GB real data set on a laptop with only 16 GB RAM to demonstrate the out-of-core computation capability of biglasso in analyzing massive data sets that cannot be accommodated by existing R packages.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2017 program

 
 
Copyright © American Statistical Association