Online Program Home
My Program

Abstract Details

Activity Number: 582 - Random Effects and Mixed Models
Type: Contributed
Date/Time: Wednesday, August 1, 2018 : 2:00 PM to 3:50 PM
Sponsor: Biometrics Section
Abstract #328904 Presentation
Title: Fast Computation of Large-Scale Mixed Effects Models
Author(s): Norman Matloff and Robin Yancey*
Companies: University of California at Davis and University of California, Davis
Keywords: mixed effect models; parallel computation

This era of Big Data includes many applications of mixed effects modeling, such as large genomics studies (Hoffman et al Bioinf. 2014), recommender systems (Gao and Owen, arXiv 2016), and salary prediction by micro-region and occupation (Kenthapadi et al, arXiv 2017). However, the time complexity for estimation in such models can grow as fast a n^1.5. Even worse, the computation may not fit into available memory, rendering direct, single-stage estimation impossible. Parallel computation on separate machines may be a remedy, using a model-specific algorithm. Here we present a model-independent approach to the problem using a technique we call Software Alchemy (Matloff, JSS 2016), and show computational speedup on various real datasets. Due to algorithmic time complexity issues, it is actually possible in some cases to achieve superlinear performance, i.e. with a speedup factor greater than the number of computational processes.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program