Online Program Home
My Program

Abstract Details

Activity Number: 320 - Practical and Realistic Variable Selection Methods
Type: Invited
Date/Time: Tuesday, July 31, 2018 : 10:30 AM to 12:20 PM
Sponsor: IMS
Abstract #326867 Presentation
Title: Statistical Inference for Online Learning and Stochastic Approximation via Hierarchical Incremental Gradient Descent
Author(s): Weijie Su* and Yuancheng Zhu
Companies: University of Pennsylvania and University of Pennsylvania
Keywords: Stochastic gradient descent; Optimization; Confidence interval; Hierarchical splitting; t-distribution; Ruppert-Polyak
Abstract:

Stochastic gradient descent (SGD) is an immensely popular approach for online learning in settings where data arrives in a stream or data sizes are very large. However, despite an ever-increasing volume of work on SGD, much less is known about the statistical inferential properties of SGD-based predictions. This talk introduces a novel procedure termed HiGrad to conduct statistical inference for online learning, without incurring additional computational cost compared with SGD. The HiGrad procedure begins by performing SGD updates for a while and then splits the single thread into several threads, and this procedure hierarchically operates in this fashion along each thread. With predictions provided by multiple threads in place, a t-based confidence interval is constructed by decorrelating predictions using covariance structures given by the Ruppert--Polyak averaging scheme. Under certain regularity conditions, the HiGrad confidence interval is shown to attain asymptotically exact coverage probability. Finally, the performance of HiGrad is evaluated through extensive simulation studies and a real data example. An R package higrad has been developed to implement the method.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program