Online Program Home
  My Program

Abstract Details

Activity Number: 219 - SLDS 2017 Student Paper Awards Session
Type: Topic Contributed
Date/Time: Monday, July 31, 2017 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #322737 View Presentation
Title: Statistical Inference for Model Parameters in Stochastic Gradient Descent
Author(s): Yichen Zhang* and Xi Chen and Jason D. Lee and Tong Thomson Xin
Companies: New York University and NYU and University of Southern California and National University of Singapore
Keywords: Stochastic gradient descent ; asymptotic variance ; batch-means estimator ; high-dimensional inference ; time-inhomogeneous Markov chain
Abstract:

The stochastic gradient descent algorithm has been widely used in statistical estimation for large-scale data due to its computational and memory efficiency. While most existing work focuses on the convergence of the objective function or the error of the obtained solution, we investigate the statistical inference of the true model parameters based on SGD. We propose two consistent estimators of the asymptotic covariance of the average iterate from SGD: (1) an intuitive plug-in estimator and (2) a computationally more efficient batch-means estimator, which only uses the iterates from SGD, inspired by the classical batch-means estimator from MCMC. Both proposed estimators allow us to construct asymptotically exact confidence intervals and hypothesis tests. We further discuss an extension to conducting inference based on SGD for high-dimensional linear regression. Using a variant of the SGD algorithm, we construct a debiased estimator of each regression coefficient that is asymptotically normal. This gives a one-pass algorithm for computing the sparse regression coefficient estimator and confidence intervals, which is computationally attractive and applicable to online data.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2017 program

 
 
Copyright © American Statistical Association