|
Activity Number:
|
440
- SLDS CSpeed 8
|
|
Type:
|
Contributed
|
|
Date/Time:
|
Thursday, August 12, 2021 : 4:00 PM to 5:50 PM
|
|
Sponsor:
|
Section on Statistical Learning and Data Science
|
|
Abstract #318486
|
|
|
Title:
|
Quasi-Monte Carlo, Quasi-Newton for Variational Bayes
|
|
Author(s):
|
Sifan A. Liu* and Art Owen
|
|
Companies:
|
Stanford University and Stanford University
|
|
Keywords:
|
randomized quasi-Monte Carlo;
quasi-Newton;
variational Bayes;
L-BFGS;
numerical optimization
|
|
Abstract:
|
Many machine learning problems optimize an objective that must be measured with noise. The primary method is a first order stochastic gradient descent using one or more Monte Carlo (MC) samples at each step. There are settings where ill-conditioning makes second order methods such as L-BFGS more effective. We study the use of randomized quasi-Monte Carlo (RQMC) sampling for such problems. When MC sampling has a root mean squared error (RMSE) of $O(n^{-1/2})$ then RQMC has an RMSE of $o(n^{-1/2})$ that can be close to $O(n^{-3/2})$ in favorable settings. We prove that improved sampling accuracy translates directly to improved optimization. In our empirical investigations for variational Bayes, using RQMC with stochastic L-BFGS greatly speeds up the optimization, and sometimes finds a better parameter value than MC does.
|
Authors who are presenting talks have a * after their name.
