Online Program Home
My Program

Abstract Details

Activity Number: 645 - Bayesian Optimization
Type: Topic Contributed
Date/Time: Thursday, August 1, 2019 : 10:30 AM to 12:20 PM
Sponsor: Section on Bayesian Statistical Science
Abstract #303100
Title: Bayesian Optimization for Policy Search via Online-Offline Experimentation
Author(s): Eytan Bakshy and Benjamin Letham*
Companies: Facebook and Facebook
Keywords: bayesian optimization; field experiments; gaussian process; kriging; multi-task; simulator

Bayesian optimization is an effective tool for tuning the parameters of machine learning systems via A/B tests. However, the ability to jointly optimize many parameters of the system policy can be limited by the low throughput of online field experiments. We describe how a simple, biased, offline simulator can be used to accelerate the optimization by learning a multi-task model that combines observations from both online and offline tests. We show results from multi-task Bayesian optimization of a Facebook ranking system, where augmenting the online tests with the simulator allows for jointly optimizing up to 20 parameters with as few as 40 total A/B tests. Finally, we analyze factors behind model generalization and identify settings where multi-task Bayesian optimization is most beneficial.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program