JSM 2015 Preliminary Program

Online Program Home
My Program

Abstract Details

Activity Number: 119
Type: Topic Contributed
Date/Time: Monday, August 10, 2015 : 8:30 AM to 10:20 AM
Sponsor: Section on Nonparametric Statistics
Abstract #316286
Title: Second-Order Calibration: A Cheap Way to Get Approximate Posteriors
Author(s): Omkar Muralidharan*
Companies: Google
Keywords: Empirical Bayes ; Calibration ; Regression ; Posterior Distribution ; Predictive Distribution
Abstract:

Many large-scale machine learning problems involve estimating an unknown parameter ?i for each of many items. For example, a key problem in sponsored search is to estimate the click through rate (CTR) of each of billions of query-ad pairs. Most common methods, though, only give a point estimate of each ?i. A posterior distribution for each ?i is usually more useful but harder to get.

We present a simple post-processing technique that takes point estimates or scores ti (from any method) and estimates an approximate posterior for each ?i. We build on the idea of calibration, a common post-processing technique that estimates E(?i | ti). Our method, second order calibration, uses empirical Bayes methods to estimate the distribution of ?i | ti and uses the estimated distribution distribution as an approximation to the posterior distribution of ?i. We show that this can yield improved point estimates and useful accuracy estimates. The method scales to large problems - our motivating example is a CTR estimation problem involving tens of billions of query-ad pairs.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2015 program





For program information, contact the JSM Registration Department or phone (888) 231-3473.

For Professional Development information, contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

2015 JSM Online Program Home