Abstract:
|
Many large-scale machine learning problems involve estimating an unknown parameter ?i for each of many items. For example, a key problem in sponsored search is to estimate the click through rate (CTR) of each of billions of query-ad pairs. Most common methods, though, only give a point estimate of each ?i. A posterior distribution for each ?i is usually more useful but harder to get.
We present a simple post-processing technique that takes point estimates or scores ti (from any method) and estimates an approximate posterior for each ?i. We build on the idea of calibration, a common post-processing technique that estimates E(?i | ti). Our method, second order calibration, uses empirical Bayes methods to estimate the distribution of ?i | ti and uses the estimated distribution distribution as an approximation to the posterior distribution of ?i. We show that this can yield improved point estimates and useful accuracy estimates. The method scales to large problems - our motivating example is a CTR estimation problem involving tens of billions of query-ad pairs.
|
ASA Meetings Department
732 North Washington Street, Alexandria, VA 22314
(703) 684-1221 • meetings@amstat.org
Copyright © American Statistical Association.