Abstract:
|
In this talk, we propose an adaptive sequential probability ratio test (Ada-SPRT) that obtains the optimal experiment selection rule, stopping time, and final decision rule under a single Bayesian decision framework. Our motivating application comes from binary labeling tasks in crowdsourcing, where the requestor needs to simultaneously decide which worker to choose to provide the label and when to stop collecting labels to save for budget. We characterize the structure of the optimal adaptive sequential design that minimizes the Bayes risk through log-likelihood ratio statistic and develop dynamic programming based algorithms for both non-truncated and truncated tests. We further propose to adopt empirical Bayes approach for estimating class priors and an EM algorithm for estimating workers' quality. This is a joint work with Xiaoou Li, Yunxiao Chen, Jingchen Liu and Zhiliang Ying.
|