Model Combining in a Covariate-Adjusted Response-Adaptive Design

• Activity Number: 36
• Type: Contributed
• Date/Time: Sunday, July 29, 2012 : 2:00 PM to 3:50 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Wei Qian*+ and Yuhong Yang
• Companies: University of Minnesota and University of Minnesota
• Keywords: bandit problem ; clinical trial ; model combining ; sequential estimation ; response adaptive design

Abstract

Multi-armed bandit problem is an important optimization game with promising applications in clinical trials and web advertising. We consider a setting where the rewards of bandit machines are associated with covariates (also known as covariate-adjusted response-adaptive design), and focus on the performance of \epsilon-greedy, one of the most well-known algorithms for bandit problem. Since many nonparametric and parametric methods in supervised learning may be applied to the sequential reward function estimation, and guidance on how to choose these methods are generally unavailable, we integrate a model combining procedure derived from AFTER algorithm (Yang, 2004) into \epsilon-greedy to obtain robust game-playing results. It is shown that our model combining procedure can maintain the asymptotic optimality of individual methods in terms of the accumulated reward. The benefits of combining various nonparametric methods are also demonstrated in both theory and simulation.