Generalized Q Learning for Binary Outcomes

• Activity Number: 12
• Type: Topic Contributed
• Date/Time: Sunday, August 3, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Biometrics Section
• Author(s): Min Qian*+ and Eric B. Laber
• Companies: Columbia University and North Carolina State University
• Keywords: GLM ; personalized treatment ; dynamic treatment regimes

Abstract

Recent research in treatment and intervention science is shifting from the traditional 'one-size-fits-all' treatment to dynamic treatment regimes, which allow greater individualization in programming over time. A dynamic treatment regime is a sequence of decision rules that specify how the dosage and/or type of treatment should be adjusted through time in response to an individual's changing needs. A commonly used approach, Q-learning, involves an iterative two-step procedure that first uses regression to model the conditional mean outcome at each stage, and second, derives the estimated policy by maximizing the estimated conditional mean functions. We consider a generalization of Q-learning to the case of Binary outcomes, where generalized linear models are employed to estimate the conditional mean function at each stage. This approach is justified via a generalization error bound and is compared with competing methods via simulation studies and a data example.