Interactive Q-Learning for Probabilities and Quantiles

• Activity Number: 69
• Type: Contributed
• Date/Time: Sunday, August 3, 2014 : 4:00 PM to 5:50 PM
• Sponsor: Biometrics Section
• Author(s): Kristin Linn*+ and Eric B. Laber and Leonard Stefanski
• Companies: North Carolina State University and North Carolina State University and North Carolina State University
• Keywords: Q-learning ; Dynamic treatment regimes

Abstract

A dynamic treatment regime is a sequence of decision rules in which each decision rule recommends treatment based on features of patient medical history such as past treatments and outcomes. Existing methods for estimating optimal dynamic treatment regimes from data optimize the expected value of a response variable. However, the mean may not always be the most appropriate summary of performance. We derive estimators of decision rules for optimizing probabilities and quantiles computed with respect to the response distribution for two-stage, binary treatment settings. The quantile framework enables estimation of a dynamic treatment regime that optimizes a pre-specified quantile such as the median. The proposed methods perform favorably in simulated experiments. We illustrate our approach by analyzing data from a sequentially randomized trial where the primary outcome is remission of depression symptoms.