Modified Q-Learning with Generalized Estimating Equations for Optimizing Dynamic Treatment Regimes with Repeated-Measures Outcomes

• Activity Number: 185 - Novel Methods for Clinical Trial Design and Characterizing Heterogeneity
• Type: Contributed
• Date/Time: Tuesday, August 4, 2020 : 10:00 AM to 2:00 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Yuan Zhang* and David Vock and Thomas Murray
• Companies: University of Minnesota and University of Minnesota and University of Minnesota
• Keywords: dynamic treatment regimes; Q-learning; repeated-measures outcomes; generalized estimating equations

Abstract

Dynamic treatment regimes (DTRs) are of increasing interest in clinical trials and personalized medicine because they allow tailoring decision making based on a patient’s treatment and covariate history. In some sequential multiple assignment randomized trials (SMARTs) for children with developmental language disorder (DLD), investigators monitor a patient’s performance by collecting repeated-measures outcomes at each stage of randomization as well as after the treatment period. Standard Q-learning with linear regression as Q-functions is widely implemented to identify the optimal DTR, but fails to provide point estimates of average treatment effect (ATE) at every time point of interest. Moreover, Q-learning in general is susceptible to misspecification of outcome model. To address these problems, we propose a modified version of Q-learning with a generalized estimating equation (GEE) as Q-function. Simulation studies demonstrate that the proposed method performs well in identifying the optimal DTR and is also robust to model misspecification.