Inference for Q-Learning Parameters Using an Adaptive M-Out-of-N Bootstrap Scheme

• Activity Number: 155
• Type: Topic Contributed
• Date/Time: Monday, July 30, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Bibhas Chakraborty*+ and Eric B Laber
• Companies: Columbia University and North Carolina State University
• Keywords: dynamic treatment regime ; Q-learning ; non-regularity ; m-out-of-n bootstrap

Abstract

A dynamic treatment regime (DTR) consists of a set of decision rules that dictate how to personalize treatment to patients based on available treatment and covariate history. A common method for estimating an optimal DTR from data is Q-learning which involves nonsmooth operations of the data. This nonsmoothness causes standard asymptotic approaches for inference like the usual bootstrap or Taylor series arguments to break down if applied without correction. Here, we consider an adaptive m-out-of-n bootstrap procedure for constructing condence intervals for the parameters indexing the optimal DTR. The proposed method produces asymptotically correct condence sets, and has the advantage of being computationally much less burdensome than competing methods possessing the same theoretical property. We provide an extensive simulation study to compare our proposed method with currently available inference procedures. Analysis of the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study is used as an illustrative example of our methodology.