Truncated Thompson Sampling for Safe and Efficient Precision Public Health

• Activity Number: 213 - Sequential Decision Making and Causal Inference
• Type: Invited
• Date/Time: Monday, July 29, 2019 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): Eric B Laber* and Jesse Clifton
• Companies: NC State University and NC State University
• Keywords: Precision medicine; Reinforcement learning; Causal inference; Spillover ; Infectious diseases; Machine Learning

Abstract

Emerging infectious diseases are a cause of humanitarian and economic crises across the world. In developing regions, a serious epidemic can result in the collapse of healthcare infrastructure or even the failure of an affected state. The most recent 2013-2015 outbreak of Ebola virus disease in West Africa is an example of such an epidemic. The economic, infrastructure, and human costs of this outbreak provide strong motivation for the examination of adaptive treatment strategies that allocate resources in response to and anticipation of the evolution of an epidemic. We formalize adaptive management of an emerging infectious disease spreading across a set of locations as a treatment regime that maps up-to-date information on the epidemic to a subset of locations identified as high-priority for treatment. Because the disease dynamics are not known at outbreak, an optimal treatment regime must be estimated online. We propose a `safe' variant of Q-learning with Thompson Sampling for the management of an emerging infectious disease based on sampling from a truncated confidence distribution for parameters indexing the Q-function. Theoretical and empirical properties are examined.