Online Program Home
My Program

Abstract Details

Activity Number: 112 - Statistical Challenges in the Processing and Analysis of Mobile Health Data
Type: Invited
Date/Time: Monday, July 29, 2019 : 8:30 AM to 10:20 AM
Sponsor: Section on Statistics in Epidemiology
Abstract #300373 Presentation
Title: Parameterizing Exploration
Author(s): Jesse Clifton* and Lili Wu and Eric B Laber
Companies: NC State University and North Carolina State University and NC State University
Keywords: reinforcement learning; mobile health; machine learning; multi armed bandit

Exploration in reinforcement learning is often accomplished by heuristics which account for neither the time horizon of the decision problem nor the decision-maker’s current state of knowledge of the dynamics of the underlying system. Accounting for these features could greatly improve the exploration-exploitation tradeoff (for instance, by avoiding over-exploration in short time horizons), and is computationally feasible in applications such as mobile health (mHealth) in which moderate computing times are acceptable. We introduce Parameterized Exploration (PE), a simple family of methods for tuning the exploration schedule which leverages an estimated model of dynamics of the decision problem and accounts for the time horizon. The proposed methods provides significant empirical gains relative to competing methods in variety of empirical examples and an application the management of type I diabetes using mobile interventions. We show that the proposed method is consistent for the optimal exploration schedule within the class under consideration and show how its estimated sampling distribution can be used to design robust and safe exploration strategies.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program