Conference Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 455 - Learning Under Nonstationarity
Type: Invited
Date/Time: Wednesday, August 10, 2022 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #319221
Title: Testing Nonstationary and Policy Optimization in Reinforcement Learning
Author(s): Chengchun Shi* and Mengbing Li and Zhenke Wu and Piotr Fryzlewicz
Companies: London School of Economics and Political Science and University of Michigan and University of Michigan, Ann Arbor and London School of Economics
Keywords: Reinforcement Learning; Nonstationarity; Hypothesis Testing; Change Point Detection; Policy Optimisation
Abstract:

We consider reinforcement learning (RL) methods in offline nonstationary environments. Most of existing RL algorithms in the literature rely on the stationarity assumption that requires the system transition and the reward function to be constant over time. However, the stationarity assumption is very restrictive in practice and is likely to be violated in a number of applications, including traffic signal control, robotics and mobile health. In this paper, we develop a consistent procedure to test the nonstationarity of the optimal policy based on a pre-collected historical data, without additional online data collection. Based on the proposed test, we further develop a sequential change point detection method that can be naturally coupled with existing state-of-the-art RL methods for policy optimisation in nonstationary environments. The usefulness of our method is illustrated by theoretical results, simulation studies and a real data example from the 2018 Intern Health Study.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2022 program