Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 299 - Machine Learning in Causal Inference with Applications in Complicated Settings
Type: Topic Contributed
Date/Time: Wednesday, August 5, 2020 : 10:00 AM to 11:50 AM
Sponsor: Biometrics Section
Abstract #312272
Title: DOES the MARKOV DECISION PROCESS FIT the DATA: TESTING for the MARKOV PROPERTY in SEQUENTIAL DECISION MAKING
Author(s): Chengchun Shi* and Runzhe Wan and Rui Song and Wenbin Lu and Ling Leng
Companies: The London School of Economics and NC State Univeristy and NC State University and North Carolina State University and Amazon
Keywords: reinforcement learning; sequential decision making; Markov decision process; forward-backward learning
Abstract:

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this talk, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an important role for identifying the optimal policy in high-order Markov decision processes and partially observable MDPs. We apply our test to both synthetic datasets and a real data example from mobile health studies to illustrate its usefulness.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program