|
Abstract:
|
A Markov decision process is a common model for sequential decision making. In settings where the state of the decision process is high-dimensional, it is difficult to form a high-quality model of process dynamics or to apply semi-parametric estimators of an optimal decision strategy. We develop a notion of sufficient dimension reduction for Markov decision processes wherein only a low-dimensional summary of the state is retained at each time point yet no information about the optimal decision strategy is lost. We illustrate the proposed methodology with an application to mobile health.
|
