Using Machine Learning to Approximate Dynamic Programming for Large-scale Sequential Decision Problems

• Activity Number: 378
• Type: Invited
• Date/Time: Wednesday, August 10, 2005 : 10:30 AM to 12:20 PM
• Sponsor: ENAR
• Author(s): Andrew Barto*+
• Companies: University of Massachusetts
• Keywords: machine learning ; sequential decision problems ; reinforcement learning ; approximate dynamic programming

Abstract

An active subarea of machine learning addresses methods for efficiently approximating solutions to large-scale stochastic sequential decision problems. Although originally motivated by the goal of building artificial intelligent agents whose activities can be modeled in terms of sequential decisionmaking, research has expanded to encompass a range of methods and applications that make contact with control engineering, operations research, psychology, and neuroscience. In this approach, called Reinforcement Learning, the goal is for an autonomous agent to learn how to increase the amount of reward it attains while interacting with an incompletely-known stochastic environment. The focus is on methods that compute, or approximate, value functions that provide predictions of future reward and can be used in a variety of ways to direct action. The relationship between three basic classes of methods (stochastic dynamic programming, Monte Carlo methods, and temporal-difference methods) provides the framework for understanding what the Machine Learning perspective has added to our collection of methods for finding high-performance policies.