Richard S. Sutton - Webdocs Cs Ualberta - University of Alberta

Research
Pioneered and made repeated contributions to reinforcement learning, an approach to
artificial and natural intelligence that emphasizes learning and planning from sample
experience. Currently seeking to extend reinforcement learning ideas to an empirically
grounded approach to knowledge representation.
Most significant contributions:
• The theory of temporal-difference learning and the TD(λ) algorithm (1988).
• The standard textbook for reinforcement learning (with Barto, 1998)
• The actor-critic (policy gradient) class of algorithms (1984, 2000).
• The Dyna architecture integrating learning, planning and reacting (1990)
• Temporal-difference models of animal learning (with Barto, 1981, 1990)
• The “options” framework for temporal abstraction (with Precup, Singh, 1999)
• Predictive state representations (with Littman, Singh, 2002)
• Algorithms for online step-size adaptation (1981, 1992)
• Temporal-difference networks for grounded knowledge representation (2005, 2006)
• Gradient temporal-difference algorithms (with Maei, Szepesvari, 2008–)
Selected Grants
NSERC Collaborative Research and Development Grant, with Nortel Networks and Bell
Canada, “Learning and Prediction in High-dimensional Stochastic Domains,” September
2006 – August 2009, funded at CAD$186,523. One of five principal investigators.
iCORE Chair and Professorship Establishment Grant, “Reinforcement Learning and
Artificial Intelligence,” September 1, 2003 – August 31, 2008, funded at CAD$3,000,000.
Principal investigator. Renewed until August 2013 at an additional CAD$2,750,000.
Alberta Ingenuity Centre Grant, “Alberta Ingenuity Centre for Machine Learning,” April
2003 – March 2008, funded at CAD$9,887,600. One of eight principal investigators.
Renewed until March 2009 at CAD$2,000,000. Renewed again in 2009 for another five
years at CAD$10,000,000.
NSERC Discovery Grant, “Reinforcement Learning and Artificial Intelligence,” April, 2004
– March 2009, funded at CAD$250,000. Principal investigator. Renewed in 2009 for a
second five years at an additional CAD$190,000.
Air Force Office of Scientific Research to the University of Massachusetts, “Stochastic
Scheduling and Planning Using Reinforcement Learning,” AFOSR Grant Number
F49620-96-1-0254, June 1, 1996 – May 31, 2000, funded at USD$446,570. Coprincipal
investigator with A. Barto.
2