Richard S. Sutton

Last updated
Richard S. Sutton

FRS FRSC
Richard Sutton, October 27, 2016.jpg
Nationality Canadian
Citizenship Canadian
Alma mater University of Massachusetts Amherst
Stanford University
Known for Temporal difference learning, Dyna, Options, GQ(λ)
Awards AAAI Fellow (2001)
President's Award (INNS) (2003)
Royal Society of Canada Fellow (2016)
Scientific career
Fields Artificial Intelligence
Reinforcement Learning
Institutions University of Alberta
Thesis Temporal credit assignment in reinforcement learning  (1984)
Doctoral advisor Andrew Barto
Doctoral students David Silver, Doina Precup
Website incompleteideas.net

Richard S. Sutton FRS FRSC is a Canadian computer scientist. He is a professor of computing science at the University of Alberta and a research scientist at Keen Technologies. [1] Sutton is considered one of the founders of modern computational reinforcement learning, [2] having several significant contributions to the field, including temporal difference learning and policy gradient methods. [3]

Contents

Life and education

Richard Sutton was born in Ohio, and grew up in Oak Brook, Illinois, a suburb of Chicago.

Sutton received his B.A. in psychology from Stanford University in 1978 before taking an M.S. (1980) and Ph.D. (1984) in computer science from the University of Massachusetts Amherst under the supervision of Andrew Barto. His doctoral dissertation, Temporal Credit Assignment in Reinforcement Learning, introduced actor-critic architectures and temporal credit assignment. [4] [3]

He was influenced by Harry Klopf's work in the 1970s, which proposed that supervised learning is insufficient for AI or explaining intelligent behavior, and trial-and-error learning, driven by "hedonic aspects of behavior", is necessary. This focussed his interest to reinforcement learning. [5]

Career

In 1984, Sutton was a postdoctoral researcher at the University of Massachusetts.

From 1985 to 1994, he was a principal member of technical staff in the Computer and Intelligent Systems Laboratory at GTE in Waltham, Massachusetts. [3] After that, he spent 3 years at the University of Massachusetts Amherst as a senior research scientist. [3]

From 1998 to 2002, Sutton worked at the AT&T Shannon Laboratory in Florham Park, New Jersey as principal technical staff member in the artificial intelligence department. [3]

Since 2003, he has been a professor of computing science at the University of Alberta. He led the institution's Reinforcement Learning and Artificial Intelligence Laboratory until 2018. [6] [3]

While retaining his professorship, Sutton joined Deepmind in June 2017 as a distinguished research scientist and co-founder of its Edmonton office. [4] [7] [8]

Sutton became a Canadian citizen in 2015 and renounced his US citizenship [8] in 2017.

In a 2019 essay, Sutton criticized the field of AI research for failing "to learn the bitter lesson that building in how we think we think does not work in the long run", arguing that "70 years of AI research [had shown] that general methods that leverage computation are ultimately the most effective, and by a large margin", beating efforts building on human knowledge about specific fields like computer vision, speech recognition, chess or Go. [9] [10]

In 2023 he and John Carmack announced a partnership for the development of AGI. [11]

Selected publications

Awards and honors

Sutton is fellow of the Association for the Advancement of Artificial Intelligence (AAAI) since 2001. [12] In 2003 he received the President's Award from the International Neural Network Society [13] and in 2013, the Outstanding Achievement in Research award from the University of Massachusetts Amherst. [14]

Sutton's nomination as a AAAI fellow reads: [12]

For significant contributions to many topics in machine learning, including reinforcement learning, temporal difference techniques, and neural networks.

In 2016, Sutton was elected Fellow of the Royal Society of Canada. [15] In 2021, he was elected Fellow of the Royal Society. [16]

Related Research Articles

<span class="mw-page-title-main">Marvin Minsky</span> American cognitive scientist (1927–2016)

Marvin Lee Minsky was an American cognitive and computer scientist concerned largely with research of artificial intelligence (AI), co-founder of the Massachusetts Institute of Technology's AI laboratory, and author of several texts concerning AI and philosophy.

Reinforcement learning (RL) is an interdisciplinary area of machine learning and optimal control concerned with how an intelligent agent ought to take actions in a dynamic environment in order to maximize the cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning.

<span class="mw-page-title-main">Conference on Neural Information Processing Systems</span> Machine learning and computational neuroscience conference

The Conference and Workshop on Neural Information Processing Systems is a machine learning and computational neuroscience conference held every December. The conference is currently a double-track meeting that includes invited talks as well as oral and poster presentations of refereed papers, followed by parallel-track workshops that up to 2013 were held at ski resorts.

Temporal difference (TD) learning refers to a class of model-free reinforcement learning methods which learn by bootstrapping from the current estimate of the value function. These methods sample from the environment, like Monte Carlo methods, and perform updates based on current estimates, like dynamic programming methods.

Q-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment, and it can handle problems with stochastic transitions and rewards without requiring adaptations.

Michael Irwin Jordan is an American scientist, professor at the University of California, Berkeley and researcher in machine learning, statistics, and artificial intelligence.

<span class="mw-page-title-main">Peter Dayan</span> Researcher in computational neuroscience

Peter Dayan is a British neuroscientist and computer scientist who is director at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany, along with Ivan De Araujo. He is co-author of Theoretical Neuroscience, an influential textbook on computational neuroscience. He is known for applying Bayesian methods from machine learning and artificial intelligence to understand neural function and is particularly recognized for relating neurotransmitter levels to prediction errors and Bayesian uncertainties. He has pioneered the field of reinforcement learning (RL) where he helped develop the Q-learning algorithm, and made contributions to unsupervised learning, including the wake-sleep algorithm for neural networks and the Helmholtz machine.

<span class="mw-page-title-main">Andrew Barto</span> Professor of computer science

Andrew G. Barto is an American computer scientist, currently Professor Emeritus of computer science at University of Massachusetts Amherst. Barto is best known for his foundational contributions to the field of modern computational reinforcement learning.

<span class="mw-page-title-main">Arthur Samuel (computer scientist)</span> American computer scientist (1901– 1990)

Arthur Lee Samuel was an American pioneer in the field of computer gaming and artificial intelligence. He popularized the term "machine learning" in 1959. The Samuel Checkers-playing Program was among the world's first successful self-learning programs, and as such a very early demonstration of the fundamental concept of artificial intelligence (AI). He was also a senior member in the TeX community who devoted much time giving personal attention to the needs of users and wrote an early TeX manual in 1983.

<span class="mw-page-title-main">Michael L. Littman</span> American computer scientist

Michael Lederman Littman is a computer scientist, researcher, educator, and author. His research interests focus on reinforcement learning. He is currently a University Professor of Computer Science at Brown University, where he has taught since 2012.

Leslie Pack Kaelbling is an American roboticist and the Panasonic Professor of Computer Science and Engineering at the Massachusetts Institute of Technology. She is widely recognized for adapting partially observable Markov decision processes from operations research for application in artificial intelligence and robotics. Kaelbling received the IJCAI Computers and Thought Award in 1997 for applying reinforcement learning to embedded control systems and developing programming tools for robot navigation. In 2000, she was elected as a Fellow of the Association for the Advancement of Artificial Intelligence.

The AAAI Conference on Artificial Intelligence (AAAI) is one of the leading international academic conference in artificial intelligence held annually. It ranks 4th in terms of H5 Index in Google Scholar's list of top AI publications, after ICLR, NeurIPS, and ICML. It is supported by the Association for the Advancement of Artificial Intelligence. Precise dates vary from year to year, but paper submissions are generally due at the end of August to beginning of September, and the conference is generally held during the following February. The first AAAI was held in 1980 at Stanford University, Stanford California.

<span class="mw-page-title-main">Eric Xing</span>

Eric Poe Xing is an American computer scientist whose research spans machine learning, computational biology, and statistical methodology. Xing is founding President of the world’s first artificial intelligence university, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI).

<span class="mw-page-title-main">Francesca Rossi</span> Italian computer scientist

Francesca Rossi is an Italian computer scientist, currently working at the IBM Thomas J. Watson Research Center as an IBM Fellow and the IBM AI Ethics Global Leader.

<span class="mw-page-title-main">Shlomo Zilberstein</span>

Shlomo Zilberstein is an Israeli-American computer scientist. He is a Professor of Computer Science and Associate Dean for Research and Engagement in the College of Information and Computer Sciences at the University of Massachusetts, Amherst. He graduated with a B.A. in Computer Science summa cum laude from Technion – Israel Institute of Technology in 1982, and received a Ph.D. in Computer Science from University of California at Berkeley in 1993, advised by Stuart J. Russell. He is known for his contributions to artificial intelligence, anytime algorithms, multi-agent systems, and automated planning and scheduling algorithms, notably within the context of Markov decision processes (MDPs), Partially Observable MDPs (POMDPs), and Decentralized POMDPs (Dec-POMDPs).

David Silver is a principal research scientist at Google DeepMind and a professor at University College London. He has led research on reinforcement learning with AlphaGo, AlphaZero and co-lead on AlphaStar.

<span class="mw-page-title-main">Thomas G. Dietterich</span>

Thomas G. Dietterich is emeritus professor of computer science at Oregon State University. He is one of the pioneers of the field of machine learning. He served as executive editor of Machine Learning (journal) (1992–98) and helped co-found the Journal of Machine Learning Research. In response to the media's attention on the dangers of artificial intelligence, Dietterich has been quoted for an academic perspective to a broad range of media outlets including National Public Radio, Business Insider, Microsoft Research, CNET, and The Wall Street Journal.

Regina Barzilay is an Israeli-American computer scientist. She is a professor at the Massachusetts Institute of Technology and a faculty lead for artificial intelligence at the MIT Jameel Clinic. Her research interests are in natural language processing and applications of deep learning to chemistry and oncology.

<span class="mw-page-title-main">Doina Precup</span> Romanian researcher of artificial intelligence

Doina Precup is a Romanian researcher currently living in Montreal, Canada. She specializes in artificial intelligence (AI). Precup is associate dean of research at the faculty of science at McGill University, Canada research chair in machine learning and a senior fellow at the Canadian Institute for Advanced Research. She also heads the Montreal office of Deepmind.

<span class="mw-page-title-main">Thomas Dean (computer scientist)</span> American computer scientist

Thomas L. Dean is an American computer scientist known for his work in robot planning, probabilistic graphical models, and computational neuroscience. He was one of the first to introduce ideas from operations research and control theory to artificial intelligence. In particular, he introduced the idea of the anytime algorithm and was the first to apply the factored Markov decision process to robotics. He has authored several influential textbooks on artificial intelligence.

References

  1. ↑ "John Carmack and Rich Sutton partner to accelerate development of Artificial General Intelligence". markets.businessinsider.com. Retrieved 2023-10-02.
  2. ↑ "Exclusive: Interview with Rich Sutton, the Father of Reinforcement Learning". 2018-01-11. Archived from the original on 2018-01-11. Retrieved 2018-12-17.
  3. 1 2 3 4 5 6 Piatetsky, Gregory (December 5, 2017). "Exclusive: Interview with Rich Sutton, the Father of Reinforcement Learning". KDnuggets. Retrieved 2024-02-10.
  4. 1 2 "Brief Biography for Richard Sutton". incompleteideas.net. Retrieved 2018-12-17.
  5. ↑ Sutton, Richard S.; Barto, Andrew (2020). Reinforcement learning: an introduction (Second ed.). Cambridge, Massachusetts: The MIT Press. pp. 22–23. ISBN   978-0-262-03924-6.
  6. ↑ Brown, Michael (May 10, 2021). "AI innovator Richard Sutton named to Royal Society". Alberta Machine Intelligence Institute. Retrieved 2024-02-10.
  7. ↑ "DeepMind expands to Canada with new research office in Edmonton, Alberta". DeepMind. Retrieved 2018-12-17.
  8. 1 2 "Edmonton AI guru Rich Sutton has lost his DeepMind but not his ambition". National Post. 2023-03-19. Retrieved 2023-07-02.
  9. ↑ Sutton, Rich (2019-03-13). "The Bitter Lesson". www.incompleteideas.net. Retrieved 2022-09-22.
  10. ↑ Tunstall, Lewis; Werra, Leandro von; Wolf, Thomas (2022-01-26). Natural Language Processing with Transformers. "O'Reilly Media, Inc.". ISBN   978-1-0981-0319-4.
  11. ↑ "John Carmack and Rich Sutton partner to accelerate development of Artificial General Intelligence". markets.businessinsider.com. Retrieved 2023-10-02.
  12. 1 2 "Elected AAAI Fellows". www.aaai.org. Retrieved 2018-12-17.
  13. ↑ "INNS Award Recipients". www.inns.org. Retrieved 2018-12-17.
  14. ↑ "Outstanding Achievement and Advocacy Award Recipients". College of Information and Computer Sciences, University of Massachusetts Amherst. 2010-10-05. Retrieved 2018-12-17.
  15. ↑ Brown, Michael (19 September 2016). "U of A Scholars Join Ranks of Royal Society". The Quad. Retrieved 24 August 2023.
  16. ↑ "Royal Society elects outstanding new Fellows and Foreign Members". royalsociety.org . Retrieved 2021-06-08.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.