Andrew Barto

Last updated
Andrew G. Barto
Andrew G. Barto.jpg
Bornc. 1948 (age 7576)
Nationality American
Alma mater University of Michigan
Awards IEEE Neural Networks Society Pioneer Award, IJCAI Award for Research Excellence
Scientific career
Fields Computer science
Institutions University of Massachusetts Amherst
Doctoral students Richard S. Sutton

Andrew G. Barto (born c. 1948) is an American computer scientist, currently Professor Emeritus of computer science at University of Massachusetts Amherst. Barto is best known for his foundational contributions to the field of modern computational reinforcement learning. [1]

Contents

Early life and education

Barto received his B.S. with distinction in mathematics from the University of Michigan in 1970, after having initially majored in naval architecture and engineering. After reading work by Michael Arbib and McCulloch and Pitts he became interested in using computers and mathematics to model the brain, and five years later was awarded a Ph.D. in computer science for a thesis on cellular automata. [2]

Career

In 1977, Barto joined the College of Information and Computer Sciences at the University of Massachusetts Amherst as a postdoctoral research associate, was promoted to associate professor in 1982, and full professor in 1991. He was department chair from 2007 to 2011 and a core faculty member of the Neuroscience and Behavior program. [3]

During this time at UMass, Barto co-directed the Autonomous Learning Laboratory (initially the Adaptive Network Laboratory), which generated several key ideas in reinforcement learning. Richard Sutton, with whom he co-authored the influential book Reinforcement Learning: An Introduction (MIT Press 1998; 2nd edition 2018), was his first PhD student. Barto graduated 27 PhD students, thirteen of which went on to become professors. [3]

Barto published over one hundred papers or chapters in journals, books, and conference and workshop proceedings. He is co-author with Richard Sutton of the book Reinforcement Learning: An Introduction, MIT Press 1998 (2nd edition 2018), and co-editor with Jennie Si, Warren Powell, and Don Wunch II of the Handbook of Learning and Approximate Dynamic Programming, Wiley-IEEE Press, 2004. [4]

Awards and honors

Barto is a Fellow of the American Association for the Advancement of Science, a Fellow and Senior Member of the IEEE, [5] and a member of the American Association for Artificial Intelligence and the Society for Neuroscience.

Barto was awarded the UMass Neurosciences Lifetime Achievement Award, 2019, the IEEE Neural Network Society Pioneer Award in 2004, [6] and the IJCAI Award for Research Excellence, 2017. His citation for the latter read:

Professor Barto is recognized for his groundbreaking and impactful research in both the theory and application of reinforcement learning. [1]

Related Research Articles

Reinforcement learning (RL) is an interdisciplinary area of machine learning and optimal control concerned with how an intelligent agent ought to take actions in a dynamic environment in order to maximize the cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning.

<span class="mw-page-title-main">Steve Furber</span> British computer scientist

Stephen Byram Furber is a British computer scientist, mathematician and hardware engineer, and Emeritus ICL Professor of Computer Engineering in the Department of Computer Science at the University of Manchester, UK. After completing his education at the University of Cambridge, he spent the 1980s at Acorn Computers, where he was a principal designer of the BBC Micro and the ARM 32-bit RISC microprocessor. As of 2023, over 250 billion ARM chips have been manufactured, powering much of the world's mobile computing and embedded systems, everything from sensors to smartphones to servers.

Terrence Joseph Sejnowski is the Francis Crick Professor at the Salk Institute for Biological Studies where he directs the Computational Neurobiology Laboratory and is the director of the Crick-Jacobs center for theoretical and computational biology. He has performed pioneering research in neural networks and computational neuroscience.

Temporal difference (TD) learning refers to a class of model-free reinforcement learning methods which learn by bootstrapping from the current estimate of the value function. These methods sample from the environment, like Monte Carlo methods, and perform updates based on current estimates, like dynamic programming methods.

Michael Anthony Arbib is an American computational neuroscientist. He is an Adjunct Professor of Psychology at the University of California at San Diego and professor emeritus at the University of Southern California; before his 2016 retirement he was the Fletcher Jones Professor of computer science, as well as a professor of biological sciences, biomedical engineering, electrical engineering, neuroscience and psychology.

<span class="mw-page-title-main">Stephen Grossberg</span> American scientist (born 1939)

Stephen Grossberg is a cognitive scientist, theoretical and computational psychologist, neuroscientist, mathematician, biomedical engineer, and neuromorphic technologist. He is the Wang Professor of Cognitive and Neural Systems and a Professor Emeritus of Mathematics & Statistics, Psychological & Brain Sciences, and Biomedical Engineering at Boston University.

<span class="mw-page-title-main">Peter Dayan</span> Researcher in computational neuroscience

Peter Dayan is a British neuroscientist and computer scientist who is director at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany, along with Ivan De Araujo. He is co-author of Theoretical Neuroscience, an influential textbook on computational neuroscience. He is known for applying Bayesian methods from machine learning and artificial intelligence to understand neural function and is particularly recognized for relating neurotransmitter levels to prediction errors and Bayesian uncertainties. He has pioneered the field of reinforcement learning (RL) where he helped develop the Q-learning algorithm, and made contributions to unsupervised learning, including the wake-sleep algorithm for neural networks and the Helmholtz machine.

<span class="mw-page-title-main">Jacek M. Zurada</span> Polish engineer

Jacek M. Zurada is a Polish engineer who serves as a Professor of Electrical and Computer Engineering Department at the University of Louisville, Kentucky. His M.S. and Ph.D. degrees are from Politechnika Gdaṅska ranked as #1 among Polish universities of technology. He has held visiting appointments at Swiss Federal Institute of Technology, Zurich, Princeton, Northeastern, Auburn, and at overseas universities in Australia, Chile, China, France, Germany, Hong Kong, Italy, Japan, Poland, Singapore, Spain, and South Africa. He is a Life Fellow of IEEE and a Fellow of International Neural Networks Society and Doctor Honoris Causa of Czestochowa Institute of Technology, Poland.

<span class="mw-page-title-main">Richard S. Sutton</span> Canadian computer scientist

Richard S. Sutton is a Canadian computer scientist. He is a professor of computing science at the University of Alberta and a research scientist at Keen Technologies. Sutton is considered one of the founders of modern computational reinforcement learning, having several significant contributions to the field, including temporal difference learning and policy gradient methods.

Hava Siegelmann is an American computer scientist and Provost Professor at the University of Massachusetts Amherst.

<span class="mw-page-title-main">Daniela L. Rus</span> American computer scientist

Daniela L. Rus is a roboticist and computer scientist, Director of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), and the Andrew and Erna Viterbi Professor in the Department of Electrical Engineering and Computer Science (EECS) at the Massachusetts Institute of Technology. She is the author of the books Computing the Future and The Heart and the Chip.

<span class="mw-page-title-main">Yann LeCun</span> French computer scientist (born 1960)

Yann André LeCun is a Turing Award winning French-American computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta.

<span class="mw-page-title-main">Dimitri Bertsekas</span> Greek electrical engineer

Dimitri Panteli Bertsekas is an applied mathematician, electrical engineer, and computer scientist, a McAfee Professor at the Department of Electrical Engineering and Computer Science in School of Engineering at the Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts, and also a Fulton Professor of Computational Decision Making at Arizona State University, Tempe.

<span class="mw-page-title-main">Christoph von der Malsburg</span> German physicist and neuroscientist

Christoph von der Malsburg is a German physicist and neuroscientist.

Klaus-Robert Müller is a German computer scientist and physicist, most noted for his work in machine learning and brain–computer interfaces.

Stefan Schaal is a German-American computer scientist specializing in robotics, machine learning, autonomous systems, and computational neuroscience.

Prashant Shenoy is an Indian-American Computer Scientist. He is a Distinguished Professor of Computer Science in the College of Information and Computer Sciences at the University of Massachusetts Amherst. He is known for his contributions to distributed computing, computer networks, cloud computing, and computational sustainability.

<span class="mw-page-title-main">Amir Hussain (cognitive scientist)</span>

Amir Hussain is a cognitive scientist, the director of Cognitive Big Data and Cybersecurity (CogBID) Research Lab at Edinburgh Napier University He is a professor of computing science. He is founding Editor-in-Chief of Springer Nature's internationally leading Cognitive Computation journal and the new Big Data Analytics journal. He is founding Editor-in-Chief for two Springer Book Series: Socio-Affective Computing and Cognitive Computation Trends, and also serves on the Editorial Board of a number of other world-leading journals including, as Associate Editor for the IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Systems, Man, and Cybernetics (Systems) and the IEEE Computational Intelligence Magazine.

Victor R. Lesser is Distinguished Professor Emeritus in the School of Computer Science at the University of Massachusetts at Amherst and the Director of Multi-Agent Systems Laboratory. He is widely considered as the founding father of multi-agent systems. He received the IJCAI Award for Research Excellence in 2009.

Donald C. Wunsch II is Mary K. Finley Distinguished Professor of computer engineering at the Missouri University of Science and Technology, and a Fellow of the Institute of Electrical and Electronics Engineers He is known for his work on " hardware implementations, reinforcement and unsupervised learning".

References

  1. 1 2 "IJCAI 2017 Awards". 19 August 2017. Retrieved September 6, 2022.
  2. "Virtual History Interview". International Neural Network Society. 7 January 2022. Retrieved September 6, 2022.
  3. 1 2 "Andrew G. Barto". University of Massachusetts Amherst. 17 February 2008. Retrieved October 18, 2020.
  4. UMass Amherst: Department of Computer Science Archived September 2, 2006, at the Wayback Machine
  5. "Barto elected IEEE fellow". University of Massachusetts Amherst. November 22, 2005. Archived from the original on December 3, 2019. Retrieved December 3, 2019.
  6. ""IEEE Computational Intelligence Society Past Recipients"". 6 September 2022. Retrieved September 6, 2022.