Geoffrey Hinton | |
---|---|
![]() Hinton speaking at the University of Toronto in 2023 | |
Born | Geoffrey Everest Hinton 6 December 1947 [1] Wimbledon, London, England |
Education | |
Known for |
|
Awards |
|
Scientific career | |
Fields | |
Institutions | |
Thesis | Relaxation and its role in vision (1977) |
Doctoral advisor | Christopher Longuet-Higgins [3] [4] [5] |
Doctoral students | |
Other notable students | |
Website | www |
Geoffrey Everest Hinton CC FRS FRSC [12] (born 6 December 1947) is a British-Canadian computer scientist and cognitive psychologist, most noted for his work on artificial neural networks. From 2013 to 2023, he divided his time working for Google (Google Brain) and the University of Toronto, before publicly announcing his departure from Google in May 2023, citing concerns about the risks of artificial intelligence (AI) technology. [13] In 2017, he co-founded and became the chief scientific advisor of the Vector Institute in Toronto. [14] [15]
With David Rumelhart and Ronald J. Williams, Hinton was co-author of a highly cited paper published in 1986 that popularized the backpropagation algorithm for training multi-layer neural networks, [16] although they were not the first to propose the approach. [17] Hinton is viewed as a leading figure in the deep learning community. [18] [19] [20] [21] [22] The image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky [23] and Ilya Sutskever for the ImageNet challenge 2012 [24] was a breakthrough in the field of computer vision. [25]
Hinton received the 2018 Turing Award, often referred to as the "Nobel Prize of Computing", together with Yoshua Bengio and Yann LeCun, for their work on deep learning. [26] They are sometimes referred to as the "Godfathers of Deep Learning", [27] [28] and have continued to give public talks together. [29] [30]
In May 2023, Hinton announced his resignation from Google to be able to "freely speak out about the risks of A.I." [31] He has voiced concerns about deliberate misuse by malicious actors, technological unemployment, and existential risk from artificial general intelligence. [32] He noted that establishing safety guidelines will require cooperation among those competing in use of AI in order to avoid the worst outcomes. [33]
Hinton was educated at Clifton College in Bristol [34] and King's College, Cambridge. After repeatedly changing his degree between different subjects like natural sciences, history of art, and philosophy, he eventually graduated in 1970 with a Bachelor of Arts in experimental psychology. [1] He continued his study at the University of Edinburgh where he was awarded a PhD in artificial intelligence in 1978 for research supervised by Christopher Longuet-Higgins. [3] [35]
After his PhD, Hinton worked at the University of Sussex and, after difficulty finding funding in Britain, [36] the University of California, San Diego and Carnegie Mellon University. [1] He was the founding director of the Gatsby Charitable Foundation Computational Neuroscience Unit at University College London. [1] He is currently [update] [37] a professor in the computer science department at the University of Toronto.
He holds a Canada Research Chair in Machine Learning [38] and, as of June 2024, is an advisor for the Learning in Machines & Brains program at the Canadian Institute for Advanced Research. [39] Hinton taught a free online course on Neural Networks on the education platform Coursera in 2012. [40] He joined Google in March 2013 when his company, DNNresearch Inc., was acquired, and was at that time planning to "divide his time between his university research and his work at Google". [41]
Hinton's research concerns ways of using neural networks for machine learning, memory, perception, and symbol processing. He has written or co-written more than 200 peer reviewed publications. [2] [42]
While Hinton was a postdoc at UC San Diego, David E. Rumelhart and Hinton and Ronald J. Williams applied the backpropagation algorithm to multi-layer neural networks. Their experiments showed that such networks can learn useful internal representations of data. [16] In a 2018 interview, [43] Hinton said that "David E. Rumelhart came up with the basic idea of backpropagation, so it's his invention". Although this work was important in popularising backpropagation, it was not the first to suggest the approach. [17] Reverse-mode automatic differentiation, of which backpropagation is a special case, was proposed by Seppo Linnainmaa in 1970, and Paul Werbos proposed to use it to train neural networks in 1974. [17]
In 1985, Hinton co-invented Boltzmann machines with David Ackley and Terry Sejnowski. [44] His other contributions to neural network research include distributed representations, time delay neural network, mixtures of experts, Helmholtz machines and Product of Experts. [45] An accessible introduction to Geoffrey Hinton's research can be found in his articles in Scientific American in September 1992 and October 1993. [46] In 2007, Hinton coauthored an unsupervised learning paper titled Unsupervised learning of image transformations. [47] In 2008, he developed the visualization method t-SNE with Laurens van der Maatens. [48] [49]
In October and November 2017 respectively, Hinton published two open access research papers on the theme of capsule neural networks, [50] [51] which according to Hinton, are "finally something that works well". [52]
At the 2022 Conference on Neural Information Processing Systems (NeurIPS) he introduced a new learning algorithm for neural networks that he calls the "Forward-Forward" algorithm. The idea of the new algorithm is to replace the traditional forward-backward passes of backpropagation with two forward passes, one with positive (i.e. real) data and the other with negative data that could be generated solely by the network. [53] [54]
In May 2023, Hinton publicly announced his resignation from Google. He explained his decision by saying that he wanted to "freely speak out about the risks of A.I." and added that a part of him now regrets his life's work. [13] [31]
Notable former PhD students and postdoctoral researchers from his group include Peter Dayan, [55] Sam Roweis, [55] Max Welling, [55] Richard Zemel, [3] [6] Brendan Frey, [7] Radford M. Neal, [8] Yee Whye Teh, [9] Ruslan Salakhutdinov, [10] Ilya Sutskever, [11] Yann LeCun, [56] Alex Graves, [55] and Zoubin Ghahramani. [55]
Hinton was elected a Fellow of the Royal Society (FRS) in 1998. [12] He was the first winner of the Rumelhart Prize in 2001. [57] His certificate of election for the Royal Society reads:
Geoffrey E. Hinton is internationally known for his work on artificial neural nets, especially how they can be designed to learn without the aid of a human teacher. He has compared effects of brain damage with effects of losses in such a net, and found striking similarities with human impairment, such as for recognition of names and losses of categorisation. His work includes studies of mental imagery, and inventing puzzles for testing originality and creative intelligence. It is conceptual, mathematically sophisticated, and experimental. He brings these skills together with striking effect to produce important work of great interest. [58]
In 2001, Hinton was awarded an honorary doctorate from the University of Edinburgh. [59] He was the 2005 recipient of the IJCAI Award for Research Excellence lifetime-achievement award. [60] He was awarded the 2011 Herzberg Canada Gold Medal for Science and Engineering. [61] In 2012, he received the Canada Council Killam Prize in Engineering. In 2013, Hinton was awarded an honorary doctorate from the Université de Sherbrooke. [62]
In 2016, he was elected a foreign member of National Academy of Engineering "for contributions to the theory and practice of artificial neural networks and their application to speech recognition and computer vision". [63] He received the 2016 IEEE/RSE Wolfson James Clerk Maxwell Award. [64]
He won the BBVA Foundation Frontiers of Knowledge Award (2016) in the Information and Communication Technologies category, "for his pioneering and highly influential work" to endow machines with the ability to learn. [65]
Together with Yann LeCun, and Yoshua Bengio, Hinton won the 2018 Turing Award for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing. [66] [67] [68]
In 2018, he became a Companion of the Order of Canada. [69] In 2021, he received the Dickson Prize in Science from the Carnegie Mellon University [70] and in 2022 the Princess of Asturias Award in the Scientific Research category, along with Yann LeCun, Yoshua Bengio, and Demis Hassabis. [71] In 2023, he was named an ACM Fellow. [72]
External videos | |
---|---|
![]() |
In 2023, Hinton expressed concerns about the rapid progress of AI. [32] [31] Hinton previously believed that artificial general intelligence (AGI) was "30 to 50 years or even longer away." [31] However, in a March 2023 interview with CBS, he stated that "general-purpose AI" may be fewer than 20 years away and could bring about changes "comparable in scale with the industrial revolution or electricity." [32]
In an interview with The New York Times published on 1 May 2023, [31] Hinton announced his resignation from Google so he could "talk about the dangers of AI without considering how this impacts Google." [73] He noted that "a part of him now regrets his life's work" due to his concerns and he expressed fears about a race between Google and Microsoft. [31]
In early May 2023, Hinton claimed in an interview with BBC that AI might soon surpass the information capacity of the human brain. He described some of the risks posed by these chatbots as "quite scary". Hinton explained that chatbots have the ability to learn independently and share knowledge. This means that whenever one copy acquires new information, it is automatically disseminated to the entire group. This allows AI chatbots to have the capability to accumulate knowledge far beyond the capacity of any individual. [74]
Hinton has expressed concerns about the possibility of an AI takeover, stating that "it's not inconceivable" that AI could "wipe out humanity". [32] Hinton states that AI systems capable of intelligent agency will be useful for military or economic purposes. [75] He worries that generally intelligent AI systems could "create sub-goals" that are unaligned with their programmers' interests. [76] He states that AI systems may become power-seeking or prevent themselves from being shut off, not because programmers intended them to, but because those sub-goals are useful for achieving later goals. [74] In particular, Hinton says "we have to think hard about how to control" AI systems capable of self-improvement. [77]
Hinton reports concerns about deliberate misuse of AI by malicious actors, stating that "it is hard to see how you can prevent the bad actors from using [AI] for bad things." [31] In 2017, Hinton called for an international ban on lethal autonomous weapons. [78]
Hinton was previously optimistic about the economic effects of AI, noting in 2018 that: "The phrase 'artificial general intelligence' carries with it the implication that this sort of single robot is suddenly going to be smarter than you. I don't think it's going to be that. I think more and more of the routine things we do are going to be replaced by AI systems." [79] Hinton also previously argued that AGI won't make humans redundant: "[AI in the future is] going to know a lot about what you're probably going to want to do... But it's not going to replace you." [79]
In 2023, however, Hinton became "worried that AI technologies will in time upend the job market" and take away more than just "drudge work." [31] He again stated in 2024 that the British government will have to establish a universal basic income to deal with the impact of AI on inequality. [80] In Hinton's view, AI will boost productivity and generate more wealth. But unless the government intervenes, it will only make the rich richer and hurt the people who might lose their jobs. "That's going to be very bad for society," he said. [81]
Hinton moved from the U.S. to Canada in part due to disillusionment with Ronald Reagan-era politics and disapproval of military funding of artificial intelligence. [36]
In August 2024, Hinton co-authored a letter with Yoshua Bengio, Stuart Russell, and Laurence Lessig in support of SB 1047, a California AI safety bill that would require companies training models which cost more than $100 million to perform risk assessments before deployment. They claimed the legislation was the "bare minimum for effective regulation of this technology." [82] [83]
Hinton's second wife, Rosalind Zalin, died of ovarian cancer in 1994; his third wife, Jackie, died in September 2018, also of cancer. [84]
Hinton is the great-great-grandson of the mathematician and educator Mary Everest Boole and her husband, the logician George Boole. [85] George Boole's work eventually became one of the foundations of modern computer science. Another great-great-grandfather of his was the surgeon and author James Hinton, [86] who was the father of the mathematician Charles Howard Hinton.
Hinton's father was the entomologist Howard Hinton. [1] [87] His middle name comes from another relative, George Everest, the Surveyor General of India after whom the mountain is named. [36] He is the nephew of the economist Colin Clark. [88]
Artificial intelligence (AI), in its broadest sense, is intelligence exhibited by machines, particularly computer systems. It is a field of research in computer science that develops and studies methods and software that enable machines to perceive their environment and use learning and intelligence to take actions that maximize their chances of achieving defined goals. Such machines may be called AIs.
In machine learning, a neural network is a model inspired by the structure and function of biological neural networks in animal brains.
Jürgen Schmidhuber is a German computer scientist noted for his work in the field of artificial intelligence, specifically artificial neural networks. He is a scientific director of the Dalle Molle Institute for Artificial Intelligence Research in Switzerland. He is also director of the Artificial Intelligence Initiative and professor of the Computer Science program in the Computer, Electrical, and Mathematical Sciences and Engineering (CEMSE) division at the King Abdullah University of Science and Technology (KAUST) in Saudi Arabia.
A Boltzmann machine, named after Ludwig Boltzmann is a stochastic spin-glass model with an external field, i.e., a Sherrington–Kirkpatrick model, that is a stochastic Ising model. It is a statistical physics technique applied in the context of cognitive science. It is also classified as a Markov random field.
In machine learning, backpropagation is a gradient estimation method commonly used for training neural networks to compute the network parameter updates.
A feedforward neural network (FNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. Its flow is uni-directional, meaning that the information in the model flows in only one direction—forward—from the input nodes, through the hidden nodes and to the output nodes, without any cycles or loops, in contrast to recurrent neural networks, which have a bi-directional flow. Modern feedforward networks are trained using the backpropagation method and are colloquially referred to as the "vanilla" neural networks.
David Everett Rumelhart was an American psychologist who made many contributions to the formal analysis of human cognition, working primarily within the frameworks of mathematical psychology, symbolic artificial intelligence, and parallel distributed processing. He also admired formal linguistic approaches to cognition, and explored the possibility of formulating a formal grammar to capture the structure of stories.
Peter Dayan is a British neuroscientist and computer scientist who is director at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany, along with Ivan De Araujo. He is co-author of Theoretical Neuroscience, an influential textbook on computational neuroscience. He is known for applying Bayesian methods from machine learning and artificial intelligence to understand neural function and is particularly recognized for relating neurotransmitter levels to prediction errors and Bayesian uncertainties. He has pioneered the field of reinforcement learning (RL) where he helped develop the Q-learning algorithm, and made contributions to unsupervised learning, including the wake-sleep algorithm for neural networks and the Helmholtz machine.
The Canadian Institute for Advanced Research (CIFAR) is a Canadian-based global research organization that brings together teams of top researchers from around the world to address important and complex questions. It was founded in 1982 and is supported by individuals, foundations and corporations, as well as funding from the Government of Canada and the provinces of Alberta and Quebec.
Yann André LeCun is a French-American computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta.
Kunihiko Fukushima is a Japanese computer scientist, most noted for his work on artificial neural networks and deep learning. He is currently working part-time as a senior research scientist at the Fuzzy Logic Systems Institute in Fukuoka, Japan.
The AAAI Conference on Artificial Intelligence (AAAI) is a leading international academic conference in artificial intelligence held annually. It ranks 4th in terms of H5 Index in Google Scholar's list of top AI publications, after ICLR, NeurIPS, and ICML. It is supported by the Association for the Advancement of Artificial Intelligence. Precise dates vary from year to year, but paper submissions are generally due at the end of August to beginning of September, and the conference is generally held during the following February. The first AAAI was held in 1980 at Stanford University, Stanford California.
There are many types of artificial neural networks (ANN).
Deep learning is the subset of machine learning methods based on neural networks with representation learning. The adjective "deep" refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.
Google Brain was a deep learning artificial intelligence research team under the umbrella of Google AI, a research division at Google dedicated to artificial intelligence. Formed in 2011, it combined open-ended machine learning research with information systems and large-scale computing resources. It created tools such as TensorFlow, which allow neural networks to be used by the public, and multiple internal AI research projects, and aimed to create research opportunities in machine learning and natural language processing. It was merged into former Google sister company DeepMind to form Google DeepMind in April 2023.
Yoshua Bengio is a Canadian computer scientist, most noted for his work on artificial neural networks and deep learning. He is a professor at the Department of Computer Science and Operations Research at the Université de Montréal and scientific director of the Montreal Institute for Learning Algorithms (MILA).
This page is a timeline of machine learning. Major discoveries, achievements, milestones and other major events in machine learning are included.
Ilya Sutskever is a computer scientist who specializes in machine learning.
AlexNet is the name of a convolutional neural network (CNN) architecture, designed by Alex Krizhevsky in collaboration with Ilya Sutskever and Geoffrey Hinton, who was Krizhevsky's Ph.D. advisor at the University of Toronto.
Artificial neural networks (ANNs) are models created using machine learning to perform a number of tasks. Their creation was inspired by neural circuitry. While some of the computational implementations ANNs relate to earlier discoveries in mathematics, the first implementation of ANNs was by psychologist Frank Rosenblatt, who developed the perceptron. Little research was conducted on ANNs in the 1970s and 1980s, with the AAAI calling that period an "AI winter".
"All text published under the heading 'Biography' on Fellow profile pages is available under Creative Commons Attribution 4.0 International License." -- "Royal Society Terms, conditions and policies". Archived from the original on 11 November 2016. Retrieved 9 March 2016.