Human Compatible: Artificial Intelligence and the Problem of Control

Last updated
Author Stuart J. Russell
CountryUnited States
LanguageEnglish
Subject AI control problem
GenreNon-fiction
PublisherViking
Publication date
October 8, 2019
Pages352
ISBN 978-0-525-55861-3
OCLC 1083694322

Human Compatible: Artificial Intelligence and the Problem of Control (shortened to Human Compatible) is a 2019 non-fiction book by computer scientist Stuart J. Russell. It asserts that risk to humanity from advanced artificial intelligence (AI) is a serious concern despite the uncertainty surrounding future progress in AI.

Contents


Russell's three principles

Russell lists three principles to guide the development of beneficial machines. He emphasizes that these principles are not meant to be explicitly coded into the machines; rather, they are intended for the human developers. The principles are as follows: [1] :173

1. The machine's only objective is to maximize the realization of human preferences.

2. The machine is initially uncertain about what those preferences are.

3. The ultimate source of information about human preferences is human behavior.

The "preferences" Russell refers to "are all-encompassing; they cover everything you might care about, arbitrarily far into the future." [1] :173 Similarly, "behavior" includes any choice between options. [1] :177

Reception

Human Compatible was reviewed by Ian Sample in The Guardian , [2] James McConnachie in The Times , [3] Matthew Hutson in The Wall Street Journal , [4] and blogger Scott Alexander. [5]

Human Compatible was criticized in a New York Times opinion essay by Melanie Mitchell, arguing that AI risk cannot be seriously engaged with until intelligence is better understood. [6] However, it was praised by Ned Desmond in TechCrunch, calling it "a carefully written explanation of the concepts underlying AI as well as the history of their development." [7]

See also

Related Research Articles

Artificial intelligence Intelligence demonstrated by machines

In computer science, artificial intelligence (AI), sometimes called machine intelligence, is intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans. Leading AI textbooks define the field as the study of "intelligent agents": any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. Colloquially, the term "artificial intelligence" is often used to describe machines that mimic "cognitive" functions that humans associate with the human mind, such as "learning" and "problem solving".

Eliezer Yudkowsky American blogger, writer, and artificial intelligence researcher

Eliezer Shlomo Yudkowsky is an American AI researcher and writer best known for popularising the idea of friendly artificial intelligence. He is a co-founder and research fellow at the Machine Intelligence Research Institute (MIRI), a private research nonprofit based in Berkeley, California. His work on the prospect of a runaway intelligence explosion was an influence on Nick Bostrom's Superintelligence: Paths, Dangers, Strategies.

A friendly artificial intelligence is a hypothetical artificial general intelligence (AGI) that would have a positive effect on humanity. It is a part of the ethics of artificial intelligence and is closely related to machine ethics. While machine ethics is concerned with how an artificially intelligent agent should behave, friendly artificial intelligence research is focused on how to practically bring about this behaviour and ensuring it is adequately constrained.

Nick Bostrom Swedish philosopher and author

Nick Bostrom is a Swedish philosopher at the University of Oxford known for his work on existential risk, the anthropic principle, human enhancement ethics, superintelligence risks, and the reversal test. In 2011, he founded the Oxford Martin Programme on the Impacts of Future Technology, and is the founding director of the Future of Humanity Institute at Oxford University. In 2009 and 2015, he was included in Foreign Policy's Top 100 Global Thinkers list.

Stuart J. Russell Computer scientist and author

Stuart Jonathan Russell is a computer scientist known for his contributions to artificial intelligence. He is a Professor of Computer Science at the University of California, Berkeley and Adjunct Professor of Neurological Surgery at the University of California, San Francisco. He holds the Smith-Zadeh Chair in Engineering at Berkeley University. He founded and leads the Center for Human-Compatible Artificial Intelligence (CHAI) at UC Berkeley.

Artificial general intelligence (AGI) is the intelligence of a machine that can understand or learn any intellectual task that a human being can. It is a primary goal of some artificial intelligence research and a common topic in science fiction and futures studies. AGI can also be referred to as strong AI, full AI, or general intelligent action.

The Machine Intelligence Research Institute (MIRI), formerly the Singularity Institute for Artificial Intelligence (SIAI), is a non-profit organization focused since 2005 on identifying and managing potential existential risks from artificial general intelligence. MIRI's work has focused on a friendly AI approach to system design and on predicting the rate of technology development.

Synthetic intelligence (SI) is an alternative term for artificial intelligence which emphasizes that the intelligence of machines need not be an imitation or in any way artificial; it can be a genuine form of intelligence. John Haugeland proposes an analogy with simulated diamonds and synthetic diamonds—only the synthetic diamond is truly a diamond. Synthetic means that which is produced by synthesis; combining parts to form a whole, colloquially, a man-made version of that which has arisen naturally. As defined, a "synthetic intelligence" would therefore be man-made, but not a simulation.

Artificial intelligence has close connections with philosophy because both share several concepts and these include intelligence, action, consciousness, epistemology, and even free will. Furthermore, the technology is concerned with the creation of artificial animals or artificial people so the discipline is of considerable interest to philosophers. These factors contributed to the emergence of the philosophy of artificial intelligence. Some scholars argue that the AI community's dismissal of philosophy is detrimental.

The ethics of artificial intelligence is the part of the ethics of technology specific to robots and other artificially intelligent beings. It is typically divided into roboethics, a concern with the moral behavior of humans as they design, construct, use and treat artificially intelligent beings, and machine ethics, which is concerned with the moral behavior of artificial moral agents (AMAs).

Progress in artificial intelligence

Artificial intelligence applications have been used in a wide range of fields including medical diagnosis, stock trading, robot control, law, scientific discovery and toys. However, many AI applications are not perceived as AI: "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore." "Many thousands of AI applications are deeply embedded in the infrastructure of every industry." In the late 1990s and early 21st century, AI technology became widely used as elements of larger systems, but the field is rarely credited for these successes.

Eric Horvitz American computer scientist

Eric Joel Horvitz is an American computer scientist, and Technical Fellow at Microsoft, where he serves as director of Microsoft Research Labs, including research centers in Redmond, WA, Cambridge, Massachusetts, New York, NY, Montreal, Canada, Cambridge, UK, and Bangalore, India.

Machine ethics is a part of the ethics of artificial intelligence concerned with adding moral behaviors to machines which use artificial intelligence, otherwise known as artificial intelligent agents. Machine ethics differs from other ethical fields related to engineering and technology. Machine ethics is a subcategory within roboethics, where roboethics is concerned with the moral behavior of humans as they design, construct, use, and treat such beings. Roboethics also focuses on whether or not machines pose a threat to humanity. Machine ethics should not be confused with computer ethics, which focuses on human use of computers. It should also be distinguished from the philosophy of technology, which concerns itself with the grander social effects of technology.

The Allen Institute for AI is a research institute founded by late Microsoft co-founder Paul Allen. The institute seeks to achieve scientific breakthroughs by constructing AI systems with reasoning, learning, and reading capabilities. Oren Etzioni was appointed by Paul Allen in September 2013 to direct the research at the institute.

Vicarious is an artificial intelligence company based in the San Francisco Bay Area, California. They are using the theorized computational principles of the brain to build software that can think and learn like a human.

Future of Life Institute Nonprofit researching existential risk

The Future of Life Institute (FLI) is a volunteer-run research and outreach organization in the Boston area that works to mitigate existential risks facing humanity, particularly existential risk from advanced artificial intelligence (AI). Its founders include MIT cosmologist Max Tegmark and Skype co-founder Jaan Tallinn, and its board of advisors includes entrepreneur Elon Musk.

Existential risk from artificial general intelligence is the hypothesis that substantial progress in artificial general intelligence (AGI) could someday result in human extinction or some other unrecoverable global catastrophe. It is argued that the human species currently dominates other species because the human brain has some distinctive capabilities that other animals lack. If AI surpasses humanity in general intelligence and becomes "superintelligent", then this new superintelligence could become powerful and difficult to control. Just as the fate of the mountain gorilla depends on human goodwill, so might the fate of humanity depend on the actions of a future machine superintelligence.

OpenAI is an independent research organization consisting of the for-profit corporation OpenAI LP and its parent organization, the non-profit OpenAI Inc. The corporation conducts research in the field of artificial intelligence (AI) with the stated aim to promote and develop friendly AI in such a way as to benefit humanity as a whole. Founded in late 2015, the San Francisco-based organization aims to “freely collaborate” with other institutions and researchers by making its patents and research open to the public. The founders are motivated in part by concerns about the existential risk from artificial general intelligence.

<i>Life 3.0</i>

Life 3.0: Being Human in the Age of Artificial Intelligence is a book by Swedish-American cosmologist Max Tegmark from MIT. Life 3.0 discusses Artificial Intelligence (AI) and its impact on the future of life on Earth and beyond. The book discusses a variety of societal implications, what can be done to maximize the chances of a positive outcome, and potential futures for humanity, technology and combinations thereof.

The Center for Human-Compatible Artificial Intelligence (CHAI) is a research center at University of California, Berkeley focusing on advanced artificial intelligence (AI) safety methods. CHAI was founded in 2016 by a group of academics led by UC Berkeley computer science professor and AI author Stuart J. Russell. Russell is known for co-authoring the widely used AI textbook Artificial Intelligence: A Modern Approach.

References

  1. 1 2 3 Russell, Stuart (October 8, 2019). Human Compatible: Artificial Intelligence and the Problem of Control. United States: Viking. ISBN   978-0-525-55861-3. OCLC   1083694322.
  2. Sample, Ian (October 24, 2019). "Human Compatible by Stuart Russell review – AI and our future". The Guardian .
  3. McConnachie, James (October 6, 2019). "Human Compatible by Stuart Russell review — an AI expert's chilling warning". The Times .
  4. Hutson, Matthew (November 19, 2019). "'Human Compatible' and 'Artificial Intelligence' Review: Learn Like a Machine". The Wall Street Journal .
  5. Alexander, Scott (January 30, 2020). "Book Review: Human Compatible".
  6. Mitchell, Melanie (October 31, 2019). "We Shouldn't be Scared by 'Superintelligent A.I.'". The New York Times .
  7. Desmond, Ned (October 6, 2019). "'Human Compatible' is a provocative prescription to re-think AI before it's too late". TechCrunch .