Never-Ending Language Learning

Last updated

Never-Ending Language Learning system (NELL) is a semantic machine learning system that as of 2010 was being developed by a research team at Carnegie Mellon University, and supported by grants from DARPA, Google, NSF, and CNPq with portions of the system running on a supercomputing cluster provided by Yahoo!. [1]

Contents

Process and goals

NELL was programmed by its developers to be able to identify a basic set of fundamental semantic relationships between a few hundred predefined categories of data, such as cities, companies, emotions and sports teams. Since the beginning of 2010, the Carnegie Mellon research team has been running NELL around the clock, sifting through hundreds of millions of web pages looking for connections between the information it already knows and what it finds through its search process – to make new connections in a manner that is intended to mimic the way humans learn new information. [2] For example, in encountering the word pair "Pikes Peak", NELL would notice that both words are capitalized and deduce from the second word that it was the name of a mountain, and then build on the relationship of words surrounding those two words to deduce other connections. [1]

The goal of NELL and other semantic learning systems, such as IBM's Watson system, is to be able to develop means of answering questions posed by users in natural language with no human intervention in the process. [3] Oren Etzioni of the University of Washington lauded the system's "continuous learning, as if NELL is exercising curiosity on its own, with little human help". [1]

By October 2010, NELL has doubled the number of relationships it has available in its knowledge base and has learned 440,000 new facts, with an accuracy of 87%. [4] [1] Team leader Tom M. Mitchell, chairman of the machine learning department at Carnegie Mellon described how NELL "self-corrects when it has more information, as it learns more", though it does sometimes arrive at incorrect conclusions. Accumulated errors, such as the deduction that Internet cookies were a kind of baked good, led NELL to deduce from the phrases "I deleted my Internet cookies" and "I deleted my files" that "computer files" also belonged in the baked goods category. [5] Clear errors like these are[ when? ] corrected every few weeks by the members of the research team and the system is allowed to continue its learning process. [1] By 2018, NELL had "acquired a knowledge base with 120mn diverse, confidence-weighted beliefs (e.g., servedWith(tea,biscuits)), while learning thousands of interrelated functions that continually improve its reading competence over time." [6]

As of September 2023, the project's most recently gathered facts dated from February 2019 (according to its Twitter feed) [7] or September 2018 (according to its home page). [8]

Reception

In his 2019 book "Human Compatible", Stuart Russell commented that 'Unfortunately NELL has confidence in only 3 percent of its beliefs and relies on human experts to clean out false or meaningless beliefs on a regular basis—such as its beliefs that “Nepal is a country also known as United States” and "value is an agricultural product that is usually cut into basis."' [9] A 2023 paper commented that "While the never-ending part seems like the right approach, NELL still had the drawback that its focus remained much too grounded on object-language descriptions, and relied on web pages as its only source, which significantly influenced the type of grammar, symbolism, slang, etc. analysed." [10]

See also

Related Research Articles

<span class="mw-page-title-main">Carnegie Mellon University</span> Private research university in Pittsburgh, Pennsylvania, U.S.

Carnegie Mellon University (CMU) is a private research university in Pittsburgh, Pennsylvania. The institution was originally established in 1900 by Andrew Carnegie as the Carnegie Technical Schools. In 1912, it became the Carnegie Institute of Technology and began granting four-year degrees. In 1967, it became the current-day Carnegie Mellon University through its merger with the Mellon Institute of Industrial Research, founded in 1913 by Andrew Mellon and Richard B. Mellon and formerly a part of the University of Pittsburgh.

<span class="mw-page-title-main">Carnegie Mellon School of Computer Science</span> School for computer science in the United States

The School of Computer Science (SCS) at Carnegie Mellon University in Pittsburgh, Pennsylvania, US is a school for computer science established in 1988. It has been consistently ranked among the top computer science programs over the decades. As of 2022 U.S. News & World Report ranks the graduate program as tied for second with Stanford University and University of California, Berkeley. It is ranked second in the United States on Computer Science Open Rankings, which combines scores from multiple independent rankings.

<span class="mw-page-title-main">Raj Reddy</span> Indian-American computer scientist (born 1937)

Dabbala Rajagopal "Raj" Reddy is an Indian-born American computer scientist and a winner of the Turing Award. He is one of the early pioneers of artificial intelligence and has served on the faculty of Stanford and Carnegie Mellon for over 50 years. He was the founding director of the Robotics Institute at Carnegie Mellon University. He was instrumental in helping to create Rajiv Gandhi University of Knowledge Technologies in India, to cater to the educational needs of the low-income, gifted, rural youth. He is the chairman of International Institute of Information Technology, Hyderabad. He is the first person of Asian origin to receive the Turing Award, in 1994, known as the Nobel Prize of Computer Science, for his work in the field of artificial intelligence.

<span class="mw-page-title-main">School of Informatics, University of Edinburgh</span>

The School of Informatics is an academic unit of the University of Edinburgh, in Scotland, responsible for research, teaching, outreach and commercialisation in informatics. It was created in 1998 from the former department of artificial intelligence, the Centre for Cognitive Science and the department of computer science, along with the Artificial Intelligence Applications Institute (AIAI) and the Human Communication Research Centre.

<span class="mw-page-title-main">Mellon Institute of Industrial Research</span> United States historic place

The Mellon Institute of Industrial Research was a research institute in Pittsburgh, Pennsylvania that became part of Carnegie Mellon University. It was founded in 1913 by Andrew Mellon and Richard B. Mellon as part of the University of Pittsburgh, and was originally located in Allen Hall. After becoming an independent research center and moving to a new building on Fifth Avenue in 1937, the Mellon Institute merged with the Carnegie Institute of Technology in 1967 to form Carnegie Mellon University. While it ceased to exist as a distinct institution, the landmark building bearing its name remains located at the corner of Fifth Avenue and Bellefield Avenue in Oakland, the city's university district. It is sited adjacent to The Carnegie Mellon Software Engineering Institute (SEI) and the University of Pittsburgh's Bellefield Hall and is across Bellefield Avenue from two other local landmarks: the University of Pittsburgh's Heinz Memorial Chapel and the Cathedral of Learning.

Microsoft Research (MSR) is the research subsidiary of Microsoft. It was created in 1991 by Richard Rashid, Bill Gates and Nathan Myhrvold with the intent to advance state-of-the-art computing and solve difficult world problems through technological innovation in collaboration with academic, government, and industry researchers. The Microsoft Research team has more than 1,000 computer scientists, physicists, engineers, and mathematicians, including Turing Award winners, Fields Medal winners, MacArthur Fellows, and Dijkstra Prize winners.

John Robert Anderson is a Canadian-born American psychologist. He is currently professor of Psychology and Computer Science at Carnegie Mellon University.

Robot learning is a research field at the intersection of machine learning and robotics. It studies techniques allowing a robot to acquire novel skills or adapt to its environment through learning algorithms. The embodiment of the robot, situated in a physical embedding, provides at the same time specific difficulties and opportunities for guiding the learning process.

<span class="mw-page-title-main">AI winter</span> Period of reduced funding and interest in AI research

In the history of artificial intelligence, an AI winter is a period of reduced funding and interest in artificial intelligence research. The field has experienced several hype cycles, followed by disappointment and criticism, followed by funding cuts, followed by renewed interest years or even decades later.

The Pittsburgh Supercomputing Center (PSC) is a high performance computing and networking center founded in 1986 and one of the original five NSF Supercomputing Centers. PSC is a joint effort of Carnegie Mellon University and the University of Pittsburgh in Pittsburgh, Pennsylvania, United States.

Ekaterini Panagiotou Sycara is a Greek computer scientist. She is an Edward Fredkin Research Professor of Robotics in the Robotics Institute, School of Computer Science at Carnegie Mellon University internationally known for her research in artificial intelligence, particularly in the fields of negotiation, autonomous agents and multi-agent systems. She directs the Advanced Agent-Robotics Technology Lab at Robotics Institute, Carnegie Mellon University. She also serves as academic advisor for PhD students at both Robotics Institute and Tepper School of Business.

A relationship extraction task requires the detection and classification of semantic relationship mentions within a set of artifacts, typically from text or XML documents. The task is very similar to that of information extraction (IE), but IE additionally requires the removal of repeated relations (disambiguation) and generally refers to the extraction of many different relationships.

Alexander G. Hauptmann is a Research Professor in the Language Technologies Institute at the Carnegie Mellon University School of Computer Science. He has been the leader of the Informedia Digital Library which has made seminal strides in multimedia information retrieval and won best paper awards at major conferences. He was also a founder of the international advisory committee for TRECVID.

CALO was an artificial intelligence project that attempted to integrate numerous AI technologies into a cognitive assistant. CALO is an acronym for "Cognitive Assistant that Learns and Organizes". The name was inspired by the Latin word "Calo" which means "soldier's servant". The project started in May 2003 and ran for five years, ending in 2008.

The Language Technologies Institute (LTI) is a research institute at Carnegie Mellon University in Pittsburgh, Pennsylvania, United States, and focuses on the area of language technologies. The institute is home to 33 faculty with the primary scholarly research of the institute focused on machine translation, speech recognition, speech synthesis, information retrieval, parsing, information extraction, and multimodal machine learning. Until 1996, the institute existed as the Center for Machine Translation, which was established in 1986. Subsequently, from 1996 onwards, it started awarding degrees, and the name was changed to The Language Technologies Institute. The institute was founded by Professor Jaime Carbonell, who served as director until his death in February 2020. He was followed by Jamie Callan, and then Carolyn Rosé, as interim directors. In August 2023, Mona Diab became the director of the institute.

<span class="mw-page-title-main">Manuela M. Veloso</span> Portuguese-American computer scientist

Manuela Maria Veloso is the Head of J.P. Morgan AI Research & Herbert A. Simon University Professor Emeritus in the School of Computer Science at Carnegie Mellon University, where she was previously Head of the Machine Learning Department. She served as president of Association for the Advancement of Artificial Intelligence (AAAI) until 2014, and the co-founder and a Past President of the RoboCup Federation. She is a fellow of AAAI, Institute of Electrical and Electronics Engineers (IEEE), American Association for the Advancement of Science (AAAS), and Association for Computing Machinery (ACM). She is an international expert in artificial intelligence and robotics.

The following outline is provided as an overview of and topical guide to natural-language processing:

<span class="mw-page-title-main">Russ Salakhutdinov</span> Canadian AI researcher

Ruslan "Russ" Salakhutdinov is a Canadian researcher of Tatar origin working in the field of artificial intelligence.

<span class="mw-page-title-main">Hideto Tomabechi</span> Japanese cognitive scientist and computer scientist

Hideto Tomabechi is a Japanese cognitive scientist computer scientist.

References

  1. 1 2 3 4 5 "Aiming to Learn as We Do, a Machine Teaches Itself". New York Times . October 4, 2010. Retrieved 2010-10-05. Since the start of the year, a team of researchers at Carnegie Mellon University — supported by grants from the Defense Advanced Research Projects Agency and Google, and tapping into a research supercomputing cluster provided by Yahoo — has been fine-tuning a computer system that is trying to master semantics by learning more like a human.
  2. ↑ Project Overview, Carnegie Mellon University. Accessed October 5, 2010.
  3. ↑ Trader, Tiffany. "Machine Learns Language Starting with the Facts", HPCwire, October 5, 2010. Accessed October 5, 2010.
  4. ↑ "NELL: Never-Ending Language Learning", Carnegie Mellon University. Accessed October 5, 2010.
  5. ↑ VanHemert, Kyle. "Right Now A Computer Is Reading Online, Teaching Itself Language", Gizmodo, October 6, 2010. Accessed October 5, 2010.
  6. ↑ Mitchell, T.; Cohen, W.; Hruschka, E.; Talukdar, P.; Yang, B.; Betteridge, J.; Carlson, A.; Dalvi, B.; Gardner, M.; Kisiel, B.; Krishnamurthy, J.; Lao, N.; Mazaitis, K.; Mohamed, T.; Nakashole, N. (2018-04-24). "Never-ending learning". Communications of the ACM. 61 (5): 103–115. doi: 10.1145/3191513 . ISSN   0001-0782.
  7. ↑ "NELL (@cmunell) | Twitter". twitter.com. Retrieved 2023-09-04.
  8. ↑ "Read the Web :: Carnegie Mellon University". rtw.ml.cmu.edu. Retrieved 2023-09-04.
  9. ↑ Russell, Stuart (2019). "3". Human Compatible: AI and the Problem of Control. Allen Lane.
  10. ↑ de Jager, S. (2023-04-11). "Semantic noise in the Winograd Schema Challenge of pronoun disambiguation". Humanities and Social Sciences Communications. 10 (1): 1–10. doi: 10.1057/s41599-023-01643-9 . ISSN   2662-9992.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.