DARPA TIPSTER Program

Last updated

The DARPA TIPSTER Text program was started in 1991 by the Defense Advanced Research Projects Agency (DARPA). It is a 9-year multi-million dollar initiative, which sought to improve HLT for the handling of multilingual corpora that are utilized within the intelligence process. [1] It involved a cluster of joint projects by the government, academia, and private sector. [2]

Contents

The program supported research to improve informational retrieval and extraction software and worked to deploy these improved technologies to government users. This technology, which was of particular interest to defense and intelligence analysts who must review increasingly large amounts of text. The program had several phases. The first entailed the development of algorithms for information retrieval and extraction while the second phase developed an architecture. [1]

The program was considered successful so that it was commercialized under the National Institute of Standards and Technology. [2] An evaluation noted that the third phase of the TIPSTER program, which involved the development of the architecture called GATE (General Architecture for Text Engineering) did not achieve its intended goals due to its short life span as well as the inability of the government to enforce standards imposed by the TIPSTER software architecture. [3]

Related Research Articles

In computing, a compiler is a computer program that translates computer code written in one programming language into another language. The name "compiler" is primarily used for programs that translate source code from a high-level programming language to a lower level language to create an executable program.

DARPA Agency of the U.S. Department of Defense responsible for the development of new technologies

The Defense Advanced Research Projects Agency (DARPA) is a research and development agency of the United States Department of Defense responsible for the development of emerging technologies for use by the military.

Information retrieval (IR) is the process of obtaining information system resources that are relevant to an information need from a collection of those resources. Searches can be based on full-text or other content-based indexing. Information retrieval is the science of searching for information in a document, searching for documents themselves, and also searching for the metadata that describes data, and for databases of texts, images or sounds.

Data mining Finding patterns in large data sets using complex computational methods

Data mining is a process of extracting and discovering patterns in large data sets involving methods at the intersection of machine learning, statistics, and database systems. Data mining is an interdisciplinary subfield of computer science and statistics with an overall goal to extract information from a data set and transform the information into a comprehensible structure for further use. Data mining is the analysis step of the "knowledge discovery in databases" process, or KDD. Aside from the raw analysis step, it also involves database and data management aspects, data pre-processing, model and inference considerations, interestingness metrics, complexity considerations, post-processing of discovered structures, visualization, and online updating.

Information Awareness Office DARPA division overseeing the "Total Information Awareness" program

The Information Awareness Office (IAO) was established by the United States Defense Advanced Research Projects Agency (DARPA) in January 2002 to bring together several DARPA projects focused on applying surveillance and information technology to track and monitor terrorists and other asymmetric threats to U.S. national security by achieving "Total Information Awareness" (TIA).

Total Information Awareness US mass detection program

Total Information Awareness (TIA) was a mass detection program by the United States Information Awareness Office. It operated under this title from February to May 2003 before being renamed Terrorism Information Awareness.

Text mining, also referred to as text data mining, similar to text analytics, is the process of deriving high-quality information from text. It involves "the discovery by computer of new, previously unknown information, by automatically extracting information from different written resources." Written resources may include websites, books, emails, reviews, and articles. High-quality information is typically obtained by devising patterns and trends by means such as statistical pattern learning. According to Hotho et al. (2005) we can differ three different perspectives of text mining: information extraction, data mining, and a KDD process. Text mining usually involves the process of structuring the input text, deriving patterns within the structured data, and finally evaluation and interpretation of the output. 'High quality' in text mining usually refers to some combination of relevance, novelty, and interest. Typical text mining tasks include text categorization, text clustering, concept/entity extraction, production of granular taxonomies, sentiment analysis, document summarization, and entity relation modeling.

Barry Boehm

Barry W. Boehm is an American software engineer, distinguished professor of computer science, industrial and systems engineering; the TRW Professor of Software Engineering; and founding director of the Center for Systems and Software Engineering at the University of Southern California. He is known for his many contributions to the area of software engineering.

Question answering (QA) is a computer science discipline within the fields of information retrieval and natural language processing (NLP), which is concerned with building systems that automatically answer questions posed by humans in a natural language.

Information extraction (IE) is the task of automatically extracting structured information from unstructured and/or semi-structured machine-readable documents and other electronically represented sources. In most of the cases this activity concerns processing human language texts by means of natural language processing (NLP). Recent activities in multimedia document processing like automatic annotation and content extraction out of images/audio/video/documents could be seen as information extraction

Text Retrieval Conference

The Text REtrieval Conference (TREC) is an ongoing series of workshops focusing on a list of different information retrieval (IR) research areas, or tracks. It is co-sponsored by the National Institute of Standards and Technology (NIST) and the Intelligence Advanced Research Projects Activity, and began in 1992 as part of the TIPSTER Text program. Its purpose is to support and encourage research within the information retrieval community by providing the infrastructure necessary for large-scale evaluation of text retrieval methodologies and to increase the speed of lab-to-product transfer of technology.

Legal informatics is an area within information science.

The Alvey Programme was a British government sponsored research program in information technology that ran from 1984 to 1990. The program was a reaction to the Japanese Fifth Generation project, which aimed to create a computer using massively parallel computing/processing. The program was not focused any specific technology such as robotics, but rather supported research in knowledge engineering in the United Kingdom. It has been likened in operations to the U.S. Defense Advanced Research Projects Agency (DARPA) and Japan's ICOT.

The Message Understanding Conferences (MUC) were initiated and financed by DARPA to encourage the development of new and better methods of information extraction. The character of this competition—many concurrent research teams competing against one another—required the development of standards for evaluation, e.g. the adoption of metrics like precision and recall.

Query expansion (QE) is the process of reformulating a given query to improve retrieval performance in information retrieval operations, particularly in the context of query understanding. In the context of search engines, query expansion involves evaluating a user's input and expanding the search query to match additional documents. Query expansion involves techniques such as:

RetrievalWare is an enterprise search engine emphasizing natural language processing and semantic networks which was commercially available from 1992 to 2007 and is especially known for its use by government intelligence agencies.

Dr. Robert L. Simpson Jr. is a computer scientist whose primary research interest is applied artificial intelligence. He served as Chief Scientist at Applied Systems Intelligence, Inc. (ASI) working with Dr. Norman D. Geddes, CEO. Dr. Simpson was responsible for the creation of the ASI core technology PreAct. ASI has since changed its name to Veloxiti Inc.

Apache cTAKES

Apache cTAKES: clinical Text Analysis and Knowledge Extraction System is an open-source Natural Language Processing (NLP) system that extracts clinical information from electronic health record unstructured text. It processes clinical notes, identifying types of clinical named entities — drugs, diseases/disorders, signs/symptoms, anatomical sites and procedures. Each named entity has attributes for the text span, the ontology mapping code, context, and negated/not negated.

The following outline is provided as an overview of and topical guide to natural language processing:

NetOwl is a suite of multilingual text and identity analytics products that analyze big data in the form of text data – reports, web, social media, etc. – as well as structured entity data about people, organizations, places, and things.

References

  1. 1 2 Métais, Elisabeth; Meziane, Farid; Sararee, Mohamad; Sugumaran, Vijayan; Vadera, Sunil (2013). Natural Language Processing and Information Systems: 18th International Conference on Applications of Natural Language to Information Systems, NLDB 2013, Salford, UK, Proceedings. Heidelberg: Springer. p. 261. ISBN   978-3-642-38823-1.
  2. 1 2 Treverton, Gregory F.; Gabbard, C. Bryan (2008). Assessing the Tradecraft of Intelligence Analysis. Santa Monica, CA: Rand Corporation. p. 24. ISBN   0-8330-3958-X.
  3. TIPSTER Text Program, Phase III: Proceedings, October 1996-October 1998. Morgan Kaufmann. 1999. p. 10. ISBN   978-1-55860-610-4.

See also