Lexical Markup Framework

Last updated

Language resource management - Lexical markup framework (LMF; ISO 24613:2008), is the ISO International Organization for Standardization ISO/TC37 standard for natural language processing (NLP) and machine-readable dictionary (MRD) lexicons. [1] The scope is standardization of principles and methods relating to language resources in the contexts of multilingual communication.

Contents

Objectives

The goals of LMF are to provide a common model for the creation and use of lexical resources, to manage the exchange of data between and among these resources, and to enable the merging of large number of individual electronic resources to form extensive global electronic resources.

Types of individual instantiations of LMF can include monolingual, bilingual or multilingual lexical resources. The same specifications are to be used for both small and large lexicons, for both simple and complex lexicons, for both written and spoken lexical representations. The descriptions range from morphology, syntax, computational semantics to computer-assisted translation. The covered languages are not restricted to European languages but cover all natural languages. The range of targeted NLP applications is not restricted. LMF is able to represent most lexicons, including WordNet, EDR and PAROLE lexicons.

History

In the past, lexicon standardization has been studied and developed by a series of projects like GENELEX, EDR, EAGLES, MULTEXT, PAROLE, SIMPLE and ISLE. Then, the ISO/TC37 National delegations decided to address standards dedicated to NLP and lexicon representation. The work on LMF started in Summer 2003 by a new work item proposal issued by the US delegation. In Fall 2003, the French delegation issued a technical proposition for a data model dedicated to NLP lexicons. In early 2004, the ISO/TC37 committee decided to form a common ISO project with Nicoletta Calzolari (CNR-ILC Italy) as convenor and Gil Francopoulo (Tagmatica France) and Monte George (ANSI USA) as editors. The first step in developing LMF was to design an overall framework based on the general features of existing lexicons and to develop a consistent terminology to describe the components of those lexicons. The next step was the actual design of a comprehensive model that best represented all of the lexicons in detail. A large panel of 60 experts contributed a wide range of requirements for LMF that covered many types of NLP lexicons. The editors of LMF worked closely with the panel of experts to identify the best solutions and reach a consensus on the design of LMF. Special attention was paid to the morphology in order to provide powerful mechanisms for handling problems in several languages that were known as difficult to handle. 13 versions have been written, dispatched (to the National nominated experts), commented and discussed during various ISO technical meetings. After five years of work, including numerous face-to-face meetings and e-mail exchanges, the editors arrived at a coherent UML model. In conclusion, LMF should be considered a synthesis of the state of the art in NLP lexicon field.

Current stage

The ISO number is 24613. The LMF specification has been published officially as an International Standard on 17 November 2008.

As one of the members of the ISO/TC37 family of standards

The ISO/TC37 standards are currently elaborated as high level specifications and deal with word segmentation (ISO 24614), annotations (ISO 24611 a.k.a. MAF, ISO 24612 a.k.a. LAF, ISO 24615 a.k.a. SynAF, and ISO 24617-1 a.k.a. SemAF/Time), feature structures (ISO 24610), multimedia containers (ISO 24616 a.k.a. MLIF), and lexicons (ISO 24613). These standards are based on low level specifications dedicated to constants, namely data categories (revision of ISO 12620), language codes (ISO 639), scripts codes (ISO 15924), country codes (ISO 3166) and Unicode (ISO 10646).

The two level organization forms a coherent family of standards with the following common and simple rules:

Key standards

The linguistics constants like /feminine/ or /transitive/ are not defined within LMF but are recorded in the Data Category Registry (DCR) that is maintained as a global resource by ISO/TC37 in compliance with ISO/IEC 11179-3:2003. [2] And these constants are used to adorn the high level structural elements.

The LMF specification complies with the modeling principles of Unified Modeling Language (UML) as defined by Object Management Group (OMG). The structure is specified by means of UML class diagrams. The examples are presented by means of UML instance (or object) diagrams.

An XML DTD is given in an annex of the LMF document.

Model structure

LMF is composed of the following components:

The extensions are specifically dedicated to morphology, MRD, NLP syntax, NLP semantics, NLP multilingual notations, NLP morphological patterns, multiword expression patterns, and constraint expression patterns.

Example

In the following example, the lexical entry is associated with a lemma clergyman and two inflected forms clergyman and clergymen. The language coding is set for the whole lexical resource. The language value is set for the whole lexicon as shown in the following UML instance diagram.

LMFMorphoClergymanInflected.svg

The elements Lexical Resource , Global Information, Lexicon, Lexical Entry, Lemma , and Word Form define the structure of the lexicon. They are specified within the LMF document. On the contrary, languageCoding, language, partOfSpeech , commonNoun, writtenForm, grammaticalNumber , singular, plural are data categories that are taken from the Data Category Registry. These marks adorn the structure. The values ISO 639-3, clergyman, clergymen are plain character strings. The value eng is taken from the list of languages as defined by ISO 639-3.

With some additional information like dtdVersion and feat, the same data can be expressed by the following XML fragment:

<LexicalResourcedtdVersion="15"><GlobalInformation><featatt="languageCoding"val="ISO 639-3"/></GlobalInformation><Lexicon><featatt="language"val="eng"/><LexicalEntry><featatt="partOfSpeech"val="commonNoun"/><Lemma><featatt="writtenForm"val="clergyman"/></Lemma><WordForm><featatt="writtenForm"val="clergyman"/><featatt="grammaticalNumber"val="singular"/></WordForm><WordForm><featatt="writtenForm"val="clergymen"/><featatt="grammaticalNumber"val="plural"/></WordForm></LexicalEntry></Lexicon></LexicalResource>

This example is rather simple, while LMF can represent much more complex linguistic descriptions the XML tagging is correspondingly complex.

Selected publications about LMF

The first publication about the LMF specification as it has been ratified by ISO (this paper became (in 2015) the 9th most cited paper within the Language Resources and Evaluation conferences from LREC papers):

About semantic representation:

About African languages:

About Asian languages:

About European languages:

About Semitic languages:

Dedicated book

There is a book published in 2013: LMF Lexical Markup Framework [12] which is entirely dedicated to LMF. The first chapter deals with the history of lexicon models, the second chapter is a formal presentation of the data model and the third one deals with the relation with the data categories of the ISO-DCR. The other 14 chapters deal with a lexicon or a system, either in the civil or military domain, either within scientific research labs or for industrial applications. These are Wordnet-LMF, Prolmf, DUELME, UBY-LMF, LG-LMF, RELISH, GlobalAtlas (or Global Atlas) and Wordscape.

See also

Related Research Articles

WordNet Computational lexicon of English

WordNet is a lexical database of semantic relations between words in more than 200 languages. WordNet links words into semantic relations including synonyms, hyponyms, and meronyms. The synonyms are grouped into synsets with short definitions and usage examples. WordNet can thus be seen as a combination and extension of a dictionary and thesaurus. While it is accessible to human users via a web browser, its primary use is in automatic text analysis and artificial intelligence applications. WordNet was first created in the English language and the English WordNet database and software tools have been released under a BSD style license and are freely available for download from that WordNet website.

The XML Metadata Interchange (XMI) is an Object Management Group (OMG) standard for exchanging metadata information via Extensible Markup Language (XML).

Semantic lexicon

A semantic lexicon is a digital dictionary of words labeled with semantic classes so associations can be drawn between words that have not previously been encountered. Semantic lexicons are built upon semantic networks, which represent the semantic relations between words. The difference between a semantic lexicon and a semantic network is that a semantic lexicon has definitions for each word, or a "gloss".

RM-ODP

Reference Model of Open Distributed Processing (RM-ODP) is a reference model in computer science, which provides a co-ordinating framework for the standardization of open distributed processing (ODP). It supports distribution, interworking, platform and technology independence, and portability, together with an enterprise architecture framework for the specification of ODP systems.

Linguistic categories include

Computational lexicology is a branch of computational linguistics, which is concerned with the use of computers in the study of lexicon. It has been more narrowly described by some scholars as the use of computers in the study of machine-readable dictionaries. It is distinguished from computational lexicography, which more properly would be the use of computers in the construction of dictionaries, though some researchers have used computational lexicography as synonymous.

ISO/TC 37

ISO/TC 37 is a technical committee within the International Organization for Standardization (ISO) that prepares standards and other documents concerning methodology and principles for terminology and language resources.

Machine-readable dictionary (MRD) is a dictionary stored as machine (computer) data instead of being printed on paper. It is an electronic dictionary and lexical database.

In digital lexicography, natural language processing, and digital humanities, a lexical resource is a language resource consisting of data regarding the lexemes of the lexicon of one or more languages e.g., in the form of a database.

A multilingual notation is a representation in a lexical resource that allows the translation between two or more words.

The Ubiquitous Knowledge Processing Lab is a research lab at the Department of Computer Science at the Technische Universität Darmstadt. It was founded in 2006 by Iryna Gurevych.

SemEval is an ongoing series of evaluations of computational semantic analysis systems; it evolved from the Senseval word sense evaluation series. The evaluations are intended to explore the nature of meaning in language. While meaning is intuitive to humans, transferring those intuitions to computational analysis has proved elusive.

BabelNet Multilingual semantic network and encyclopedic dictionary

BabelNet is a multilingual lexicalized semantic network and ontology developed at the NLP group of the Sapienza University of Rome. BabelNet was automatically created by linking Wikipedia to the most popular computational lexicon of the English language, WordNet. The integration is done using an automatic mapping and by filling in lexical gaps in resource-poor languages by using statistical machine translation. The result is an encyclopedic dictionary that provides concepts and named entities lexicalized in many languages and connected with large amounts of semantic relations. Additional lexicalizations and definitions are added by linking to free-license wordnets, OmegaWiki, the English Wiktionary, Wikidata, FrameNet, VerbNet and others. Similarly to WordNet, BabelNet groups words in different languages into sets of synonyms, called Babel synsets. For each Babel synset, BabelNet provides short definitions in many languages harvested from both WordNet and Wikipedia.

UBY-LMF is a format for standardizing lexical resources for Natural Language Processing (NLP). UBY-LMF conforms to the ISO standard for lexicons: LMF, designed within the ISO-TC37, and constitutes a so-called serialization of this abstract standard. In accordance with the LMF, all attributes and other linguistic terms introduced in UBY-LMF refer to standardized descriptions of their meaning in ISOCat.

UBY is a large-scale lexical-semantic resource for natural language processing (NLP) developed at the Ubiquitous Knowledge Processing Lab (UKP) in the department of Computer Science of the Technische Universität Darmstadt . UBY is based on the ISO standard Lexical Markup Framework (LMF) and combines information from several expert-constructed and collaboratively constructed resources for English and German.

In natural language processing, linguistics, and neighboring fields, Linguistic Linked Open Data (LLOD) describes a method and an interdisciplinary community concerned with creating, sharing, and (re-)using language resources in accordance with Linked Data principles. The Linguistic Linked Open Data Cloud was conceived and is being maintained by the Open Linguistics Working Group (OWLG) of the Open Knowledge Foundation, but has been a point of focal activity for several W3C community groups, research projects, and infrastructure efforts since then.

Joseph Mariani

Joseph Mariani is a French computer science researcher and pioneer in the field of speech processing.

Helen Aristar-Dry is an American linguist who currently serves as the series editor for SpringerBriefs in Linguistics. Most notably, from 1991 to 2013 she co-directed The LINGUIST List with Anthony Aristar. She has served as principal investigator or co-Principal Investigator on over $5,000,000 worth of research grants from the National Science Foundation and the National Endowment for the Humanities. She retired as Professor of English Language and Literature from Eastern Michigan University in 2013.

OntoLex is the short name of a vocabulary for lexical resources in the web of data (OntoLex-Lemon) and the short name of the W3C community group that created it.

In linguistics and language technology, a language resource is a "[composition] of linguistic material used in the construction, improvement and/or evaluation of language processing applications, (...) in language and language-mediated research studies and applications."

References

  1. ↑ "ISO 24613:2008 - Language resource management - Lexical markup framework (LMF)". Iso.org. Retrieved 2016-01-24.
  2. 1 2 "The relevance of standards for research infrastructures" (PDF). Hal.inria.fr. Retrieved 2016-01-24.
  3. ↑ "Lexical Markup Framework (LMF)" (PDF). Hal.inria.fr. Retrieved 2016-01-24.
  4. ↑ "Lexical markup framework (LMF) for NLP multilingual resources" (PDF). Hal.inria.fr. Retrieved 2016-01-24.
  5. ↑ "Vers la mise en place d'un lexique basé sur LMF pour la langue Wolof" (PDF). Aclweb.org. Retrieved 2016-01-24.
  6. ↑ "Standardizing Wordnets in the ISO Standard LMF: Wordnet-LMF for GermaNet" (PDF). Aclweb.org. Retrieved 2016-01-24.
  7. ↑ "Subcat-LMF: Fleshing out a standardized format for subcategorization frame interoperability" (PDF). Aclweb.org: 550–560. April 2012. Retrieved 2016-01-24.
  8. ↑ "UBY – A Large-Scale Unified Lexical-Semantic Resource Based on LMF" (PDF). Aclweb.org. Retrieved 2016-01-24.
  9. ↑ "Building a standardized Wordnet in the ISO LMF for aeb language" (PDF). Aclweb.org. Retrieved 2016-01-24.
  10. ↑ "LREC 2008 Proceedings". Lrec-conf.org. Retrieved 2016-01-24.
  11. ↑ "Modélisation des paradigmes de flexion des verbes arabes selon la norme LMF - ISO 24613" (PDF). Aclweb.org. Retrieved 2016-01-24.
  12. ↑ Gil Francopoulo (edited by) LMF Lexical Markup Framework, ISTE / Wiley 2013 ( ISBN   978-1-84821-430-9)
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.