Semantic parsing

Last updated

Semantic parsing is the task of converting a natural language utterance to a logical form: a machine-understandable representation of its meaning. [1] Semantic parsing can thus be understood as extracting the precise meaning of an utterance. Applications of semantic parsing include machine translation, [2] question answering, [1] [3] ontology induction, [4] automated reasoning, [5] and code generation. [6] [7] The phrase was first used in the 1970s by Yorick Wilks as the basis for machine translation programs working with only semantic representations. [8] Semantic parsing is one of the important tasks in computational linguistics and natural language processing.

Contents

Semantic Parsing System Architecture Semantic Parsing System Architecture.png
Semantic Parsing System Architecture

Semantic parsing maps text to formal meaning representations. This contrasts with semantic role labeling and other forms of shallow semantic processing, which do not aim to produce complete formal meanings. [9] In computer vision, semantic parsing is a process of segmentation for 3D objects. [10] [11]

Major levels of linguistic structure Majorlevelsoflinguisticstructure.png
Major levels of linguistic structure

History & Background

Early research of semantic parsing included the generation of grammar manually [12] as well as utilizing applied programming logic. [13] In the 2000s, most of the work in this area involved the creation/learning and use of different grammars and lexicons on controlled tasks, [14] [15] particularly general grammars such as SCFGs. [16] This improved upon manual grammars primarily because they leveraged the syntactical nature of the sentence, but they still couldn’t cover enough variation and weren’t robust enough to be used in the real world. However, following the development of advanced neural network techniques, especially the Seq2Seq model, [17] and the availability of powerful computational resources, neural semantic parsing started emerging. Not only was it providing competitive results on the existing datasets, but it was robust to noise and did not require a lot of supervision and manual intervention. The current transition of traditional parsing to neural semantic parsing has not been perfect though. Neural semantic parsing, even with its advantages, still fails to solve the problem at a deeper level. Neural models like Seq2Seq treat the parsing problem as a sequential translation problem, and the model learns patterns in a black-box manner, which means we cannot really predict whether the model is truly solving the problem. Intermediate efforts and modifications to the Seq2Seq to incorporate syntax and semantic meaning have been attempted, [18] [19] with a marked improvement in results, but there remains a lot of ambiguity to be taken care of.

Types

Shallow Semantic Parsing

Shallow semantic parsing is concerned with identifying entities in an utterance and labelling them with the roles they play. Shallow semantic parsing is sometimes known as slot-filling or frame semantic parsing, since its theoretical basis comes from frame semantics, wherein a word evokes a frame of related concepts and roles. Slot-filling systems are widely used in virtual assistants in conjunction with intent classifiers, which can be seen as mechanisms for identifying the frame evoked by an utterance. [20] [21] Popular architectures for slot-filling are largely variants of an encoder-decoder model, wherein two recurrent neural networks (RNNs) are trained jointly to encode an utterance into a vector and to decode that vector into a sequence of slot labels. [22] This type of model is used in the Amazon Alexa spoken language understanding system. [20] This parsing follow an unsupervised learning techniques.

Deep Semantic Parsing

Deep semantic parsing, also known as compositional semantic parsing, is concerned with producing precise meaning representations of utterances that can contain significant compositionality. [23] Shallow semantic parsers can parse utterances like "show me flights from Boston to Dallas" by classifying the intent as "list flights", and filling slots "source" and "destination" with "Boston" and "Dallas", respectively. However, shallow semantic parsing cannot parse arbitrary compositional utterances, like "show me flights from Boston to anywhere that has flights to Juneau". Deep semantic parsing attempts to parse such utterances, typically by converting them to a formal meaning representation language. Nowadays, compositional semantic parsing are using Large Language Models to solve artificial compositional generalization tasks such as SCAN. [24]

Neural Semantic Parsing

Semantic parsers play a crucial role in natural language understanding systems because they transform natural language utterances into machine-executable logical structures or programmes. A well-established field of study, semantic parsing finds use in voice assistants, question answering, instruction following, and code generation. Since Neural approaches have been available for two years, many of the presumptions that underpinned semantic parsing have been rethought, leading to a substantial change in the models employed for semantic parsing. Though Semantic neural network and Neural Semantic Parsing [25] both deal with Natural Language Processing (NLP) and semantics, they are not same. The models and executable formalisms used in semantic parsing research have traditionally been strongly dependent on concepts from formal semantics in linguistics, like the λ-calculus produced by a CCG parser. Nonetheless, more approachable formalisms, like conventional programming languages, and NMT-style models that are considerably more accessible to a wider NLP audience, are made possible by recent work with neural encoder-decoder semantic parsers. We'll give a summary of contemporary neural approaches to semantic parsing and discuss how they've affected the field's understanding of semantic parsing.

Representation languages

Early semantic parsers used highly domain-specific meaning representation languages, [26] with later systems using more extensible languages like Prolog, [27] lambda calculus, [28] lambda dependency-based compositional semantics (λ-DCS), [29] SQL, [30] [31] Python, [32] Java, [33] the Alexa Meaning Representation Language, [20] and the Abstract Meaning Representation (AMR). Some work has used more exotic meaning representations, like query graphs, [34] semantic graphs, [35] or vector representations. [36]

Models

Most modern deep semantic parsing models are either based on defining a formal grammar for a chart parser or utilizing RNNs to directly translate from a natural language to a meaning representation language. Examples of systems built on formal grammars are the Cornell Semantic Parsing Framework, [37] Stanford University's Semantic Parsing with Execution (SEMPRE), [3] and the Word Alignment-based Semantic Parser (WASP). [38]

Datasets

Datasets used for training statistical semantic parsing models are divided into two main classes based on application: those used for question answering via knowledge base queries, and those used for code generation.

Question answering

Semantic Parsing for Conversational Question Answering Question Answer Semantic Parsing.jpg
Semantic Parsing for Conversational Question Answering

A standard dataset for question answering via semantic parsing is the Air Travel Information System (ATIS) dataset, which contains questions and commands about upcoming flights as well as corresponding SQL. [30] Another benchmark dataset is the GeoQuery dataset which contains questions about the geography of the U.S. paired with corresponding Prolog. [27] The Overnight dataset is used to test how well semantic parsers adapt across multiple domains; it contains natural language queries about 8 different domains paired with corresponding λ-DCS expressions. [39] Recently, semantic parsing is gaining significant popularity as a result of new research works and many large companies, namely Google, Microsoft, Amazon, etc. are working on this area. One on the recent works of Semantic Parsing for question answering is attached here. [40] Shown in this picture is a representation of an example conversation from SPICE. The left column shows dialogue turns (T1–T3) with user (U) and system (S) utterances. The middle column shows the annotations provided in CSQA. Blue boxes on the right show the sequence of actions (AS) and corresponding SPARQL semantic parses (SP).

Code generation

Popular datasets for code generation include two trading card datasets that link the text that appears on cards to code that precisely represents those cards. One was constructed linking Magic: The Gathering card texts to Java snippets; the other by linking Hearthstone card texts to Python snippets. [33] The IFTTT dataset [41] uses a specialized domain-specific language with short conditional commands. The Django dataset [42] pairs Python snippets with English and Japanese pseudocode describing them. The RoboCup dataset [43] pairs English rules with their representations in a domain-specific language that can be understood by virtual soccer-playing robots.

Application Areas

Within the field of natural language processing (NLP), semantic parsing deals with transforming human language into a format that is easier for machines to understand and comprehend. This method is useful in a number of contexts:

Semantic parsing aims to improve various applications' efficiency and efficacy by bridging the gap between human language and machine processing in each of these domains.

Evaluation

The performance of Semantic parsers is also measured using standard evaluation metrics as like syntactic parsing. This can be evaluated for the ratio of exact matches (percentage of sentences that were perfectly parsed), and precision, recall, and F1-score calculated based on the correct constituency or dependency assignments in the parse relative to that number in reference and/or hypothesis parses. The latter are also known as the PARSEVAL metrics. [44]

See also

Related Research Articles

Computational linguistics is an interdisciplinary field concerned with the computational modelling of natural language, as well as the study of appropriate computational approaches to linguistic questions. In general, computational linguistics draws upon linguistics, computer science, artificial intelligence, mathematics, logic, philosophy, cognitive science, cognitive psychology, psycholinguistics, anthropology and neuroscience, among others.

Natural language processing (NLP) is an interdisciplinary subfield of computer science and information retrieval. It is primarily concerned with giving computers the ability to support and manipulate human language. It involves processing natural language datasets, such as text corpora or speech corpora, using either rule-based or probabilistic machine learning approaches. The goal is a computer capable of "understanding" the contents of documents, including the contextual nuances of the language within them. To this end, natural language processing often borrows ideas from theoretical linguistics. The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves.

Natural-language understanding (NLU) or natural-language interpretation (NLI) is a subset of natural-language processing in artificial intelligence that deals with machine reading comprehension. Natural-language understanding is considered an AI-hard problem.

Parsing, syntax analysis, or syntactic analysis is the process of analyzing a string of symbols, either in natural language, computer languages or data structures, conforming to the rules of a formal grammar. The term parsing comes from Latin pars (orationis), meaning part.

Question answering (QA) is a computer science discipline within the fields of information retrieval and natural language processing (NLP) that is concerned with building systems that automatically answer questions that are posed by humans in a natural language.

A symbolic linguistic representation is a representation of an utterance that uses symbols to represent linguistic information about the utterance, such as information about phonetics, phonology, morphology, syntax, or semantics. Symbolic linguistic representations are different from non-symbolic representations, such as recordings, because they use symbols to represent linguistic information rather than measurements.

A language model is a probabilistic model of a natural language. In 1980, the first significant statistical language model was proposed, and during the decade IBM performed ‘Shannon-style’ experiments, in which potential sources for language modeling improvement were identified by observing and analyzing the performance of human subjects in predicting or correcting text.

Grammar induction is the process in machine learning of learning a formal grammar from a set of observations, thus constructing a model which accounts for the characteristics of the observed objects. More generally, grammatical inference is that branch of machine learning where the instance space consists of discrete combinatorial objects such as strings, trees and graphs.

In linguistics, statistical semantics applies the methods of statistics to the problem of determining the meaning of words or phrases, ideally through unsupervised learning, to a degree of precision at least sufficient for the purpose of information retrieval.

<span class="mw-page-title-main">Distributional semantics</span> Field of linguistics

Distributional semantics is a research area that develops and studies theories and methods for quantifying and categorizing semantic similarities between linguistic items based on their distributional properties in large samples of language data. The basic idea of distributional semantics can be summed up in the so-called distributional hypothesis: linguistic items with similar distributions have similar meanings.

William Aaron Woods, generally known as Bill Woods, is a researcher in natural language processing, continuous speech understanding, knowledge representation, and knowledge-based search technology. He is currently a Software Engineer at Google.

In natural language processing, textual entailment (TE), also known as natural language inference (NLI), is a directional relation between text fragments. The relation holds whenever the truth of one text fragment follows from another text.

The following outline is provided as an overview of and topical guide to natural-language processing:

In natural language processing (NLP), a word embedding is a representation of a word. The embedding is used in text analysis. Typically, the representation is a real-valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning. Word embeddings can be obtained using language modeling and feature learning techniques, where words or phrases from the vocabulary are mapped to vectors of real numbers.

Paraphrase or paraphrasing in computational linguistics is the natural language processing task of detecting and generating paraphrases. Applications of paraphrasing are varied including information retrieval, question answering, text summarization, and plagiarism detection. Paraphrasing is also useful in the evaluation of machine translation, as well as semantic parsing and generation of new samples to expand existing corpora.

In natural language processing, a sentence embedding refers to a numeric representation of a sentence in the form of a vector of real numbers which encodes meaningful semantic information.

Bidirectional Encoder Representations from Transformers (BERT) is a language model based on the transformer architecture, notable for its dramatic improvement over previous state of the art models. It was introduced in October 2018 by researchers at Google. A 2020 literature survey concluded that "in a little over a year, BERT has become a ubiquitous baseline in Natural Language Processing (NLP) experiments counting over 150 research publications analyzing and improving the model."

Emotion recognition in conversation (ERC) is a sub-field of emotion recognition, that focuses on mining human emotions from conversations or dialogues having two or more interlocutors. The datasets in this field are usually derived from social platforms that allow free and plenty of samples, often containing multimodal data. Self- and inter-personal influences play critical role in identifying some basic emotions, such as, fear, anger, joy, surprise, etc. The more fine grained the emotion labels are the harder it is to detect the correct emotion. ERC poses a number of challenges, such as, conversational-context modeling, speaker-state modeling, presence of sarcasm in conversation, emotion shift across consecutive utterances of the same interlocutor.

Mona Talat Diab is a computer science professor and director of Carnegie Mellon University's Language Technologies Institute. Previously, she was a professor at George Washington University and a research scientist with Facebook AI. Her research focuses on natural language processing, computational linguistics, cross lingual/multilingual processing, computational socio-pragmatics, Arabic language processing, and applied machine learning.

DisCoCat is a mathematical framework for natural language processing which uses category theory to unify distributional semantics with the principle of compositionality. The grammatical derivations in a categorial grammar are interpreted as linear maps acting on the tensor product of word vectors to produce the meaning of a sentence or a piece of text. String diagrams are used to visualise information flow and reason about natural language semantics.

References

  1. 1 2 Jia, Robin; Liang, Percy (2016-06-11). "Data Recombination for Neural Semantic Parsing". arXiv: 1606.03622 [cs.CL].
  2. Andreas, Jacob, Andreas Vlachos, and Stephen Clark. "Semantic parsing as machine translation." Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Vol. 2. 2013.
  3. 1 2 Berant, Jonathan, et al. "Semantic Parsing on Freebase from Question-Answer Pairs." EMNLP. Vol. 2. No. 5. 2013.
  4. Poon, Hoifung, and Pedro Domingos. "Unsupervised ontology induction from text." Proceedings of the 48th annual meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2010.
  5. Kaliszyk, Cezary, Josef Urban, and Jiří Vyskočil. "Automating formalization by statistical and semantic parsing of mathematics." International Conference on Interactive Theorem Proving. Springer, Cham, 2017.
  6. Rabinovich, Maxim; Stern, Mitchell; Klein, Dan (2017-04-25). "Abstract Syntax Networks for Code Generation and Semantic Parsing". arXiv: 1704.07535 [cs.CL].
  7. Yin, Pengcheng; Neubig, Graham (2017-04-05). "A Syntactic Neural Model for General-Purpose Code Generation". arXiv: 1704.01696 [cs.CL].
  8. Wilks, Y. and Fass, D. (1992) The Preference Semantics Family, In Computers and Mathematics with Applications, Volume 23, Issues 2-5, Pages 205-221.
  9. Hoifung Poon, Pedro Domingos Unsupervised Semantic Parsing , Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, 2009
  10. Armeni, Iro, et al. "3d semantic parsing of large-scale indoor spaces." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
  11. Qi, Charles R., et al. "Pointnet: Deep learning on point sets for 3d classification and segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  12. Warren, D. H. D. and Pereira, F. C. N. et al. "An efficient easily adaptable system for interpreting natural language queries." Comput. Linguist., 8(3-4):110–122. 1982
  13. Zelle, J. M. and Mooney, R. J et al. "Learning to parse database queries using inductive logic programming." Proceedings of the national conference on artificial intelligence, pages 1050–1055, 1996.
  14. Zettlemoyer and Collins, et al. "Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars." Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, UAI’05, pages 658–666, 2005.
  15. Kwiatkowski, T., Zettlemoyer, L., Goldwater, S., and Steedman, M. et al. "Inducing probabilistic ccg grammars from logical form with higher-order unification." Proceedings of the 2010 conference on empirical methods in natural language processing, pages 1223–1233, 2010.
  16. Wah Wong, Y. and J. Mooney, R. et al. "Learning synchronous grammars for semantic parsing with lambda calculus." Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 960–967, 2007.
  17. Dong, L. and Lapata, M. et al. "Language to logical form with neural attention.." Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 33–43, 2016.
  18. Pengcheng Yin, Graham Neubig et al. "A Syntactic Neural Model for General-Purpose Code Generation." Proceedings of 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 440–450, 2017.
  19. Shi, T., Tatwawadi, K., Chakrabarti, K., Mao, Y., Polozov, O., and Chen, W. et al. "A Incsql: Training incremental text-to-sql parsers with non-deterministic oracles." Published by Microsoft Research, 2018.
  20. 1 2 3 Kumar, Anjishnu, et al. "Just ASK: Building an Architecture for Extensible Self-Service Spoken Language Understanding." arXiv preprint arXiv:1711.00549 (2017).
  21. Bapna, Ankur, et al. "Towards zero-shot frame semantic parsing for domain scaling." arXiv preprint arXiv:1707.02363(2017).
  22. Liu, Bing, and Ian Lane. "Attention-based recurrent neural network models for joint intent detection and slot filling." arXiv preprint arXiv:1609.01454 (2016).
  23. Liang, Percy, and Christopher Potts. "Bringing machine learning and compositional semantics together." Annu. Rev. Linguist. 1.1 (2015): 355-376.
  24. Andrew Drozdov, Nathanael Schärli, Ekin Akyürek, Nathan Scales, Xinying Song, Xinyun Chen, Olivier Bousquet, Denny Zhou. "COMPOSITIONAL SEMANTIC PARSING WITH LARGE LANGUAGE MODELS" Cornell University,30 Sep 2022.
  25. Matt Gardner, Pradeep Dasigi, Srinivasan Iyer, Alane Suhr, Luke Zettlemoyer. "Neural Semantic Parsing" Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, July 2018.
  26. Woods, William A. Semantics for a question-answering system. Vol. 27. Garland Pub., 1979.
  27. 1 2 Zelle, John M., and Raymond J. Mooney. "Learning to parse database queries using inductive logic programming." Proceedings of the national conference on artificial intelligence. 1996.
  28. Wong, Yuk Wah, and Raymond Mooney. "Learning synchronous grammars for semantic parsing with lambda calculus." Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. 2007.
  29. Liang, Percy. "Lambda dependency-based compositional semantics." arXiv preprint arXiv:1309.4408 (2013).
  30. 1 2 Hemphill, Charles T., John J. Godfrey, and George R. Doddington. "The ATIS spoken language systems pilot corpus." Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24–27, 1990. 1990.
  31. Iyer, Srinivasan, et al. "Learning a neural semantic parser from user feedback." arXiv preprint arXiv:1704.08760 (2017).
  32. Yin, Pengcheng, and Graham Neubig. "A syntactic neural model for general-purpose code generation." arXiv preprint arXiv:1704.01696 (2017).
  33. 1 2 Ling, Wang, et al. "Latent predictor networks for code generation." arXiv preprint arXiv:1603.06744 (2016).
  34. Yih, Scott Wen-tau, et al. "Semantic parsing via staged query graph generation: Question answering with knowledge base." (2015).
  35. Reddy, Siva, Mirella Lapata, and Mark Steedman. "Large-scale semantic parsing without question-answer pairs." Transactions of the Association of Computational Linguistics 2.1 (2014): 377-392.
  36. Guu, Kelvin, John Miller, and Percy Liang. "Traversing knowledge graphs in vector space." arXiv preprint arXiv:1506.01094 (2015).
  37. Artzi, Yoav. "Cornell SPF: Cornell semantic parsing framework." arXiv preprint arXiv:1311.3011 (2013).
  38. Wong, Yuk Wah; Mooney, Raymond J. (2006-06-04). Learning for semantic parsing with statistical machine translation. Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics -. Association for Computational Linguistics. pp. 439–446. CiteSeerX   10.1.1.135.7209 . doi:10.3115/1220835.1220891.
  39. Wang, Yushi, Jonathan Berant, and Percy Liang. "Building a semantic parser overnight." Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Vol. 1. 2015.
  40. Laura Perez-Beltrachini, Parag Jain, Emilio Monti, Mirella Lapata. Semantic Parsing for Conversational Question Answering over Knowledge Graphs 'Proceedings on EACL 2023'. 28 January 2023.
  41. Quirk, Chris, Raymond Mooney, and Michel Galley. "Language to code: Learning semantic parsers for if-this-then-that recipes." Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Vol. 1. 2015.
  42. Oda, Yusuke, et al. "Learning to generate pseudo-code from source code using statistical machine translation (t)." Automated Software Engineering (ASE), 2015 30th IEEE/ACM International Conference on. IEEE, 2015.
  43. Kuhlmann, Gregory, et al. "Guiding a reinforcement learner with natural language advice: Initial results in RoboCup soccer." The AAAI-2004 workshop on supervisory control of learning and adaptive systems. 2004.
  44. Black, E.; Abney, S.; Flickenger, D.; Gdaniec, C.; Grishman, R.; Harrison, P.; Hindle, D.; Ingria, R.; Jelinek, F.; Klavans, J.; Liberman, M.; Marcus, M.; Roukos, S.; Santorini, B.; Strzalkowski, T. (1991). A Procedure for Quantitatively Comparing the Syntactic Coverage of English Grammars. Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, California, February 19–22, 1991.