Related Research Articles
Corpus linguistics is the study of a language as that language is expressed in its text corpus, its body of "real world" text. Corpus linguistics proposes that a reliable analysis of a language is more feasible with corpora collected in the field—the natural context ("realia") of that language—with minimal experimental interference. The large collections of text allow linguistics to run quantitative analyses on linguistic concepts, otherwise harder to quantify.
In linguistics and natural language processing, a corpus or text corpus is a dataset, consisting of natively digital and older, digitalized, language resources, either annotated or unannotated.
Word-sense disambiguation (WSD) is the process of identifying which sense of a word is meant in a sentence or other segment of context. In human language processing and cognition, it is usually subconscious/automatic but can often come to conscious attention when ambiguity impairs clarity of communication, given the pervasive polysemy in natural language. In computational linguistics, it is an open problem that affects other computer-related writing, such as discourse, improving relevance of search engines, anaphora resolution, coherence, and inference.
A parallel text is a text placed alongside its translation or translations. Parallel text alignment is the identification of the corresponding sentences in both halves of the parallel text. The Loeb Classical Library and the Clay Sanskrit Library are two examples of dual-language series of texts. Reference Bibles may contain the original languages and a translation, or several translations by themselves, for ease of comparison and study; Origen's Hexapla placed six versions of the Old Testament side by side. A famous example is the Rosetta Stone, whose discovery allowed the Ancient Egyptian language to begin being deciphered.
Eur-Lex is an official website of European Union law and other public documents of the European Union (EU), published in 24 official languages of the EU. The Official Journal (OJ) of the European Union is also published on EUR-Lex. Users can access EUR-Lex free of charge and also register for a free account, which offers extra features.
In linguistics, a treebank is a parsed text corpus that annotates syntactic or semantic sentence structure. The construction of parsed corpora in the early 1990s revolutionized computational linguistics, which benefitted from large-scale empirical data.
The British National Corpus (BNC) is a 100-million-word text corpus of samples of written and spoken English from a wide range of sources. The corpus covers British English of the late 20th century from a wide variety of genres, with the intention that it be a representative sample of spoken and written British English of that time. It is used in corpus linguistics for analysis of corpora.
Internet linguistics is a domain of linguistics advocated by the English linguist David Crystal. It studies new language styles and forms that have arisen under the influence of the Internet and of other new media, such as Short Message Service (SMS) text messaging. Since the beginning of human–computer interaction (HCI) leading to computer-mediated communication (CMC) and Internet-mediated communication (IMC), experts, such as Gretchen McCulloch have acknowledged that linguistics has a contributing role in it, in terms of web interface and usability. Studying the emerging language on the Internet can help improve conceptual organization, translation and web usability. Such study aims to benefit both linguists and web users combined.
The Corpus of Contemporary American English (COCA) is a one-billion-word corpus of contemporary American English. It was created by Mark Davies, retired professor of corpus linguistics at Brigham Young University (BYU).
The knowledge acquisition bottleneck is perhaps the major impediment to solving the word-sense disambiguation (WSD) problem. Unsupervised learning methods rely on knowledge about word senses, which is barely formulated in dictionaries and lexical databases. Supervised learning methods depend heavily on the existence of manually annotated examples for every word sense, a requisite that can so far be met only for a handful of words for testing purposes, as it is done in the Senseval exercises.
Macmillan English Dictionary for Advanced Learners, also known as MEDAL, is an advanced learner's dictionary first published in 2002 by Macmillan Education. It shares most of the features of this type of dictionary: it provides definitions in simple language, using a controlled defining vocabulary; most words have example sentences to illustrate how they are typically used; and information is given about how words combine grammatically or in collocations. MEDAL also introduced a number of innovations. These include:
Tatoeba is a free collection of example sentences with translations geared towards foreign language learners. It is available in more than 400 languages. Its name comes from the Japanese phrase "tatoeba" (例えば), meaning "for example". It is written and maintained by a community of volunteers through a model of open collaboration. Individual contributors are known as Tatoebans. It is run by Association Tatoeba, a French non-profit organization funded through donations.
The Europarl Corpus is a corpus that consists of the proceedings of the European Parliament from 1996 to 2012. In its first release in 2001, it covered eleven official languages of the European Union. With the political expansion of the EU the official languages of the ten new member states have been added to the corpus data. The latest release (2012) comprised up to 60 million words per language with the newly added languages being slightly underrepresented as data for them is only available from 2007 onwards. This latest version includes 21 European languages: Romanic, Germanic, Slavic, Finno-Ugric, Baltic, and Greek.
In natural language processing and information retrieval, explicit semantic analysis (ESA) is a vectoral representation of text that uses a document corpus as a knowledge base. Specifically, in ESA, a word is represented as a column vector in the tf–idf matrix of the text corpus and a document is represented as the centroid of the vectors representing its words. Typically, the text corpus is English Wikipedia, though other corpora including the Open Directory Project have been used.
Sketch Engine is a corpus manager and text analysis software developed by Lexical Computing CZ s.r.o. since 2003. Its purpose is to enable people studying language behaviour to search large text collections according to complex and linguistically motivated queries. Sketch Engine gained its name after one of the key features, word sketches: one-page, automatic, corpus-derived summaries of a word's grammatical and collocational behaviour. Currently, it supports and provides corpora in 90+ languages.
The Bulgarian National Corpus (BulNC) is a large representative corpus of Bulgarian comprising about 200,000 texts and amounting to over 1 billion words.
A word sketch is a one-page, automatic, corpus-derived summary of a word’s grammatical and collocational behaviour. Word sketches were first introduced by the British corpus linguist Adam Kilgarriff and exploited within the Sketch Engine corpus management system. They are an extension of the general collocation concept used in corpus linguistics in that they group collocations according to particular grammatical relations. The collocation candidates in a word sketch are sorted either by their frequency or using a lexicographic association score like Dice, T-score or MI-score.
SkELL is a free corpus-based web tool that allows language learners and teachers find authentic sentences for specific target words. For any word or a phrase, SkELL displays a concordance that lists example sentences drawn from a special text corpus crawled from the World Wide Web, which has been cleaned of spam and includes only high-quality texts covering everyday, standard, formal, and professional language. There are versions of SkELL for English, Russian, German, Italian, Czech and Estonian.
The TenTen Corpus Family (also called TenTen corpora) is a set of comparable web text corpora, i.e. collections of texts that have been crawled from the World Wide Web and processed to match the same standards. These corpora are made available through the Sketch Engine corpus manager. There are TenTen corpora for more than 35 languages. Their target size is 10 billion (1010) words per language, which gave rise to the corpus family's name.
The Czech National Corpus (CNC) is a large electronic corpus of written and spoken Czech language, developed by the Institute of the Czech National Corpus (ICNC) in the Faculty of Arts at Charles University in Prague. The collection is used for teaching and research in corpus linguistics. The ICNC collaborates with over 200 researchers and students, 270 publishers, and other similar research projects.
References
- ↑ Leech, Geoffrey (2007). "Teaching and language corpora: a convergence". In Wichmann, A.; et al. (eds.). Teaching and Language Corpora. London: Longman. p. 9.
- ↑ "Corpus Resource Database (CoRD)". Department of English, University of Helsinki.
- ↑ Wahle, Jan Philip; Ruas, Terry; Mohammad, Saif; Gipp, Bela (2022). "D3: A Massive Dataset of Scholarly Metadata for Analyzing the State of Computer Science Research". Proceedings of the Thirteenth Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association: 2642–2651. arXiv: 2204.13384 .
- ↑ Professor Mark Davies at BYU created an online tool to search Google's English language corpus, drawn from Google Books, at http://googlebooks.byu.edu/x.asp.
- ↑ "PhraseFinder". A search engine for the Google Books Ngram Corpus that supports wildcard queries and offers an API.
- ↑ ,Basque corpora
- ↑ (in Spanish) "Molinolabs - corpus". molinolabs.com. Retrieved 12 January 2014.
- ↑ "CorALit – CorALit - Lietuvių mokslo kalbos tekstynas". coralit.lt. Retrieved 12 January 2014.
- ↑ "Turkish National Corpus - Türkçe Ulusal Derlemi - Homepage". tnc.org.tr. Retrieved 12 January 2014.
- ↑ Glazkova, A (2020). "Topical Classification of Text Fragments Accounting for Their Nearest Context". Automation and Remote Control. 81 (12): 2262–2276. doi:10.1134/S0005117920120097. S2CID 231929892.
- ↑ Rubtsova, Yu (2015). "Constructing a corpus for sentiment classification training". Software & Systems. 1: 72–78. doi:10.15827/0236-235X.109.072-078.
- ↑ "Under Update". search.dcl.bas.bg. Retrieved 12 January 2014.
- ↑ "Електронски корупус на македонски книжевни текстови".
- ↑ "Portál | Český národní korpus".
- ↑ Zdravkova, Katrina; Tufiş, Dan; Simov, Kiril; Radziszewski, Adam; Qasemizadeh, Behrang; Priest-Dorman, Greg; Petkevič, Vladimír; Oravecz, Csaba; Krstev, Cvetana; Kotsyba, Natalia; Kaalep, Heiki-Jaan; Ide, Nancy; Garabík, Radovan; Dimitrova, Ludmila; Derzhanski, Ivan; Barbu, Ana-Maria; Erjavec, Tomaž (2010-05-14). "Available from CLARIN". http://nl.ijs.si/me/v4/ .
{{cite journal}}: External link in|journal= - 1 2 "University of Tehran NLP Lab". ece.ut.ac.ir. Archived from the original on 28 January 2014. Retrieved 12 January 2014.
- ↑ Hadi Veisi, Mohammad MohammadAmini, Hawre Hosseini; Toward Kurdish language processing: Experiments in collecting and processing the AsoSoft text corpus, Digital Scholarship in the Humanities, fqy074, https://doi.org/10.1093/llc/fqy074
- ↑ "KOTONOHA「現代日本語書き言葉均衡コーパス」 少納言". kotonoha.gr.jp. Retrieved 12 January 2014.
- ↑ https://wortschatz.uni-leipzig.de/en/download/Hindi
- ↑ D. Upeksha, C. Wijayarathna, M. Siriwardena, L. Lasandun, C. Wimalasuriya, N. de Silva, and G. Dias . 2015. Implementing a Corpus for Sinhala Language. In Symposium on Language Technology for South Asia.
- ↑ Glossa (uio.no)
- ↑ https://aclanthology.org/L14-1376/
- ↑ https://arxiv.org/pdf/2102.06991.pdf, https://wortschatz.uni-leipzig.de/en/download/Hausa
- ↑ https://www.sketchengine.eu/igtenten-igbo-corpus/
- ↑ https://www.sketchengine.eu/corpora-and-languages/oromo-text-corpora/
- ↑ https://www.researchgate.net/publication/336274457_Digital_Yoruba_Corpus, https://www.sketchengine.eu/corpora-and-languages/yoruba-text-corpora/
- ↑ https://wortschatz.uni-leipzig.de/en/download/Zulu
- ↑ Pan, Jun (2019). "The Chinese/English Political Interpreting Corpus (CEPIC). Hong Kong Baptist University Library" . Retrieved January 3, 2022.
- ↑ Pan, Jun (2019-10-30). "The Chinese/English Political Interpreting Corpus (CEPIC): A New Electronic Resource for Translators and Interpreters". Proceedings of the Second Workshop Human-Informed Translation and Interpreting Technology Associated with RANLP 2019. Incoma Ltd., Shoumen, Bulgaria: 82–88. doi: 10.26615/issn.2683-0078.2019_010 . S2CID 211257773.
- ↑ "EUR-Lex Corpus". sketchengine.co.uk. 2 June 2016. Retrieved 27 October 2016.
- ↑ "OPUS - an open source parallel corpus". opus.lingfil.uu.se. Retrieved 12 January 2014.
- ↑ "Tatoeba - Number of sentences per language". tatoeba.org. Retrieved 23 November 2020.
- ↑ Liling Tan and Francis Bond (14 May 2012). "Building and Annotating the Linguistically Diverse NTU-MC (NTU — Multilingual Corpus)" (PDF). International Journal of Asian Language Processing. 22 (4): 161–174. Archived from the original (PDF) on 16 January 2014. Retrieved 12 January 2014.
- ↑ Guy Emerson, Liling Tan, Susanne Fertmann, Alexis Palmer and Michaela Regneri . 2014. SeedLing: Building and using a seed corpus for the Human Language Project. In Proceedings of the use of Computational methods in the study of Endangered Languages (ComputEL) Workshop. Baltimore, USA.
- ↑ H. Sanjurjo-González and M. Izquierdo. 2019. P-ACTRES 2.0: A parallel corpus for cross-linguistic research. In Parallel Corpora for Contrastive and Translation Studies: New resources and applications (pp. 215-231). John Benjamins Publishing.
- ↑ Ralf, Ralf Steinberger; Pouliquen, Bruno; Widiger, Anna; Ignat, Camelia; Erjavec, Tomaž; Tufiş, Dan; Varga, Dániel (2006). The JRC-Acquis: A multilingual aligned parallel corpus with 20+ languages. Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC'2006). Genoa, Italy, 24–26 May 2006.
- ↑ Liling Tan, Marcos Zampieri, Nikola Ljubešic, and Jörg Tiedemann. Merging comparable data sources for the discrimination of similar languages: The DSL corpus collection. In Proceedings of the 7th Workshop on Building and Using Comparable Corpora (BUCC). 2014.
- ↑ Kilgarriff, Adam (2012). "Getting to Know Your Corpus". Text, Speech and Dialogue. Lecture Notes in Computer Science. Vol. 7499. pp. 3–15. CiteSeerX 10.1.1.452.8074 . doi:10.1007/978-3-642-32790-2_1. ISBN 978-3-642-32789-6.
- ↑ Belinkov, Y., Habash, N., Kilgarriff, A., Ordan, N., Roth, R., & Suchomel, V. (2013). arTen-Ten: a new, vast corpus for Arabic. Proceedings of WACL.
- ↑ Kilgarriff, A., & Renau, I. (2013). esTenTen, a vast web corpus of Peninsular and American Spanish. Procedia - Social and Behavioral Sciences, 95, 12-19.
- ↑ Хохлова, М. В. (2016). Обзор больших русскоязычных корпусов текстов. In Материалы научной конференции" Интернет и современное общество" (pp. 74-77).
- ↑ Khokhlova, M. (2016). Comparison of High-Frequency Nouns from the Perspective of Large Corpora. RASLAN 2016 Recent Advances in Slavonic Natural Language Processing, 9.
- ↑ Trampuš, M., & Novak, B. (2012, October). Internals of an aggregated web news feed. In Proceedings of the Fifteenth International Information Science Conference IS SiKDD 2012 (pp. 431-434)
- ↑ "Cambridge English Corpus", Wikipedia, 2019-09-27, retrieved 2020-01-07
- ↑ "CAWSE Corpus - The University of Nottingham Ningbo China - 宁波诺丁汉大学". nottingham.edu.cn. Retrieved 2020-01-07.
- ↑ "English as a Lingua Franca in Academic Settings". University of Helsinki. 2018-03-23. Retrieved 2020-01-07.
- 1 2 "English as a lingua franca", Wikipedia, 2019-12-14, retrieved 2020-01-07
- ↑ Mauranen, A (2010). "English as an academic lingua franca: The ELFA project". English for Specific Purposes. 29 (3): 183–190. doi:10.1016/j.esp.2009.10.001.
- ↑ "ICLE". UCLouvain. Retrieved 2020-01-07.
- ↑ "LINDSEI". UCLouvain (in French). Retrieved 2020-01-07.
- ↑ "Trinity Lancaster Corpus | ESRC Centre for Corpus Approaches to Social Science (CASS)" . Retrieved 2020-01-07.
- ↑ Gablasova, D (2019). "The Trinity Lancaster Corpus: Development, Description and Application". International Journal of Learner Corpus Research. 5 (2): 126–158. doi: 10.1075/ijlcr.19001.gab .
- ↑ Juffs, A., Han, N-R., & Naismith, B. (2020). The University of Pittsburgh English Language Corpus (PELIC) [Data set]. doi : 10.5281/zenodo.3991977
- ↑ "Project". univie.ac.at. Retrieved 2020-01-07.