This article needs to be updated. Please help update this article to reflect recent events or newly available information.(July 2026)
In natural language processing, a topic model is a type of probabilistic, neural, or algebraic model for discovering the abstract topics that occur in a collection of documents. Topic modeling is a frequently used text mining tool for discovering hidden semantic features and structures in a text. The topics produced by topic models are generated through a variety of mathematical frameworks, including probabilistic generative models, matrix factorization methods based on word co-occurrence, and clustering algorithms applied to semantic embeddings.[1][2][3]
An early topic model was described by Papadimitriou, Raghavan, Tamaki and Vempala in 1998.[8] Another one, called probabilistic latent semantic analysis (PLSA), was created by Thomas Hofmann in 1999.[9]Latent Dirichlet allocation (LDA), perhaps the most common topic model currently in use, is a generalization of PLSA. Developed by David Blei, Andrew Ng, and Michael I. Jordan in 2002, LDA introduces sparse Dirichlet prior distributions over document-topic and topic-word distributions, encoding the intuition that documents cover a small number of topics and that topics often use a small number of words.[10] Other topic models are generally extensions on LDA, such as Pachinko allocation, which improves on LDA by modeling correlations between topics in addition to the word correlations which constitute topics. Hierarchical latent tree analysis (HLTA) is an alternative to LDA, which models word co-occurrence using a tree of latent variables and the states of the latent variables, which correspond to soft clusters of documents, are interpreted as topics.
Animation of the topic detection process in a document-word matrix through biclustering. Every column corresponds to a document, every row to a word. A cell stores the frequency of a word in a document, with dark cells indicating high word frequencies. This procedure groups documents, which use similar words, as it groups words occurring in a similar set of documents. Such groups of words are then called topics. More usual topic models, such as LDA, only group documents, based on a more sophisticated and probabilistic mechanism.
Topic models for context information
Approaches for temporal information include Block and Newman's determination of the temporal dynamics of topics in the Pennsylvania Gazette during 1728–1800. Griffiths & Steyvers used topic modeling on abstracts from the journal PNAS to identify topics that rose or fell in popularity from 1991 to 2001 whereas Lamba & Madhusushan [11] used topic modeling on full-text research articles retrieved from DJLIT journal from 1981 to 2018. In the field of library and information science, Lamba & Madhusudhan [11][12][13][14] applied topic modeling on different Indian resources like journal articles and electronic theses and resources (ETDs). Nelson [15] has been analyzing change in topics over time in the Richmond Times-Dispatch to understand social and political changes and continuities in Richmond during the American Civil War. Yang, Torget and Mihalcea applied topic modeling methods to newspapers from 1829 to 2008. Mimno used topic modelling with 24 journals on classical philology and archaeology spanning 150 years to look at how topics in the journals change over time and how the journals become more different or similar over time.
Yin et al.[16] introduced a topic model for geographically distributed documents, where document positions are explained by latent regions which are detected during inference.
Chang and Blei[17] included network information between linked documents in the relational topic model, to model the links between websites.
The author-topic model by Rosen-Zvi et al.[18] models the topics associated with authors of documents to improve the topic detection for documents with authorship information.
To improve the qualitative aspects and coherency of generated topics, some researchers have explored the efficacy of "coherence scores", or otherwise how computer-extracted clusters (i.e. topics) align with a human benchmark.[19][20] Coherence scores are metrics for optimising the number of topics to extract from a document corpus.[21]
Methods
In practice, researchers attempt to fit appropriate model parameters to the data corpus using one of several heuristics for maximum likelihood fit. A survey by D. Blei describes this suite of algorithms.[22] Several groups of researchers starting with Papadimitriou et al.[8] have attempted to design algorithms with provable guarantees. Assuming that the data were actually generated by the model in question, they try to design algorithms that probably find the model that was used to create the data. Techniques used here include singular value decomposition (SVD) and the method of moments. In 2012 an algorithm based upon non-negative matrix factorization (NMF) was introduced that also generalizes to topic models with correlations among topics.[23]
Since 2017, neural networks has been leveraged in topic modeling in order to improve the speed of inference,[24] and leading to further advancements like vONTSS, which allows humans to incorporate domain knowledge via weakly supervised learning.[25]
Topic modeling has leveraged LLMs through contextual embedding[27] and fine tuning.[28]
Applications of topic models
Quantitative biomedicine
Topic models are also used in other contexts. For example, uses of topic models in biology and bioinformatics research have emerged.[29] Recently, topic models have been used to extract information from a dataset of genomic samples of cancers.[30] In this case, topics are biological latent variables to be inferred.
Music and creativity
Topic models can be used for analysis of music. For instance, topic models were used to quantify how musical styles change in time, and identify the influence of specific artists on later music creation.[31]
↑Song, W.; Zou, L. (2016). "LDA-TM: A Two-step Approach to Twitter Topic Data Clustering". Proceedings of the IEEE International Conference on Cloud Computing and Big Data Analysis. IEEE. pp.342–347. doi:10.1109/ICCCBDA.2016.7529581.
↑Hofmann, Thomas (1999). "Probabilistic Latent Semantic Indexing"(PDF). Proceedings of the Twenty-Second Annual International SIGIR Conference on Research and Development in Information Retrieval. Archived from the original(PDF) on 2010-12-14.
↑Rosen-Zvi, Michal (2004). "The author-topic model for authors and documents". Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence: 487–494. arXiv:1207.4169.
↑Reverter-Rambaldi, Marcel (2022). Topic Modelling in Spontaneous Speech Data (Honours thesis). Australian National University. doi:10.25911/M1YF-ZF55.
↑Newman, David (2010). "Automatic evaluation of topic coherence". Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: 100–108.
↑Xu, Weijie; Hu, Wenxiang; Wu, Fanyou; Sengamedu, Srinivasan (2023). "DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM". Findings of the Association for Computational Linguistics: EMNLP 2023. Stroudsburg, PA, USA: Association for Computational Linguistics: 9040–9057. arXiv:2310.15296. doi:10.18653/v1/2023.findings-emnlp.606.
↑Shalit, Uri; Weinshall, Daphna; Chechik, Gal (2013-05-13). "Modeling Musical Influence with Topic Models". Proceedings of the 30th International Conference on Machine Learning. PMLR: 244–252.
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
