Latent class model

Last updated November 30, 2023

In statistics, a latent class model (LCM) relates a set of observed (usually discrete) multivariate variables to a set of latent variables. It is a type of latent variable model. It is called a latent class model because the latent variable is discrete. A class is characterized by a pattern of conditional probabilities that indicate the chance that variables take on certain values.

Confronted with a situation as follows, a researcher might choose to use LCA to understand the data: Imagine that symptoms a-d have been measured in a range of patients with diseases X, Y, and Z, and that disease X is associated with the presence of symptoms a, b, and c, disease Y with symptoms b, c, d, and disease Z with symptoms a, c and d.

The LCA will attempt to detect the presence of latent classes (the disease entities), creating patterns of association in the symptoms. As in factor analysis, the LCA can also be used to classify case according to their maximum likelihood class membership.^[1]^[3]

Because the criterion for solving the LCA is to achieve latent classes within which there is no longer any association of one symptom with another (because the class is the disease which causes their association), and the set of diseases a patient has (or class a case is a member of) causes the symptom association, the symptoms will be "conditionally independent", i.e., conditional on class membership, they are no longer related.^[1]

Model

Within each latent class, the observed variables are statistically independent. This is an important aspect. Usually the observed variables are statistically dependent. By introducing the latent variable, independence is restored in the sense that within classes variables are independent (local independence). We then say that the association between the observed variables is explained by the classes of the latent variable (McCutcheon, 1987).

In one form, the latent class model is written as

p_{i_{1},i_{2},\ldots ,i_{N}}\approx \sum _{t}^{T}p_{t}\,\prod _{n}^{N}p_{i_{n},t}^{n},

where $T$ is the number of latent classes and $p_{t}$ are the so-called recruitment or unconditional probabilities that should sum to one. $p_{i_{n},t}^{n}$ are the marginal or conditional probabilities.

For a two-way latent class model, the form is

p_{ij}\approx \sum _{t}^{T}p_{t}\,p_{it}\,p_{jt}.

This two-way model is related to probabilistic latent semantic analysis and non-negative matrix factorization.

The probability model used in LCA is closely related to the Naive Bayes classifier. The main difference is that in LCA, the class membership of an individual is a latent variable, whereas in Naive Bayes classifiers the class membership is an observed label.

Related methods

There are a number of methods with distinct names and uses that share a common relationship. Cluster analysis is, like LCA, used to discover taxon-like groups of cases in data. Multivariate mixture estimation (MME) is applicable to continuous data, and assumes that such data arise from a mixture of distributions: imagine a set of heights arising from a mixture of men and women. If a multivariate mixture estimation is constrained so that measures must be uncorrelated within each distribution, it is termed latent profile analysis. Modified to handle discrete data, this constrained analysis is known as LCA. Discrete latent trait models further constrain the classes to form from segments of a single dimension: essentially allocating members to classes on that dimension: an example would be assigning cases to social classes on a dimension of ability or merit.

As a practical instance, the variables could be multiple choice items of a political questionnaire. The data in this case consists of a N-way contingency table with answers to the items for a number of respondents. In this example, the latent variable refers to political opinion and the latent classes to political groups. Given group membership, the conditional probabilities specify the chance certain answers are chosen.

Application

LCA may be used in many fields, such as: collaborative filtering,^[4] Behavior Genetics ^[5] and Evaluation of diagnostic tests.^[6]

Related Research Articles

In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events.

A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobservable ("hidden") states. As part of the definition, HMM requires that there be an observable process $whose outcomes are "influenced" by the outcomes of in a known way. Since cannot be observed directly, the goal is to learn about by observing HMM has an additional requirement that the outcome of at time must be "influenced" exclusively by the outcome of at and that the outcomes of and at must be conditionally independent of at given at time$

A Bayesian network is a probabilistic graphical model that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG). While it is one of several forms of causal notation, causal networks are special cases of Bayesian networks. Bayesian networks are ideal for taking an event that occurred and predicting the likelihood that any one of several possible known causes was the contributing factor. For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms. Given symptoms, the network can be used to compute the probabilities of the presence of various diseases.

In statistics, the logistic model is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression is estimating the parameters of a logistic model. Formally, in binary logistic regression there is a single binary dependent variable, coded by an indicator variable, where the two values are labeled "0" and "1", while the independent variables can each be a binary variable or a continuous variable. The corresponding probability of the value labeled "1" can vary between 0 and 1, hence the labeling; the function that converts log-odds to probability is the logistic function, hence the name. The unit of measurement for the log-odds scale is called a logit, from logistic unit, hence the alternative names. See § Background and § Definition for formal mathematics, and § Example for a worked example.

In statistics, an expectation–maximization (EM) algorithm is an iterative method to find (local) maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. The EM iteration alternates between performing an expectation (E) step, which creates a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

In statistics, Gibbs sampling or a Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm for obtaining a sequence of observations which are approximated from a specified multivariate probability distribution, when direct sampling is difficult. This sequence can be used to approximate the joint distribution ; to approximate the marginal distribution of one of the variables, or some subset of the variables ; or to compute an integral. Typically, some of the variables correspond to observations whose values are known, and hence do not need to be sampled.

In probability and statistics, a mixture distribution is the probability distribution of a random variable that is derived from a collection of other random variables as follows: first, a random variable is selected by chance from the collection according to given probabilities of selection, and then the value of the selected random variable is realized. The underlying random variables may be random real numbers, or they may be random vectors, in which case the mixture distribution is a multivariate distribution.

In statistics, a mixture model is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed data set should identify the sub-population to which an individual observation belongs. Formally a mixture model corresponds to the mixture distribution that represents the probability distribution of observations in the overall population. However, while problems associated with "mixture distributions" relate to deriving the properties of the overall population from those of the sub-populations, "mixture models" are used to make statistical inferences about the properties of the sub-populations given only observations on the pooled population, without sub-population identity information.

In statistical classification, two main approaches are called the generative approach and the discriminative approach. These compute classifiers by different approaches, differing in the degree of statistical modelling. Terminology is inconsistent, but three major types can be distinguished, following Jebara (2004):

A generative model is a statistical model of the joint probability distribution $on given observable variable X and target variable Y;$
A discriminative model is a model of the conditional probability $of the target Y, given an observation x; and$
Classifiers computed without using a probability model are also referred to loosely as "discriminative".

Probabilistic latent semantic analysis (PLSA), also known as probabilistic latent semantic indexing is a statistical technique for the analysis of two-mode and co-occurrence data. In effect, one can derive a low-dimensional representation of the observed variables in terms of their affinity to certain hidden variables, just as in latent semantic analysis, from which PLSA evolved.

This glossary of statistics and probability is a list of definitions of terms and concepts used in the mathematical sciences of statistics and probability, their sub-disciplines, and related fields. For additional related terms, see Glossary of mathematics and Glossary of experimental design.

Conditional random fields (CRFs) are a class of statistical modeling methods often applied in pattern recognition and machine learning and used for structured prediction. Whereas a classifier predicts a label for a single sample without considering "neighbouring" samples, a CRF can take context into account. To do so, the predictions are modelled as a graphical model, which represents the presence of dependencies between the predictions. What kind of graph is used depends on the application. For example, in natural language processing, "linear chain" CRFs are popular, for which each prediction is dependent only on its immediate neighbours. In image processing, the graph typically connects locations to nearby and/or similar locations to enforce that they receive similar predictions.

A latent variable model is a statistical model that relates a set of observable variables to a set of latent variables.

In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes. That is, it is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables.

In statistics, binomial regression is a regression analysis technique in which the response has a binomial distribution: it is the number of successes in a series of $independent Bernoulli trials, where each trial has probability of success . In binomial regression, the probability of a success is related to explanatory variables: the corresponding concept in ordinary regression is to relate the mean value of the unobserved response to explanatory variables.$

In probability theory and statistics, the Dirichlet-multinomial distribution is a family of discrete multivariate probability distributions on a finite support of non-negative integers. It is also called the Dirichlet compound multinomial distribution (DCM) or multivariate Pólya distribution. It is a compound probability distribution, where a probability vector p is drawn from a Dirichlet distribution with parameter vector $, and an observation drawn from a multinomial distribution with probability vector p and number of trials n . The Dirichlet parameter vector captures the prior belief about the situation and can be seen as a pseudocount: observations of each outcome that occur before the actual data is collected. The compounding corresponds to a Pólya urn scheme. It is frequently encountered in Bayesian statistics, machine learning, empirical Bayes methods and classical statistics as an overdispersed multinomial distribution.$

In probability theory and statistics, a categorical distribution is a discrete probability distribution that describes the possible results of a random variable that can take on one of K possible categories, with the probability of each category separately specified. There is no innate underlying ordering of these outcomes, but numerical labels are often attached for convenience in describing the distribution,. The K-dimensional categorical distribution is the most general distribution over a K-way event; any other discrete distribution over a size-K sample space is a special case. The parameters specifying the probabilities of each possible outcome are constrained only by the fact that each must be in the range 0 to 1, and all must sum to 1.

In statistics and econometrics, the multinomial probit model is a generalization of the probit model used when there are several possible categories that the dependent variable can fall into. As such, it is an alternative to the multinomial logit model as one method of multiclass classification. It is not to be confused with the multivariate probit model, which is used to model correlated binary outcomes for more than one independent variable.

A vine is a graphical tool for labeling constraints in high-dimensional probability distributions. A regular vine is a special case for which all constraints are two-dimensional or conditional two-dimensional. Regular vines generalize trees, and are themselves specializations of Cantor tree.

References

1 2 3 Lazarsfeld, P.F. and Henry, N.W. (1968) Latent structure analysis. Boston: Houghton Mifflin
↑ Formann, A. K. (1984). Latent Class Analyse: Einführung in die Theorie und Anwendung [Latent class analysis: Introduction to theory and application]. Weinheim: Beltz.
↑ Teichert, Thorsten (2000). "Das Latent-Ciass Verfahren zur Segmentierung von wahlbasierten Conjoint-Daten. Befunde einer empirischen Anwendung". Marketing ZFP. 22 (3): 227–240. doi:10.15358/0344-1369-2000-3-227. ISSN 0344-1369.
↑ Cheung, Kwok-Wai; Tsui, Kwok-Ching; Liu, Jiming (2004). "Extended latent class models for collaborative recommendation". IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. 34 (1): 143–148. CiteSeerX 10.1.1.6.2234 . doi:10.1109/TSMCA.2003.818877. S2CID 11628144.
↑ Eaves, L. J., Silberg, J. L., Hewitt, J. K., Rutter, M., Meyer, J. M., Neale, M. C., & Pickles, A (1993). "Analyzing twin resemblance in multisymptom data: genetic applications of a latent class model for symptoms of conduct disorder in juvenile boys". Behavior Genetics. 23 (1): 5–19. doi:10.1007/bf01067550. PMID 8476390. S2CID 40678009.{{cite journal}}: CS1 maint: multiple names: authors list (link)
↑ Bermingham, M. L., Handel, I. G., Glass, E. J., Woolliams, J. A., de Clare Bronsvoort, B. M., McBride, S. H., Skuce, R. A., Allen, A . R., McDowell, S. W. J., & Bishop, S. C. (2015). "Hui and Walter's latent-class model extended to estimate diagnostic test properties from surveillance data: a latent model for latent data". Scientific Reports. 5: 11861. Bibcode:2015NatSR...511861B. doi:10.1038/srep11861. PMC 4493568 . PMID 26148538.{{cite journal}}: CS1 maint: multiple names: authors list (link)

Linda M. Collins; Stephanie T. Lanza (2010). Latent class and latent transition analysis for the social, behavioral, and health sciences. New York: Wiley. ISBN 978-0-470-22839-5.
Allan L. McCutcheon (1987). Latent class analysis. Quantitative Applications in the Social Sciences Series No. 64. Thousand Oaks, California: SAGE Publications. ISBN 978-0-521-59451-6.
Leo A. Goodman (1974). "Exploratory latent structure analysis using both identifiable and unidentifiable models". Biometrika . 61 (2): 215–231. doi:10.1093/biomet/61.2.215.
Paul F. Lazarsfeld, Neil W. Henry (1968). Latent Structure Analysis.

External links

Statistical Innovations, Home Page, 2016. Website with latent class software (Latent GOLD 5.1), free demonstrations, tutorials, user guides, and publications for download. Also included: online courses, FAQs, and other related software.
The Methodology Center, Latent Class Analysis, a research center at Penn State, free software, FAQ
John Uebersax, Latent Class Analysis, 2006. A web-site with bibliography, software, links and FAQ for latent class analysis

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[Lazarsfeld-1] 1 2 3 Lazarsfeld, P.F. and Henry, N.W. (1968) Latent structure analysis. Boston: Houghton Mifflin

[2] Formann, A. K. (1984). Latent Class Analyse: Einführung in die Theorie und Anwendung [Latent class analysis: Introduction to theory and application]. Weinheim: Beltz.

[3] Teichert, Thorsten (2000). "Das Latent-Ciass Verfahren zur Segmentierung von wahlbasierten Conjoint-Daten. Befunde einer empirischen Anwendung". Marketing ZFP. 22 (3): 227–240. doi:10.15358/0344-1369-2000-3-227. ISSN 0344-1369.

[Cheung2004-4] Cheung, Kwok-Wai; Tsui, Kwok-Ching; Liu, Jiming (2004). "Extended latent class models for collaborative recommendation". IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. 34 (1): 143–148. CiteSeerX 10.1.1.6.2234 . doi:10.1109/TSMCA.2003.818877. S2CID 11628144.

[Eaves2013-5] Eaves, L. J., Silberg, J. L., Hewitt, J. K., Rutter, M., Meyer, J. M., Neale, M. C., & Pickles, A (1993). "Analyzing twin resemblance in multisymptom data: genetic applications of a latent class model for symptoms of conduct disorder in juvenile boys". Behavior Genetics. 23 (1): 5–19. doi:10.1007/bf01067550. PMID 8476390. S2CID 40678009.{{cite journal}}: CS1 maint: multiple names: authors list (link)

[Bermingham2015-6] Bermingham, M. L., Handel, I. G., Glass, E. J., Woolliams, J. A., de Clare Bronsvoort, B. M., McBride, S. H., Skuce, R. A., Allen, A . R., McDowell, S. W. J., & Bishop, S. C. (2015). "Hui and Walter's latent-class model extended to estimate diagnostic test properties from surveillance data: a latent model for latent data". Scientific Reports. 5: 11861. Bibcode:2015NatSR...511861B. doi:10.1038/srep11861. PMC 4493568 . PMID 26148538.{{cite journal}}: CS1 maint: multiple names: authors list (link)

[1]

[2]

[3]

[4]

[5]

[6]