Structural equation modeling (SEM) is a label for a diverse set of methods used by scientists in both experimental and observational research across the sciences,business, and other fields. It is used most in the social and behavioral sciences. A definition of SEM is difficult without reference to highly technical language, but a good starting place is the name itself.
SEM involves the construction of a model , an informative representation of some observable or theoretical phenomenon. In this model, different aspects of a phenomenon are theorized to be related to one another with a structure . This structure is a system of equations, but it is usually designed on paper or using a computer with arrows and symbols (also known as path notation as shown in Figure 1). The structure implies statistical and often causal relationships between variables, error terms and can include multiple equations. The equation (or equations) in SEM are mathematical and statistical properties that are implied by the model and its structural features, and then estimated with statistical algorithms (usually based on matrix algebra and generalized linear models) using experimental or observational data.
Criticisms of SEM methods hint at mathematical formulation problems, a tendency to accept models without establishing external validity, and philosophical bias inherent in the standard procedures.
Although there are not always clear boundaries of what is and what is not SEM,it generally involves path models (see also path analysis) and measurement models (see also factor analysis) and always employs statistical models and computer programs to investigate the structural connections between latent variables underlying the actual variables taken from observed data.
The SEM toolkit includes confirmatory factor analysis, confirmatory composite analysis, path analysis, multi-group modeling, longitudinal modeling, partial least squares path modeling, latent growth modeling and hierarchical or multi-level modeling.Use of SEM is commonly justified in the social sciences because it is a way to identify latent variables that are believed to exist, but cannot be directly observed in reality.
Researchers using SEM employ software programs (such as Mplus, lavaan (in R), LISREL, SPSS, Stata) to estimate the strength and sign of a coefficient for each modeled arrow (the numbers shown in Figure 1 for example), and to provide diagnostic clues suggesting which indicators or model components might produce inconsistency between the model and the data.
A hypothetical model suggesting that intelligence (as measured by four questions) can predict academic performance (as measured by SAT, ACT, and high school GPA) is shown in Figure 1. The concept of human intelligence cannot be measured directly in the way that one could measure height or weight. Instead, researchers have a theory and conceptualization of intelligence and then design measurement instruments such as a questionnaire or test that provides them with multiple indicators of intelligence. These indicators are then combined in a model to create a plausible way of measuring intelligence as a latent variable (the circle for intelligence in Figure 1) from the indicators (square boxes with scale 1–4 in Figure 1).
In SEM diagrams, latent variables are commonly shown as ovals and observed variables as rectangles. The diagram above shows how error (e) influences each intelligence question and the SAT, ACT, and GPA scores, but does not influence the latent variables. When applying this model to observed data generated from the instruments, the researcher can recover a measure of intelligence and academic performance for each individual observed with the instruments with a margin of error that is implied by the instruments. The researcher can then use intelligence to test a hypothesis, for example that intelligence causes academic performance (which is another latent variable in Figure 1) defined by a path model drawing an arrow from intelligence to performance. Figure 1 is therefore a general example of a SEM involving measurement of latent variables and estimation of a hypothesized effect between at least one latent variable and another observed or latent variable (in this case latent academic performance).
A great advantage of SEM is that all of these measurements and tests occur simultaneously in one statistical estimation procedure, where the errors throughout the model are calculated using all information from the model. This means the errors are more accurate than if a researcher were to calculate each part of the model separately.
Structural equation modeling (SEM) has its roots in the work of Sewall Wright who applied explicit causal interpretations to regression equations based on direct and indirect effects of observed variables in population genetics.Lee M. Wolfle compiled an annotated bibliographic history of Sewall Wright's path coefficient method which we know today as path modeling. Wright added two important elements to the standard practice of using regression to predict an outcome. These were (1) to combine information from more than one regression equation using (2) a causal approach to regression modeling rather than merely predictive. Sewall Wright consolidated his method of path analysis in his 1934 article "The Method of Path Coefficients".
Otis Dudley Duncan introduced SEM to the social sciences in 1975and it flourished throughout the 1970s and 80s. Different yet mathematically related modeling approaches developed in psychology, sociology, and economics. The convergence of two of these developmental streams (factor analysis from psychology, and path analysis from sociology via Duncan) produced the current core of SEM although there is great overlap with econometric practices employing simultaneous equations and exogenous (causal variables).
One of several programs Karl Gustav Jöreskog developed in the early 1970s at Educational Testing Services (LISREL) embedded latent variables (which psychologists knew as the latent factors from factor analysis) within path-analysis-style equations (which sociologists had inherited from Wright and Duncan).The factor-structured portion of the model incorporated measurement errors and thereby permitted measurement-error-adjusted estimation of effects connecting latent variables.
Loose and confusing terminology has been used to obscure weaknesses in the methods. In particular, PLS-PA (also known as PLS-PM) has been conflated with partial least squares regression PLSR, which is a substitute for ordinary least squares regression and has nothing to do with path analysis. PLS-PA has been falsely promoted as a method that works with small datasets when other estimation approaches fail; in fact, it has been shown that minimum required sample sizes for this method are consistent with those required in multiple regression.
Both LISREL and PLS-PA were conceived as iterative computer algorithms, with an emphasis from the start on creating an accessible graphical and data entry interface and extension of Wright's (1921) path analysis. Early Cowles Commission work on simultaneous equations estimation centered on Koopman and Hood's (1953) algorithms from the economics of transportation and optimal routing, with maximum likelihood estimation, and closed form algebraic calculations, as iterative solution search techniques were limited in the days before computers.
Anderson and Rubin (1949, 1950) developed the limited information maximum likelihood estimator for the parameters of a single structural equation, which indirectly included the two-stage least squares estimator and its asymptotic distribution (Anderson, 2005) and Farebrother (1999). Two-stage least squares was originally proposed as a method of estimating the parameters of a single structural equation in a system of linear simultaneous equations, being introduced by Theil (1953a, 1953b, 1961) and more or less independently by Basmann (1957) and Sargan (1958). Anderson's limited information maximum likelihood estimation was eventually implemented in a computer search algorithm, where it competed with other iterative SEM algorithms. Of these, two-stage least squares was by far the most widely used method in the 1960s and the early 1970s.
Systems of regression equation approaches were developed at the Cowles Commission from the 1950s on, extending the transportation modeling of Tjalling Koopmans. Sewall Wright and other statisticians attempted to promote path analysis methods at Cowles (then at the University of Chicago). University of Chicago statisticians identified many faults with path analysis applications to the social sciences; faults which did not pose significant problems for identifying gene transmission in Wright's context, but which made path methods such as PLS-PA and LISREL problematic in the social sciences. Freedman (1987) summarized these objections in path analyses: "failure to distinguish among causal assumptions, statistical implications, and policy claims has been one of the main reasons for the suspicion and confusion surrounding quantitative methods in the social sciences" (see also Wold's (1987) response). Wright's path analysis never gained a large following among US econometricians, but was successful in influencing Hermann Wold and his student Karl Jöreskog. Jöreskog's student Claes Fornell promoted LISREL in the US.
Advances in computers made it simple for novices to apply structural equation methods in the computer-intensive analysis of large datasets in complex, unstructured problems. The most popular solution techniques fall into three classes of algorithms: (1) ordinary least squares algorithms applied independently to each path, such as applied in the so-called PLS path analysis packages which estimate with OLS; (2) covariance analysis algorithms evolving from seminal work by Wold and his student Karl Jöreskog implemented in LISREL, AMOS, and EQS; and (3) simultaneous equations regression algorithms developed at the Cowles Commission by Tjalling Koopmans.
Pearlhas extended SEM from linear to nonparametric models, and proposed causal and counterfactual interpretations of the equations. For example, excluding a variable Z from the arguments of an equation asserts that the dependent variable is independent of interventions on the excluded variable, once we hold constant the remaining arguments. Nonparametric SEMs permit the estimation of total, direct and indirect effects without making any commitment to the form of the equations or to the distributions of the error terms. This extends mediation analysis to systems involving categorical variables in the presence of nonlinear interactions. Bollen and Pearl survey the history of the causal interpretation of SEM and why it has become a source of confusions and controversies.
SEM path analysis methods are popular in the social sciences because of their accessibility; packaged computer programs allow researchers to obtain results without the inconvenience of understanding experimental design and control, effect and sample sizes, and numerous other factors that are part of good research design. Supporters say that this reflects a holistic, and less blatantly causal, interpretation of many real world phenomena – especially in psychology and social interaction – than may be adopted in the natural sciences; detractors suggest that many flawed conclusions have been drawn because of this lack of experimental control.
Direction in the directed network models of SEM arises from presumed cause-effect assumptions made about reality. Social interactions and artifacts are often epiphenomena – secondary phenomena that are difficult to directly link to causal factors. An example of a physiological epiphenomenon is, for example, time to complete a 100-meter sprint. A person may be able to improve their sprint speed from 12 seconds to 11 seconds, but it will be difficult to attribute that improvement to any direct causal factors, like diet, attitude, weather, etc. The 1 second improvement in sprint time is an epiphenomenon – the holistic product of interaction of many individual factors.
Although each technique in the SEM family is different, the following aspects are common to many SEM methods, as it can be summarized as a 4E framework by many SEM scholars like Alex Liu, that is 1) Equation (model or equation specification), 2) Estimation of free parameters, 3) Evaluation of models and model fit, 4) Explanation and communication, as well as execution of results.
Two main components of models are distinguished in SEM: the structural model showing potential causal dependencies between endogenous and exogenous variables, and the measurement model showing the relations between latent variables and their indicators. Exploratory and confirmatory factor analysis models, for example, contain only the measurement part, while path diagrams can be viewed as SEMs that contain only the structural part.
In specifying pathways in a model, the modeler can posit two types of relationships: (1) free pathways, in which hypothesized causal (in fact counterfactual) relationships between variables are tested, and therefore are left 'free' to vary, and (2) relationships between variables that already have an estimated relationship, usually based on previous studies, which are 'fixed' in the model.
A modeler will often specify a set of theoretically plausible models in order to assess whether the model proposed is the best of the set of possible models. Not only must the modeler account for the theoretical reasons for building the model as it is, but the modeler must also take into account the number of data points and the number of parameters that the model must estimate to identify the model.
An identified model is a model where a specific parameter value uniquely identifies the model (recursive definition), and no other equivalent formulation can be given by a different parameter value. A data point is a variable with observed scores, like a variable containing the scores on a question or the number of times respondents buy a car. The parameter is the value of interest, which might be a regression coefficient between the exogenous and the endogenous variable or the factor loading (regression coefficient between an indicator and its factor). If there are fewer data points than the number of estimated parameters, the resulting model is "unidentified", since there are too few reference points to account for all the variance in the model. The solution is to constrain one of the paths to zero, which means that it is no longer part of the model.
Parameter estimation is done by comparing the actual covariance matrices representing the relationships between variables and the estimated covariance matrices of the best fitting model. This is obtained through numerical maximization via expectation–maximization of a fit criterion as provided by maximum likelihood estimation, quasi-maximum likelihood estimation, weighted least squares or asymptotically distribution-free methods. This is often accomplished by using a specialized SEM analysis program, of which several exist.
This section needs additional citations for verification .(February 2019)
Having estimated a model, analysts will want to interpret the model. Estimated paths may be tabulated and/or presented graphically as a path model. The impact of variables is assessed using path tracing rules (see path analysis).
It is important to examine the "fit" of an estimated model to determine how well it models the data. This is a basic task in SEM modeling, forming the basis for accepting or rejecting models and, more usually, accepting one competing model over another. The output of SEM programs includes matrices of the estimated relationships between variables in the model. Assessment of fit essentially calculates how similar the predicted data are to matrices containing the relationships in the actual data.
Formal statistical tests and fit indices have been developed for these purposes. Individual parameters of the model can also be examined within the estimated model in order to see how well the proposed model fits the driving theory. Most, though not all, estimation methods make such tests of the model possible.
Of course as in all statistical hypothesis tests, SEM model tests are based on the assumption that the correct and complete relevant data have been modeled. In the SEM literature, discussion of fit has led to a variety of different recommendations on the precise application of the various fit indices and hypothesis tests.
There are differing approaches to assessing fit. Traditional approaches to modeling start from a null hypothesis, rewarding more parsimonious models (i.e. those with fewer free parameters), to others such as AIC that focus on how little the fitted values deviate from a saturated model [ citation needed ] (i.e. how well they reproduce the measured values), taking into account the number of free parameters used. Because different measures of fit capture different elements of the fit of the model, it is appropriate to report a selection of different fit measures. Guidelines (i.e., "cutoff scores") for interpreting fit measures, including the ones listed below, are the subject of much debate among SEM researchers.
Some of the more commonly used measures of fit include:
For each measure of fit, a decision as to what represents a good-enough fit between the model and the data must reflect other contextual factors such as sample size, the ratio of indicators to factors, and the overall complexity of the model. For example, very large samples make the Chi-squared test overly sensitive and more likely to indicate a lack of model-data fit.
The model may need to be modified in order to improve the fit, thereby estimating the most likely relationships between variables. Many programs provide modification indices which may guide minor modifications. Modification indices report the change in χ² that result from freeing fixed parameters: usually, therefore adding a path to a model which is currently set to zero. Modifications that improve model fit may be flagged as potential changes that can be made to the model. Modifications to a model, especially the structural model, are changes to the theory claimed to be true. Modifications therefore must make sense in terms of the theory being tested, or be acknowledged as limitations of that theory. Changes to measurement model are effectively claims that the items/data are impure indicators of the latent variables specified by theory.
Models should not be led by MI, as Maccallum (1986) demonstrated: "even under favorable conditions, models arising from specification searches must be viewed with caution."
While researchers agree that large sample sizes are required to provide sufficient statistical power and precise estimates using SEM, there is no general consensus on the appropriate method for determining adequate sample size.Generally, the considerations for determining sample size include the number of observations per parameter, the number of observations required for fit indexes to perform adequately, and the number of observations per degree of freedom. Researchers have proposed guidelines based on simulation studies, professional experience, and mathematical formulas.
Sample size requirements to achieve a particular significance and power in SEM hypothesis testing are similar for the same model when any of the three algorithms (PLS-PA, LISREL or systems of regression equations) are used for testing.[ citation needed ]
The set of models are then interpreted so that claims about the constructs can be made, based on the best fitting model.
Caution should always be taken when making claims of causality even when experimentation or time-ordered studies have been done. The term causal model must be understood to mean "a model that conveys causal assumptions", not necessarily a model that produces validated causal conclusions. Collecting data at multiple time points and using an experimental or quasi-experimental design can help rule out certain rival hypotheses but even a randomized experiment cannot rule out all such threats to causal inference. Good fit by a model consistent with one causal hypothesis invariably entails equally good fit by another model consistent with an opposing causal hypothesis. No research design, no matter how clever, can help distinguish such rival hypotheses, save for interventional experiments.
As in any science, subsequent replication and perhaps modification will proceed from the initial finding.
Several software packages exist for fitting structural equation models. LISREL was the first such software, initially released in the 1970s.
There are also several packages for the R open source statistical environment. The OpenMx R package provides an open source and enhanced version of the Mx application. Another open source R package for SEM is lavaan.
Scholars consider it good practice to report which software package and version was used for SEM analysis because they have different capabilities and may use slightly different methods to perform similarly named techniques.
Psychological statistics is application of formulas, theorems, numbers and laws to psychology. Statistical Methods for psychology include development and application statistical theory and methods for modeling psychological data. These methods include psychometrics, Factor analysis, Experimental Designs, and Multivariate Behavioral Research. The article also discusses journals in the same field.
Simultaneous equations models are a type of statistical model in which the dependent variables are functions of other dependent variables, rather than just independent variables. This means some of the explanatory variables are jointly determined with the dependent variable, which in economics usually is the consequence of some underlying equilibrium mechanism. Take the typical supply and demand model: whilst typically one would determine the quantity supplied and demanded to be a function of the price set by the market, it is also possible for the reverse to be true, where producers observe the quantity that consumers demand and then set the price.
In statistics, path analysis is used to describe the directed dependencies among a set of variables. This includes models equivalent to any form of multiple regression analysis, factor analysis, canonical correlation analysis, discriminant analysis, as well as more general families of models in the multivariate analysis of variance and covariance analyses.
In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables. The most common form of regression analysis is linear regression, in which one finds the line that most closely fits the data according to a specific mathematical criterion. For example, the method of ordinary least squares computes the unique line that minimizes the sum of squared differences between the true data and that line. For specific mathematical reasons, this allows the researcher to estimate the conditional expectation of the dependent variable when the independent variables take on a given set of values. Less common forms of regression use slightly different procedures to estimate alternative location parameters or estimate the conditional expectation across a broader collection of non-linear models.
In statistics, econometrics, epidemiology and related disciplines, the method of instrumental variables (IV) is used to estimate causal relationships when controlled experiments are not feasible or when a treatment is not successfully delivered to every unit in a randomized experiment. Intuitively, IVs are used when an explanatory variable of interest is correlated with the error term, in which case ordinary least squares and ANOVA give biased results. A valid instrument induces changes in the explanatory variable but has no independent effect on the dependent variable, allowing a researcher to uncover the causal effect of the explanatory variable on the dependent variable.
Latent growth modeling is a statistical technique used in the structural equation modeling (SEM) framework to estimate growth trajectories. It is a longitudinal analysis technique to estimate growth over a period of time. It is widely used in the field of psychology, behavioral science, education and social science. It is also called latent growth curve analysis. The latent growth model was derived from theories of SEM. General purpose SEM software, such as OpenMx, lavaan, AMOS, Mplus, LISREL, or EQS among others may be used to estimate growth trajectories.
In statistics, a mediation model seeks to identify and explain the mechanism or process that underlies an observed relationship between an independent variable and a dependent variable via the inclusion of a third hypothetical variable, known as a mediator variable. Rather than a direct causal relationship between the independent variable and the dependent variable, a mediation model proposes that the independent variable influences the mediator variable, which in turn influences the dependent variable. Thus, the mediator variable serves to clarify the nature of the relationship between the independent and dependent variables.
In statistics, confirmatory factor analysis (CFA) is a special form of factor analysis, most commonly used in social research. It is used to test whether measures of a construct are consistent with a researcher's understanding of the nature of that construct. As such, the objective of confirmatory factor analysis is to test whether the data fit a hypothesized measurement model. This hypothesized model is based on theory and/or previous analytic research. CFA was first developed by Jöreskog (1969) and has built upon and replaced older methods of analyzing construct validity such as the MTMM Matrix as described in Campbell & Fiske (1959).
LISREL is a proprietary statistical software package used in structural equation modeling (SEM) for manifest and latent variables. It requires a "fairly high level of statistical sophistication".
Karl Gustav Jöreskog is a Swedish statistician. Jöreskog is a Professor Emeritus at Uppsala University, and a co-author of the LISREL statistical program. He is also a member of the Royal Swedish Academy of Sciences. Jöreskog received his bachelor's, master's, and doctoral degrees in Uppsala University. He is also a former student of Herman Wold. He was a statistician at Educational Testing Service (ETS) and a visiting professor at Princeton University.
OpenMx is an open source program for extended structural equation modeling. It runs as a package under R. Cross platform, it runs under Linux, Mac OS and Windows.
One application of multilevel modeling (MLM) is the analysis of repeated measures data. Multilevel modeling for repeated measures data is most often discussed in the context of modeling change over time ; however, it may also be used for repeated measures data in which time is not a factor.
Measurement invariance or measurement equivalence is a statistical property of measurement that indicates that the same construct is being measured across some specified groups. For example, measurement invariance can be used to study whether a given measure is interpreted in a conceptually similar manner by respondents representing different genders or cultural backgrounds. Violations of measurement invariance may preclude meaningful interpretation of measurement data. Tests of measurement invariance are increasingly used in fields such as psychology to supplement evaluation of measurement quality rooted in classical test theory.
In statistics, econometrics, epidemiology, genetics and related disciplines, causal graphs are probabilistic graphical models used to encode assumptions about the data-generating process.
The partial least squares path modeling or partial least squares structural equation modeling is a method for structural equation modeling that allows estimation of complex cause-effect relationships in path models with latent variables.
SmartPLS is a software with graphical user interface for variance-based structural equation modeling (SEM) using the partial least squares (PLS) path modeling method. Users can estimate models with their data by using basic PLS-SEM, weighted PLS-SEM (WPLS), consistent PLS-SEM (PLSc-SEM), and sumscores regression algorithms. The software computes standard results assessment criteria and it supports additional statistical analyses . Since SmartPLS is programmed in Java, it can be executed and run on different computer operating systems such as Windows and Mac.
WarpPLS is a software with graphical user interface for variance-based and factor-based structural equation modeling (SEM) using the partial least squares and factor-based methods. The software can be used in empirical research to analyse collected data and test hypothesized relationships. Since it runs on the MATLAB Compiler Runtime, it does not require the MATLAB software development application to be installed; and can be installed and used on various operating systems in addition to Windows, with virtual installations.
Structural Equations with Latent Variables is a book by Kenneth Bollen. It explains basic ideas and methods in the field of structural equation modeling and is considered to be an important technical reference. It is held to be a classic textbook on the topic.
In statistics, confirmatory composite analysis (CCA) is a sub-type of structural equation modeling (SEM). Although, historically, CCA emerged from a re-orientation and re-start of partial least squares path modeling (PLS-PM), it has become an independent approach and the two should not be confused. In many ways it is similar to, but also quite distinct from confirmatory factor analysis (CFA). It shares with CFA the process of model specification, model identification, model estimation, and model assessment. However, in contrast to CFA which always assumes the existence of latent variables, in CCA all variables can be observable, with their interrelationships expressed in terms of composites, i.e., linear compounds of subsets of the variables. The composites are treated as the fundamental objects and path diagrams can be used to illustrate their relationships. This makes CCA particularly useful for disciplines examining theoretical concepts that are designed to attain certain goals, so-called artifacts, and their interplay with theoretical concepts of behavioral sciences.