Moderation (statistics)

Last updated

In statistics and regression analysis, moderation (also known as effect modification) occurs when the relationship between two variables depends on a third variable. The third variable is referred to as the moderator variable (or effect modifier) or simply the moderator (or modifier). [1] [2] The effect of a moderating variable is characterized statistically as an interaction; [1] that is, a categorical (e.g., sex, ethnicity, class) or continuous (e.g., age, level of reward) variable that is associated with the direction and/or magnitude of the relation between dependent and independent variables. Specifically within a correlational analysis framework, a moderator is a third variable that affects the zero-order correlation between two other variables, or the value of the slope of the dependent variable on the independent variable. In analysis of variance (ANOVA) terms, a basic moderator effect can be represented as an interaction between a focal independent variable and a factor that specifies the appropriate conditions for its operation. [3]

Contents

Example

Conceptual diagram of a simple moderation model in which the effect of the focal antecedent (X) on the outcome (Y) is influenced or dependent on a moderator (W). Simple moderation model conceptual diagram.jpg
Conceptual diagram of a simple moderation model in which the effect of the focal antecedent (X) on the outcome (Y) is influenced or dependent on a moderator (W).
A statistical diagram of a simple moderation model. Statistical Diagram, Simple Moderation Model.png
A statistical diagram of a simple moderation model.

Moderation analysis in the behavioral sciences involves the use of linear multiple regression analysis or causal modelling. [1] To quantify the effect of a moderating variable in multiple regression analyses, regressing random variable Y on X, an additional term is added to the model. This term is the interaction between X and the proposed moderating variable. [1]

Thus, for a response Y and two variables x1 and moderating variable x2,:

In this case, the role of x2 as a moderating variable is accomplished by evaluating b3, the parameter estimate for the interaction term. [1] See linear regression for discussion of statistical evaluation of parameter estimates in regression analyses.

Multicollinearity in moderated regression

In moderated regression analysis, a new interaction predictor () is calculated. However, the new interaction term may be correlated with the two main effects terms used to calculate it. This is the problem of multicollinearity in moderated regression. Multicollinearity tends to cause coefficients to be estimated with higher standard errors and hence greater uncertainty.

Mean-centering (subtracting raw scores from the mean) may reduce multicollinearity, resulting in more interpretable regression coefficients. [4] [5] However, it does not affect the overall model fit.

Post-hoc probing of interactions

Like simple main effect analysis in ANOVA, in post-hoc probing of interactions in regression, we are examining the simple slope of one independent variable at the specific values of the other independent variable. Below is an example of probing two-way interactions. In what follows the regression equation with two variables A and B and an interaction term A*B,

will be considered. [6]

Two categorical independent variables

If both of the independent variables are categorical variables, we can analyze the results of the regression for one independent variable at a specific level of the other independent variable. For example, suppose that both A and B are single dummy coded (0,1) variables, and that A represents ethnicity (0 = European Americans, 1 = East Asians) and B represents the condition in the study (0 = control, 1 = experimental). Then the interaction effect shows whether the effect of condition on the dependent variable Y is different for European Americans and East Asians and whether the effect of ethnic status is different for the two conditions. The coefficient of A shows the ethnicity effect on Y for the control condition, while the coefficient of B shows the effect of imposing the experimental condition for European American participants.

To probe if there is any significant difference between European Americans and East Asians in the experimental condition, we can simply run the analysis with the condition variable reverse-coded (0 = experimental, 1 = control), so that the coefficient for ethnicity represents the ethnicity effect on Y in the experimental condition. In a similar vein, if we want to see whether the treatment has an effect for East Asian participants, we can reverse code the ethnicity variable (0 = East Asians, 1 = European Americans).

One categorical and one continuous independent variable

A statistical diagram that depicts a moderation model with X as a multicategorical independent variable. Statistical Diagram of Moderation with a Multicategorical X.png
A statistical diagram that depicts a moderation model with X as a multicategorical independent variable.

If the first independent variable is a categorical variable (e.g. gender) and the second is a continuous variable (e.g. scores on the Satisfaction With Life Scale (SWLS)), then b1 represents the difference in the dependent variable between males and females when life satisfaction is zero. However, a zero score on the Satisfaction With Life Scale is meaningless as the range of the score is from 7 to 35. This is where centering comes in. If we subtract the mean of the SWLS score for the sample from each participant's score, the mean of the resulting centered SWLS score is zero. When the analysis is run again, b1 now represents the difference between males and females at the mean level of the SWLS score of the sample.

An example of conceptual moderation model with one categorical and one continuous independent variable. Conceptual Model - Categorical & Continuous Antecedent.jpg
An example of conceptual moderation model with one categorical and one continuous independent variable.

Cohen et al. (2003) recommended using the following to probe the simple effect of gender on the dependent variable (Y) at three levels of the continuous independent variable: high (one standard deviation above the mean), moderate (at the mean), and low (one standard deviation below the mean). [7] If the scores of the continuous variable are not standardized, one can just calculate these three values by adding or subtracting one standard deviation of the original scores; if the scores of the continuous variable are standardized, one can calculate the three values as follows: high = the standardized score minus 1, moderate (mean = 0), low = the standardized score plus 1. Then one can explore the effects of gender on the dependent variable (Y) at high, moderate, and low levels of the SWLS score. As with two categorical independent variables, b2 represents the effect of the SWLS score on the dependent variable for females. By reverse coding the gender variable, one can get the effect of the SWLS score on the dependent variable for males.

Coding in moderated regression

Multicategorical Model.png
A statistical diagram that depicts a moderation model with W with three levels, as a multi-categorical independent variable. A statistical diagram that depicts a moderation model with W with 3 levels, as a multi-categorical independent variable.png
A statistical diagram that depicts a moderation model with W with three levels, as a multi-categorical independent variable.

When treating categorical variables such as ethnic groups and experimental treatments as independent variables in moderated regression, one needs to code the variables so that each code variable represents a specific setting of the categorical variable. There are three basic ways of coding: dummy-variable coding, effects coding, and contrast coding. Below is an introduction to these coding systems. [8] [9]

Orthogonal contrast.png

Dummy coding is used when one has a reference group or one condition in particular (e.g. a control group in the experiment) that is to be compared to each of the other experimental groups. In this case, the intercept is the mean of the reference group, and each of the unstandardized regression coefficients is the difference in the dependent variable between one of the treatment groups and the mean of the reference group (or control group). This coding system is similar to ANOVA analysis, and is appropriate when researchers have a specific reference group and want to compare each of the other groups with it.

Effects coding is used when one does not have a particular comparison or control group and does not have any planned orthogonal contrasts. The intercept is the grand mean (the mean of all the conditions). The regression coefficient is the difference between the mean of one group and the mean of all the group means (e.g. the mean of group A minus the mean of all groups). This coding system is appropriate when the groups represent natural categories.

Contrast coding is used when one has a series of orthogonal contrasts or group comparisons that are to be investigated. In this case, the intercept is the unweighted mean of the individual group means. The unstandardized regression coefficient represents the difference between the unweighted mean of the means of one group (A) and the unweighted mean of another group (B), where A and B are two sets of groups in the contrast. This coding system is appropriate when researchers have an a priori hypothesis concerning the specific differences among the group means.

Two continuous independent variables

An example of two continuous variables moderation model.png
A conceptual diagram of an additive multiple moderation model An Additive Multiple Moderation Model.png
A conceptual diagram of an additive multiple moderation model
An example of a two-way interaction effect plot Two-way interaction effect example.png
An example of a two-way interaction effect plot

If both of the independent variables are continuous, it is helpful for interpretation to either center or standardize the independent variables, X and Z. (Centering involves subtracting the overall sample mean score from the original score; standardizing does the same followed by dividing by the overall sample standard deviation.) By centering or standardizing the independent variables, the coefficient of X or Z can be interpreted as the effect of that variable on Y at the mean level of the other independent variable. [10]

To probe the interaction effect, it is often helpful to plot the effect of X on Y at low and high values of Z (some people prefer to also plot the effect at moderate values of Z, but this is not necessary). Often values of Z that are one standard deviation above and below the mean are chosen for this, but any sensible values can be used (and in some cases there are more meaningful values to choose). The plot is usually drawn by evaluating the values of Y for high and low values of both X and Z, and creating two lines to represent the effect of X on Y at the two values of Z. Sometimes this is supplemented by simple slope analysis, which determines whether the effect of X on Y is statistically significant at particular values of Z. A common technique for simple slope analysis is the Johnson-Neyman approach. [11] Various internet-based tools exist to help researchers plot and interpret such two-way interactions. [12]

A conceptual diagram of a moderated moderation model, otherwise known as a three-way interaction. Conceptual Moderated Moderation Model (Three-way Interaction).jpg
A conceptual diagram of a moderated moderation model, otherwise known as a three-way interaction.

Higher-level interactions

The principles for two-way interactions apply when we want to explore three-way or higher-level interactions. For instance, if we have a three-way interaction between A, B, and C, the regression equation will be as follows:

Spurious higher-order effects

It is worth noting that the reliability of the higher-order terms depends on the reliability of the lower-order terms. For example, if the reliability for variable A is 0.70, the reliability for variable B is 0.80, and their correlation is r = 0.2, then the reliability for the interaction variable A * B is . [13] In this case, low reliability of the interaction term leads to low power; therefore, we may not be able to find the interaction effects between A and B that actually exist. The solution for this problem is to use highly reliable measures for each independent variable.

Another caveat for interpreting the interaction effects is that when variable A and variable B are highly correlated, then the A * B term will be highly correlated with the omitted variable A2; consequently what appears to be a significant moderation effect might actually be a significant nonlinear effect of A alone. If this is the case, it is worth testing a nonlinear regression model by adding nonlinear terms in individual variables into the moderated regression analysis to see if the interactions remain significant. If the interaction effect A*B is still significant, we will be more confident in saying that there is indeed a moderation effect; however, if the interaction effect is no longer significant after adding the nonlinear term, we will be less certain about the existence of a moderation effect and the nonlinear model will be preferred because it is more parsimonious.

Moderated regression analyses also tend to include additional variables, which are conceptualized as covariates of no interest. However, the presence of these covariates can induce spurious effects when either (1) the covariate (C) is correlated with one of the primary variables of interest (e.g. variable A or B), or (2) when the covariate itself is a moderator of the correlation between either A or B with Y. [14] [15] [16] The solution is to include additional interaction terms in the model, for the interaction between each confounder and the primary variables as follows:

See also

Related Research Articles

<span class="mw-page-title-main">Logistic regression</span> Statistical model for a binary dependent variable

In statistics, the logistic model is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression is estimating the parameters of a logistic model. Formally, in binary logistic regression there is a single binary dependent variable, coded by an indicator variable, where the two values are labeled "0" and "1", while the independent variables can each be a binary variable or a continuous variable. The corresponding probability of the value labeled "1" can vary between 0 and 1, hence the labeling; the function that converts log-odds to probability is the logistic function, hence the name. The unit of measurement for the log-odds scale is called a logit, from logistic unit, hence the alternative names. See § Background and § Definition for formal mathematics, and § Example for a worked example.

Analysis of covariance (ANCOVA) is a general linear model which blends ANOVA and regression. ANCOVA evaluates whether the means of a dependent variable (DV) are equal across levels of one or more categorical independent variables and across one or more continuous variables. For example, the categorical variable(s) might describe treatment and the continuous variable(s) might be covariates or nuisance variables; or vice versa. Mathematically, ANCOVA decomposes the variance in the DV into variance explained by the CV(s), variance explained by the categorical IV, and residual variance. Intuitively, ANCOVA can be thought of as 'adjusting' the DV by the group means of the CV(s).

<span class="mw-page-title-main">Dependent and independent variables</span> Concept in mathematical modeling, statistical modeling and experimental sciences

Dependent and independent variables are variables in mathematical modeling, statistical modeling and experimental sciences. Dependent variables are studied under the supposition or demand that they depend, by some law or rule, on the values of other variables. Independent variables, in turn, are not seen as depending on any other variable in the scope of the experiment in question. In this sense, some common independent variables are time, space, density, mass, fluid flow rate, and previous values of some observed value of interest to predict future values.

<span class="mw-page-title-main">Interaction (statistics)</span> Statistical term

In statistics, an interaction may arise when considering the relationship among three or more variables, and describes a situation in which the effect of one causal variable on an outcome depends on the state of a second causal variable. Although commonly thought of in terms of causal relationships, the concept of an interaction can also describe non-causal associations. Interactions are often considered in the context of regression analyses or factorial experiments.

In statistics, a categorical variable is a variable that can take on one of a limited, and usually fixed, number of possible values, assigning each individual or other unit of observation to a particular group or nominal category on the basis of some qualitative property. In computer science and some branches of mathematics, categorical variables are referred to as enumerations or enumerated types. Commonly, each of the possible values of a categorical variable is referred to as a level. The probability distribution associated with a random categorical variable is called a categorical distribution.

Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification.

In statistics, multicollinearity is a phenomenon in which one predictor variable in a multiple regression model can be linearly predicted from the others with a substantial degree of accuracy. In this situation, the coefficient estimates of the multiple regression may change erratically in response to small changes in the model or the data. Multicollinearity does not reduce the predictive power or reliability of the model as a whole, at least within the sample data set; it only affects calculations regarding individual predictors. That is, a multivariable regression model with collinear predictors can indicate how well the entire bundle of predictors predicts the outcome variable, but it may not give valid results about any individual predictor, or about which predictors are redundant with respect to others.

In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model by the principle of least squares: minimizing the sum of the squares of the differences between the observed dependent variable in the input dataset and the output of the (linear) function of the independent variable.

In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes. That is, it is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables.

Multilevel models are statistical models of parameters that vary at more than one level. An example could be a model of student performance that contains measures for individual students as well as measures for classrooms within which the students are grouped. These models can be seen as generalizations of linear models, although they can also extend to non-linear models. These models became much more popular after sufficient computing power and software became available.

Proportional hazards models are a class of survival models in statistics. Survival models relate the time that passes, before some event occurs, to one or more covariates that may be associated with that quantity of time. In a proportional hazards model, the unique effect of a unit increase in a covariate is multiplicative with respect to the hazard rate. For example, taking a drug may halve one's hazard rate for a stroke occurring, or, changing the material from which a manufactured component is constructed may double its hazard rate for failure. Other types of survival models such as accelerated failure time models do not exhibit proportional hazards. The accelerated failure time model describes a situation where the biological or mechanical life history of an event is accelerated.

Omnibus tests are a kind of statistical test. They test whether the explained variance in a set of data is significantly greater than the unexplained variance, overall. One example is the F-test in the analysis of variance. There can be legitimate significant effects within a model even if the omnibus test is not significant. For instance, in a model with two independent variables, if only one variable exerts a significant effect on the dependent variable and the other does not, then the omnibus test may be non-significant. This fact does not affect the conclusions that may be drawn from the one significant variable. In order to test effects within an omnibus test, researchers often use contrasts.

<span class="mw-page-title-main">Mediation (statistics)</span> Statistical model

In statistics, a mediation model seeks to identify and explain the mechanism or process that underlies an observed relationship between an independent variable and a dependent variable via the inclusion of a third hypothetical variable, known as a mediator variable. Rather than a direct causal relationship between the independent variable and the dependent variable, a mediation model proposes that the independent variable influences the mediator variable, which in turn influences the dependent variable. Thus, the mediator variable serves to clarify the nature of the relationship between the independent and dependent variables.

In causal models, controlling for a variable means binning data according to measured values of the variable. This is typically done so that the variable can no longer act as a confounder in, for example, an observational study or experiment.

In statistics, the variance inflation factor (VIF) is the ratio (quotient) of the variance of estimating some parameter in a model that includes multiple other terms (parameters) by the variance of a model constructed using only one term. It quantifies the severity of multicollinearity in an ordinary least squares regression analysis. It provides an index that measures how much the variance of an estimated regression coefficient is increased because of collinearity. Cuthbert Daniel claims to have invented the concept behind the variance inflation factor, but did not come up with the name.

In statistics, principal component regression (PCR) is a regression analysis technique that is based on principal component analysis (PCA). More specifically, PCR is used for estimating the unknown regression coefficients in a standard linear regression model.

In statistics and machine learning, lasso is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the resulting statistical model. It was originally introduced in geophysics, and later by Robert Tibshirani, who coined the term.

In statistics and in machine learning, a linear predictor function is a linear function of a set of coefficients and explanatory variables, whose value is used to predict the outcome of a dependent variable. This sort of function usually comes in linear regression, where the coefficients are called regression coefficients. However, they also occur in various types of linear classifiers, as well as in various other models, such as principal component analysis and factor analysis. In many of these models, the coefficients are referred to as "weights".

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables. The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

In statistics, the class of vector generalized linear models (VGLMs) was proposed to enlarge the scope of models catered for by generalized linear models (GLMs). In particular, VGLMs allow for response variables outside the classical exponential family and for more than one parameter. Each parameter can be transformed by a link function. The VGLM framework is also large enough to naturally accommodate multiple responses; these are several independent responses each coming from a particular statistical distribution with possibly different parameter values.

References

  1. 1 2 3 4 5 Cohen, Jacob; Cohen, Patricia; Leona S. Aiken; West, Stephen H. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, N.J: L. Erlbaum Associates. ISBN   0-8058-2223-2.
  2. Schandelmaier, Stefan; Briel, Matthias; Varadhan, Ravi; Schmid, Christopher H.; Devasenapathy, Niveditha; Hayward, Rodney A.; Gagnier, Joel; Borenstein, Michael; van der Heijden, Geert J.M.G.; Dahabreh, Issa J.; Sun, Xin; Sauerbrei, Willi; Walsh, Michael; Ioannidis, John P.A.; Thabane, Lehana (2020-08-10). "Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses". Canadian Medical Association Journal. 192 (32): E901–E906. doi:10.1503/cmaj.200077. ISSN   0820-3946. PMC   7829020 . PMID   32778601.
  3. Baron, R. M., & Kenny, D. A. (1986). "The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations", Journal of Personality and Social Psychology, 5 (6), 1173–1182 (page 1174)
  4. Iacobucci, Dawn; Schneider, Matthew J.; Popovich, Deidre L.; Bakamitsos, Georgios A. (2016). "Mean centering helps alleviate "micro" but not "macro" multicollinearity". Behavior Research Methods. 48 (4): 1308–1317. doi: 10.3758/s13428-015-0624-x . ISSN   1554-3528. PMID   26148824.
  5. Olvera Astivia, Oscar L.; Kroc, Edward (2019). "Centering in Multiple Regression Does Not Always Reduce Multicollinearity: How to Tell When Your Estimates Will Not Benefit From Centering". Educational and Psychological Measurement. 79 (5): 813–826. doi:10.1177/0013164418817801. ISSN   0013-1644. PMC   6713984 . PMID   31488914.
  6. Taylor, Alan. "Testing and Interpreting Interactions in Regression-In a Nutshell" (PDF).
  7. Cohen Jacob; Cohen Patricia; West Stephen G.; Aiken Leona S. Applied multiple regression/correlation analysis for the behavioral sciences (3. ed.). Mahwah, NJ [u.a.]: Erlbaum. pp. 255–301. ISBN   0-8058-2223-2.
  8. Aiken L.S., West., S.G. (1996). Multiple regression testing and interpretation (1. paperback print. ed.). Newbury Park, Calif. [u.a.]: Sage Publications, Inc. ISBN   0-7619-0712-2.{{cite book}}: CS1 maint: multiple names: authors list (link)
  9. Cohen Jacob; Cohen Patricia; West Stephen G.; Aiken Leona S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3. ed.). Mahwah, NJ [u.a.]: Erlbaum. pp. 302–353. ISBN   0-8058-2223-2.
  10. Dawson, J. F. (2013). Moderation in management research: What, why, when and how. Journal of Business and Psychology. doi : 10.1007/s10869-013-9308-7.
  11. Johnson, Palmer O.; Fay, Leo C. (1950-12-01). "The Johnson-Neyman technique, its theory and application". Psychometrika. 15 (4): 349–367. doi:10.1007/BF02288864. ISSN   1860-0980. PMID   14797902. S2CID   43748836.
  12. "Interpreting interaction effects".
  13. Busemeyer, Jerome R.; Jones, Lawrence E. (1983). "Analysis of multiplicative combination rules when the causal variables are measured with error". Psychological Bulletin. 93 (3): 549–562. doi:10.1037/0033-2909.93.3.549. ISSN   1939-1455.
  14. Keller, Matthew C. (2014). "Gene × Environment Interaction Studies Have Not Properly Controlled for Potential Confounders: The Problem and the (Simple) Solution". Biological Psychiatry. 75 (1): 18–24. doi:10.1016/j.biopsych.2013.09.006. PMC   3859520 . PMID   24135711.
  15. Yzerbyt, Vincent Y.; Muller, Dominique; Judd, Charles M. (2004). "Adjusting researchers' approach to adjustment: On the use of covariates when testing interactions". Journal of Experimental Social Psychology. 40 (3): 424–431. doi:10.1016/j.jesp.2003.10.001.
  16. Hull, Jay G.; Tedlie, Judith C.; Lehn, Daniel A. (1992). "Moderator Variables in Personality Research: The Problem of Controlling for Plausible Alternatives". Personality and Social Psychology Bulletin. 18 (2): 115–117. doi:10.1177/0146167292182001. ISSN   0146-1672. S2CID   145366173.