Discriminant validity

Last updated

In psychology, discriminant validity tests whether concepts or measurements that are not supposed to be related are actually unrelated.

Campbell and Fiske (1959) introduced the concept of discriminant validity within their discussion on evaluating test validity. They stressed the importance of using both discriminant and convergent validation techniques when assessing new tests. A successful evaluation of discriminant validity shows that a test of a concept is not highly correlated with other tests designed to measure theoretically different concepts.

In showing that two scales do not correlate, it is necessary to correct for attenuation in the correlation due to measurement error. It is possible to calculate the extent to which the two scales overlap by using the following formula where is correlation between x and y, is the reliability of x, and is the reliability of y:

Although there is no standard value for discriminant validity, a result less than 0.70 suggests that discriminant validity likely exists between the two scales. A result greater than 0.70, however, suggests that the two constructs overlap greatly and they are likely measuring the same thing, and therefore, discriminant validity between them cannot be claimed. [1]

Consider researchers developing a new scale designed to measure narcissism. They may want to show discriminant validity with a scale measuring self-esteem. Narcissism and self-esteem are theoretically different concepts, and therefore it is important that the researchers show that their new scale measures narcissism and not simply self-esteem.

First, the average inter-item correlations within and between the two scales can be calculated:

Narcissism — Narcissism: 0.47
Narcissism — Self-esteem: 0.30
Self-esteem — Self-esteem: 0.52

The correction for attenuation formula can then be applied:

Since 0.607 is less than 0.85, it can be concluded that discriminant validity exists between the scale measuring narcissism and the scale measuring self-esteem. The two scales measure theoretically different constructs.

Recommended approaches to test for discriminant validity on the construct level are AVE-SE comparisons (Fornell & Larcker, 1981; note: hereby the measurement error-adjusted inter-construct correlations derived from the CFA model should be used rather than raw correlations derived from the data.) [2] and the assessment of the HTMT ratio (Henseler et al., 2014). [3] Simulation tests reveal that the former performs poorly for variance-based structural equation models (SEM), e.g. PLS, but well for covariance-based SEM, e.g. Amos, and the latter performs well for both types of SEM. [3] [4] Voorhees et al. (2015) recommend combining both methods for covariance-based SEM with a HTMT cutoff of 0.85. [4] A recommended approach to test for discriminant validity on the item level is exploratory factor analysis (EFA).

See also

Related Research Articles

<span class="mw-page-title-main">Psychological statistics</span>

Psychological statistics is application of formulas, theorems, numbers and laws to psychology. Statistical methods for psychology include development and application statistical theory and methods for modeling psychological data. These methods include psychometrics, factor analysis, experimental designs, and Bayesian statistics. The article also discusses journals in the same field.

<span class="mw-page-title-main">Psychometrics</span> Theory and technique of psychological measurement

Psychometrics is a field of study within psychology concerned with the theory and technique of measurement. Psychometrics generally refers to specialized fields within psychology and education devoted to testing, measurement, assessment, and related activities. Psychometrics is concerned with the objective measurement of latent constructs that cannot be directly observed. Examples of latent constructs include intelligence, introversion, mental disorders, and educational achievement. The levels of individuals on nonobservable latent variables are inferred through mathematical modeling based on what is observed from individuals' responses to items on tests and scales.

<span class="mw-page-title-main">Correlation</span> Statistical concept

In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics it usually refers to the degree to which a pair of variables are linearly related. Familiar examples of dependent phenomena include the correlation between the height of parents and their offspring, and the correlation between the price of a good and the quantity the consumers are willing to purchase, as it is depicted in the so-called demand curve.

<span class="mw-page-title-main">Pearson correlation coefficient</span> Measure of linear correlation

In statistics, the Pearson correlation coefficient ― also known as Pearson's r, the Pearson product-moment correlation coefficient (PPMCC), the bivariate correlation, or colloquially simply as the correlation coefficient ― is a measure of linear correlation between two sets of data. It is the ratio between the covariance of two variables and the product of their standard deviations; thus, it is essentially a normalized measurement of the covariance, such that the result always has a value between −1 and 1. As with covariance itself, the measure can only reflect a linear correlation of variables, and ignores many other types of relationships or correlations. As a simple example, one would expect the age and height of a sample of teenagers from a high school to have a Pearson correlation coefficient significantly greater than 0, but less than 1.

In statistics and psychometrics, reliability is the overall consistency of a measure. A measure is said to have a high reliability if it produces similar results under consistent conditions:

"It is the characteristic of a set of test scores that relates to the amount of random error from the measurement process that might be embedded in the scores. Scores that are highly reliable are precise, reproducible, and consistent from one testing occasion to another. That is, if the testing process were repeated with a group of test takers, essentially the same results would be obtained. Various kinds of reliability coefficients, with values ranging between 0.00 and 1.00, are usually used to indicate the amount of error in the scores."

In the social sciences, scaling is the process of measuring or ordering entities with respect to quantitative attributes or traits. For example, a scaling technique might involve estimating individuals' levels of extraversion, or the perceived quality of products. Certain methods of scaling permit estimation of magnitudes on a continuum, while other methods provide only for relative ordering of the entities.

Cronbach's alpha, also known as tau-equivalent reliability or coefficient alpha, is a reliability coefficient that provides a method of measuring internal consistency of tests and measures. Numerous studies warn against using it unconditionally, and note that reliability coefficients based on structural equation modeling (SEM) are in many cases a suitable alternative.

Construct validity concerns how well a set of indicators represent or reflect a concept that is not directly measurable. Construct validation is the accumulation of evidence to support the interpretation of what a measure reflects. Modern validity theory defines construct validity as the overarching concern of validity research, subsuming all other types of validity evidence such as content validity and criterion validity.

Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification.

<span class="mw-page-title-main">Coefficient of determination</span> Indicator for how well data points fit a line or curve

In statistics, the coefficient of determination, denoted R2 or r2 and pronounced "R squared", is the proportion of the variation in the dependent variable that is predictable from the independent variable(s).

Convergent validity, for human cognition, especially within sociology, psychology, and other behavioral sciences, refers to the degree to which two measures that theoretically should be related, are in fact related. Convergent validity, along with discriminant validity, is a subtype of construct validity. Convergent validity can be established if two similar constructs correspond with one another, while discriminant validity applies to two dissimilar constructs that are easily differentiated.

In mathematics, a quadratic equation is a polynomial equation of the second degree. The general form is

In statistics, confirmatory factor analysis (CFA) is a special form of factor analysis, most commonly used in social science research. It is used to test whether measures of a construct are consistent with a researcher's understanding of the nature of that construct. As such, the objective of confirmatory factor analysis is to test whether the data fit a hypothesized measurement model. This hypothesized model is based on theory and/or previous analytic research. CFA was first developed by Jöreskog (1969) and has built upon and replaced older methods of analyzing construct validity such as the MTMM Matrix as described in Campbell & Fiske (1959).

<span class="mw-page-title-main">Multitrait-multimethod matrix</span> Statistical technique used to examine construct validity

The multitrait-multimethod (MTMM) matrix is an approach to examining construct validity developed by Campbell and Fiske (1959). It organizes convergent and discriminant validity evidence for comparison of how a measure relates to other measures. The conceptual approach has influenced experimental design and measurement theory in psychology, including applications in structural equation models.

The Narcissistic Personality Inventory (NPI) was developed in 1979 by Raskin and Hall, and since then, has become one of the most widely utilized personality measures for non-clinical levels of the trait narcissism. Since its initial development, the NPI has evolved from 220 items to the more commonly employed NPI-40 (1984) and NPI-16 (2006), as well as the novel NPI-1 inventory (2014). Derived from the DSM-III criteria for Narcissistic personality disorder (NPD), the NPI has been employed heavily by personality and social psychology researchers.

In applied statistics,, common-method variance (CMV) is the spurious "variance that is attributable to the measurement method rather than to the constructs the measures are assumed to represent" or equivalently as "systematic error variance shared among variables measured with and introduced as a function of the same method and/or source". For example, an electronic survey method might influence results for those who might be unfamiliar with an electronic survey interface differently than for those who might be familiar. If measures are affected by CMV or common-method bias, the intercorrelations among them can be inflated or deflated depending upon several factors. Although it is sometimes assumed that CMV affects all variables, evidence suggests that whether or not the correlation between two variables is affected by CMV is a function of both the method and the particular constructs being measured.

Measurement invariance or measurement equivalence is a statistical property of measurement that indicates that the same construct is being measured across some specified groups. For example, measurement invariance can be used to study whether a given measure is interpreted in a conceptually similar manner by respondents representing different genders or cultural backgrounds. Violations of measurement invariance may preclude meaningful interpretation of measurement data. Tests of measurement invariance are increasingly used in fields such as psychology to supplement evaluation of measurement quality rooted in classical test theory.

<span class="mw-page-title-main">Average variance extracted</span>

In statistics, average variance extracted (AVE) is a measure of the amount of variance that is captured by a construct in relation to the amount of variance due to measurement error.

In statistics, confirmatory composite analysis (CCA) is a sub-type of structural equation modeling (SEM). Although, historically, CCA emerged from a re-orientation and re-start of partial least squares path modeling (PLS-PM), it has become an independent approach and the two should not be confused. In many ways it is similar to, but also quite distinct from confirmatory factor analysis (CFA). It shares with CFA the process of model specification, model identification, model estimation, and model assessment. However, in contrast to CFA which always assumes the existence of latent variables, in CCA all variables can be observable, with their interrelationships expressed in terms of composites, i.e., linear compounds of subsets of the variables. The composites are treated as the fundamental objects and path diagrams can be used to illustrate their relationships. This makes CCA particularly useful for disciplines examining theoretical concepts that are designed to attain certain goals, so-called artifacts, and their interplay with theoretical concepts of behavioral sciences.

The Dark Triad Dirty Dozen (DTDD) is a brief 12-question personality inventory test to assesses the possible presence of three co-morbid socially maladaptive, dark triad traits: Machiavellianism, narcissism, and psychopathy. The DTDD was developed to identify the dark triad traits among subclinical adult populations. It is a screening test. High scores on the DTDD do not necessary correlate with clinical diagnoses.

References

  1. Hodson, G. (2021). Construct jangle or construct mangle? Thinking straight about (nonredundant) psychological constructs. Journal of Theoretical Social Psychology. Advance online publication. https://doi.org/10.1002/jts5.120
  2. Claes Fornell, David F. Larcker: Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. In: Journal of Marketing Research. 18, February 1981, S. 39-50.
  3. 1 2 Henseler, J., Ringle, C.M., Sarstedt, M., 2014. A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science 43 (1), 115–135.
  4. 1 2 Voorhees, C.M., Brady, M.K., Calantone, R., Ramirez, E., 2015. Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies. Journal of the Academy of Marketing Science 1–16.