Cultural consensus theory

Last updated

Cultural consensus theory is an approach to information pooling or aggregation which supports a framework for the measurement and evaluation of beliefs as cultural; shared to some extent by a group of individuals. [1] [2] [3] The initial or "formal" cultural consensus model is a mathematical model that proceeds from assumptions and axioms to derive parameters to represent shared knowledge (answers to questions) and individual knowledge concerning those answers (cultural competence). Cultural consensus models optimally aggregate of responses from individuals to a series of related questions to estimate (1) the culturally appropriate answers to the questions (when the answers are unknown) and (2) individual competence (cultural knowledge) in answering those questions. The theory is applicable when there is sufficient agreement across respondents to estimate answers. The agreement between pairs of individuals is used to estimate whether a single, shared set of answers or beliefs is present and to estimate individual cultural competence. Answers are estimated by weighting responses of individuals by their competence prior to aggregating responses. An advantage of cultural consensus models over other aggregation methods is that weighting responses by individual competence prior to aggregation improves the efficiency and accuracy of the aggregation and allows for estimation of cultural beliefs from small samples. [4] Cultural consensus theory refers to a family of models all based on the agreement between respondents and a "weighted" aggregate of their responses, but which may differ in the types of response data they accommodate and the parameters that are estimated.

Theory

Cultural consensus theory assumes that cultural beliefs are learned and shared across people and that there is a common understanding of what the world and society are all about. [1] Since the amount of information in a culture is too large for any one individual to master, individuals know different subsets of the cultural knowledge and vary in their cultural knowledge or cultural competence. Cultural beliefs are beliefs held by a majority of culture members. Given a set of questions, on the same topic, shared cultural beliefs or norms regarding the answers can be estimated by aggregating the responses across a sample of culture members. When an agreement is close to absolute, estimating answers is straightforward. The problem addressed by cultural consensus theory is how to estimate beliefs when there is some degree of heterogeneity present in responses. In general, cultural consensus theory provides a framework for determining whether responses are sufficiently homogeneous to estimate a single set of shared “culturally correct” answers and then estimating the answers and individual cultural competence in answering the questions.

Cultural consensus models do not create consensus or explain why consensus exists; they simply facilitate the discovery and description of possible consensus. A high degree of agreement among respondents must be present in their responses in order to use consensus theory – only with high agreement does it make sense to aggregate responses to estimate beliefs of the group. Although there are statistical methods to evaluate whether agreement among raters is greater than chance (Binomial test, Friedman test, or Kendall's coefficient of concordance), these methods do not provide a best estimate of the “true” answers nor do they estimate respondent competence. Cultural consensus theory estimates competence from the agreement between respondents and then, answers are estimated by “weighting” individual responses by competence prior to aggregation.

An important feature in the aggregation of responses is that the combined responses of individuals will be more accurate than the responses of each individual included in the aggregation. This fact has been observed in political science by Condorcet’s 1785 “jury problem” and in psychology in reliability theory. In reliability theory, the reliability coefficient and the Spearman–Brown prophesy formula provide a mathematical estimate of the accuracy or validity of aggregated responses from the number of units being combined and the level of agreement among the units. In this case, the accuracy of aggregated responses can be calculated from the number of respondents and the average Pearson correlation coefficient between all pairs of respondents (across questions).

Assumptions

In the initial formulation of the cultural consensus model, that is most widely disseminated, at least three assumptions must be met: [1]

  1. Informants must be asked a series of questions. The questions should all be on the same topic and at the same level of difficulty. This assumption concerns the homogeneity of items and means that items should represent only one topic or domain of knowledge and that competency should be consistent across items, so that if someone does well on one subset of questions, they should also do well on another subset of questions. Responses to questions are not corrected, recoded, transformed, or reflected as they are with knowledge tests and attitudinal scales because the purpose is to use the original responses to estimate culturally correct answers.
  2. Each informant should provide answers independently of all other informants. This means that answers should be provided by individuals and not groups, and without consultation with others. Cultural consensus models are not appropriate for group interviews.
  3. The cultural consensus model is applicable only if there is a single set of answers to the questions. This means that there must be a high level of consistency (agreement) in responses among informants, indicating a shared or common truth. An aggregation of responses is not a valid estimate unless there is reasonable consistency in the underlying data. An initial step in applying consensus theory is to check whether there is a high degree of agreement among the informants (e.g., to verify that there is only one response pattern present). (Newer versions in the family of consensus models can estimate multiple sets of answers for subculture beliefs. See below.)

Models

Cultural consensus theory encompasses formal and informal models. [3] The formal cultural consensus model is based on the decision-making process for answering questions. [1] [2] [4] This version is limited to categorical-type response data: multiple-choice questions (including those with dichotomous true/false or yes/no responses) and responses to open-ended questions (with a single word or short phrase response for each question). This version of the model has additional assumptions that must be met, i.e., no response bias, although results are robust when low to moderate response bias is present. [5] The formal model is based on a mathematical model with a set of logical axioms and derived propositions that allow for the estimation of model parameters. The formal model has direct parallels in signal detection theory and latent class analysis.

An informal version of the model is available as a set of analytic procedures and obtains similar information with fewer assumptions. [6] The informal model parallels a factor analysis on people (without rotation) [7] and thus has similarities to Q factor analysis (as in Q Methodology). The informal version of the model can accommodate interval estimates and ranked response data. Both approaches provide estimates of the culturally correct answers and estimates of individual differences in the accuracy of reported information. A reliability analysis on respondents also can be used to approximate results with factor analysis on respondents. The reliability analysis, however, uses the simple, unweighted average or modal response to estimate the culturally correct answers.

Competence

Cultural competence or cultural knowledge is estimated from the similarity in responses between pairs of respondents since their agreement is a function of their individual competencies. In the formal model competence can be interpreted as the proportion of answers correctly identified by a respondent and in the informal model competence indicates how well each respondent’s answers correspond to the culturally “correct” answers. In the formal model, the similarity is the probability that matched responses occur (match method. [1] or the probability of particular response combinations occur (covariance method [2] [4] ). Simple match or covariance measures are then corrected for guessing or the proportion of estimated positive responses in the answer set, respectively. In the informal model, similarity is calculated with a Pearson correlation coefficient. [6]

A matrix of agreement coefficients between all pairs of respondents is then factored with a minimum residual factoring method (principal axis factoring without rotation) to solve for the unknown competence values on the main diagonal. (For the informal model, the maximum likelihood factor analysis algorithm is preferred, but principal axis factoring can be used as well.) To determine whether the solution meets cultural consensus criteria, that only a single set of answers is present, a goodness of fit rule is used to determine whether a single factor (representing a single response pattern) is present. If the ratio of the first to second eigenvalues is large with subsequently small values and all first factor loadings are positive, then it is assumed that the data contain only a single factor or a single response pattern. When a reliability analysis is used, competence is estimated by how well each respondent’s answers correspond to the culturally “correct” answers (the item-to-total correlation).

Culturally Correct Answers

Individual competence values are used to weight the responses and estimate the culturally correct answers. In the formal model, a Bayesian confidence level (Bayesian adjusted probabilities) is obtained for each answer from the pattern of responses and the individual competence scores. In the informal model, responses are weighted using a linear model. When factoring a correlation matrix, the estimated (weighted) answers appear as the first set of factor scores. Note that factor scores are usually provided as normalized variables (mean of zero), but may be transformed back to your original data collection units. When a reliability analysis is used, answers are estimated with the simple, unweighted mean or modal response to each question.

Analysis

Cultural consensus procedures have been implemented in a variety of software applications. The formal consensus model is currently only available in the software packages ANTHROPAC or UCINET. Analysis procedures for the informal model are available in most statistical packages and are thus, widely available. The informal model can be run within a factor analysis procedure, requesting the minimum-residual (principal axis factoring) algorithm method that solves for the missing diagonal without rotation. When factor analysis (or a reliability analysis) is used for consensus applications, ~~the data must be transposed, so that people are the variables (the columns in the data matrix) and questions are the unit of analysis (the rows in a data matrix) ~~ .

Cultural consensus theory provides estimated sample size information to achieve different degrees of accuracy. Sample size determination in a consensus analysis is similar to other types of analyses; namely, that when variability is low, power is high and small samples will suffice. Here, variability is the agreement (competence) among subjects. An added advantage is that weighted responses achieve accurate results with relatively small sample sizes. For the formal model, sample size can be conservatively estimated with a low level of agreement (e.g., average competence level of .50), a high proportion of items to be correctly classified (.95), and with high confidence (.999) so that a minimum sample size of 29 (per subgroup) would be necessary. [1] For higher levels of competence and lower levels of accuracy and confidence, smaller samples sizes are necessary. Similarly, sample size can be estimated with reliability theory and the Spearman–Brown prophesy formula (applied to people instead of items). Conservatively, using a low level of agreement (an average correlation of .25 between people, comparable to an average competence of .50) [8] and a high degree of desired validity (.95 correlation between the estimated answers and the true answers), a study would require a minimum sample size of 30 subjects. [3]

Impact and Extensions

Cultural consensus theory offers a systematic framework for estimating cultural beliefs and provides a quantitative estimate of those beliefs and their accuracy. Cultural consensus theory has had a large impact on anthropology and has diffused widely through other disciplines. Because cultural consensus theory has provided a technique for the quantifying cultural beliefs, investigators have been able to measure variation around normative cultural beliefs and link that variation to other social and psychological variables. For example, those whose lifestyle corresponds more closely to cultural ideals and norms tend to experience less stress and fewer depressive symptoms. [9] [10]

The formal and informal models described above focus on the most widely disseminated, original cultural consensus models. Newer versions and implementations have expanded the available models to a fully Bayesian model. [11] Results from this model now suggest that models are best estimated with large eigenvalue ratios (much larger than 3:1, possibly as small as 5:1). Models can accommodate multiple sets of answers (subgroups or subcultures), [12] [13] and can estimate parameters for guessing and response bias. [14] Models are also available that can accommodate ordinal data [15] and social network data. [16] [17]

References

  1. 1 2 3 4 5 6 Romney, A. Kimball; Weller, Susan C.; Batchelder, William H. (1986). "Culture as Consensus: A Theory of Culture and Informant Accuracy". American Anthropologist. 88 (2): 313–338. doi:10.1525/aa.1986.88.2.02a00020. ISSN   0002-7294.
  2. 1 2 3 Batchelder, William H., and A. Kimball Romney. (1986). "The statistical analysis of a general Condorcet model for dichotomous choice situations." Information pooling and group decision making. pp. 103–112.{{cite book}}: CS1 maint: date and year (link) CS1 maint: multiple names: authors list (link)
  3. 1 2 3 Weller, Susan C. (2007) "Cultural consensus theory: Applications and frequently asked questions." Field Methods 19 (4): 339-368. https://doi.org/10.1177/1525822X07303502
  4. 1 2 3 Batchelder, William H., and A. Kimball Romney. (1988) "Test theory without an answer key." Psychometrika 53 (1): 71-92. DOI: https://doi.org/10.1007/BF02294195
  5. ↑ Weller, SC and Mann NC. (1997) Assessing rater performance without a “gold standard” using consensus theory. Medical Decision Making; 17:71–79. https://doi.org/10.1177/0272989X9701700108
  6. 1 2 Romney, A. Kimball; Batchelder, William H.; Weller, Susan C. (1987). "Recent Applications of Cultural Consensus Theory". American Behavioral Scientist. 31 (2): 163–177. doi:10.1177/000276487031002003. ISSN   0002-7642.
  7. ↑ Romney, A. Kimball (1999). "Culture Consensus as a Statistical Model". Current Anthropology. 40 (Supplement: Special Issue: Culture. A Second Chance?): S103. doi:10.1086/200063.
  8. ↑ Weller, Susan C. (1987). "Shared Knowledge, Intracultural Variation, and Knowledge Aggregation". American Behavioral Scientist. 31 (2): 178–193. doi:10.1177/000276487031002004. ISSN   0002-7642.
  9. ↑ Dressler, William W. (2020). "Cultural Consensus and Cultural Consonance: Advancing a Cognitive Theory of Culture". Field Methods. 32 (4): 383–398. doi:10.1177/1525822X20935599. ISSN   1525-822X.
  10. ↑ Dressler, William W. (2017). Culture and the Individual: Theory and Method of Cultural Consonance (1 ed.). Routledge. doi:10.4324/9781315164007. ISBN   978-1-315-16400-7.
  11. ↑ Oravecz, Zita; Vandekerckhove, Joachim; Batchelder, William H. (2014). "Bayesian Cultural Consensus Theory". Field Methods. 26 (3): 207–222. doi:10.1177/1525822X13520280. ISSN   1525-822X.
  12. ↑ Anders, Royce; Batchelder, William H. (2012). "Cultural consensus theory for multiple consensus truths". Journal of Mathematical Psychology. 56 (6): 452–469. doi:10.1016/j.jmp.2013.01.004.
  13. ↑ Batchelder, William H.; Anders, Royce (2012). "Cultural Consensus Theory: Comparing different concepts of cultural truth". Journal of Mathematical Psychology. 56 (5): 316–332. doi:10.1016/j.jmp.2012.06.002.
  14. ↑ Karabatsos, George; Batchelder, William H. (2003). "Markov Chain Estimation for Test Theory Without An Answer Key". Psychometrika. 68 (3): 373–389. doi:10.1007/BF02294733. ISSN   0033-3123.
  15. ↑ Anders, Royce; Batchelder, William H. (2015). "Cultural Consensus Theory for the Ordinal Data Case" . Psychometrika. 80 (1): 151–181. doi:10.1007/s11336-013-9382-9. ISSN   0033-3123. PMID   24318769. S2CID   255164220.
  16. ↑ Batchelder, William H. (2009), "Cultural Consensus Theory: Aggregating Expert Judgments about Ties in a Social Network", Social Computing and Behavioral Modeling, Boston, MA: Springer US, pp. 1–9.
  17. ↑ Batchelder, William H.; Kumbasar, Ece; Boyd, John P. (1997). "Consensus analysis of three‐way social network data". The Journal of Mathematical Sociology. 22 (1): 29–58. doi:10.1080/0022250X.1997.9990193. ISSN   0022-250X.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.