Acquiescence bias

Last updated

Acquiescence bias, also known as agreement bias, [1] is a category of response bias common to survey research [2] in which respondents have a tendency to select a positive response option [1] [3] or indicate a positive connotation disproportionately more frequently. [2] [4] [5] Respondents do so without considering the content [3] of the question or their 'true' preference. [1] Acquiescence is sometimes referred to as "yea-saying" and is the tendency of a respondent to agree with a statement when in doubt. Questions affected by acquiescence bias take the following format: a stimulus in the form of a statement is presented, followed by 'agree/disagree,' 'yes/no' or 'true/false' response options. [2] [6] For example, a respondent might be presented with the statement "gardening makes me feel happy," and would then be expected to select either 'agree' or 'disagree.' Such question formats are favoured by both survey designers and respondents because they are straightforward to produce and respond to. [3] [6] The bias is particularly prevalent in the case of surveys or questionnaires that employ truisms as the stimuli, such as: "It is better to give than to receive" or "Never a lender nor a borrower be". Acquiescence bias can introduce systematic errors that affect the validity of research by confounding attitudes and behaviours with the general tendency to agree, which can result in misguided inference. [2] Research suggests that the proportion of respondents who carry out this behaviour is between 10% and 20%. [2]

Contents

Causes

Agreeableness

A prominent psychological explanation attributes acquiescence bias to social norms that encourage agreeable behaviour. [3] [6] Evidence indicates that respondents approach surveys as though they are common conversations. [2] A consequence of this is that the conventions that govern conversations influence the interpretation of survey questions and responses to them. Accordingly, pressure to conform to such norms and conventions prompts people to agree with stimulus statements. Based on research into the "Big Five" personality traits, individuals are predisposed to agreeable behaviour to differing degrees. [3]

Perceived authority of the interviewer

An explanation favoured by sociologists is that acquiescence bias is a product of the combination of the inclination to yield to the opinions of high authority individuals, and respondents' perception of the researcher/ interviewer as having higher authority. [6] [2] According to this explanation, when selecting answers, a tendency to agree with statements made by the interviewer arises from the respondent's intention of being polite or respectful. [3] In support of this, there is evidence that indicates that respondents of lower social status acquiesce more frequently than respondents of higher social status. [3] However, several studies have failed to replicate this finding. [3]

Satisficing

Acquiescence bias is proposed to be a product of 'satisficing' behaviour. [2] 'Satisficing' sees respondents select responses that are satisfactory or good enough, rather than engage in 'optimizing,' which produces best possible selection. This is done to conserve cognitive energy. [3]

Four-stage model of the response process

Roger Tourangeau, Lance J. Rips, Kenneth Rasinski have developed a cognitive model that proposes four stages to the process of survey response selection. Each stage entails several specific cognitive processes. For example, the Comprehension stage entails paying attention to a question or instruction set. They propose that some cognitive processes are required in order to select an answer whereas others serve as optional aids. A respondent's decision of which processes to employ from the model is determined by a number of influences, notably speed and precision. [7]

The four stages:

  • Comprehension – understanding the question and the information required. [7]
  • Retrieval (for factual questions) – remembering or calling to mind the appropriate information. [7]
  • Judgement (for factual questions) – the processing of recalled information to form judgements. [7]
  • Selection – choosing and communicating an answer. Two sets of processes have been proposed: translating a judgement into the scales provided, and revising the response based on factors such as 'consistency'. [7]

Survey response effects can arise at any stage of the response process. [7] [1]

Strong and weak satisficing

Two forms of 'satisficing' have been proposed: [3]

  • Weak satisficing: Respondents still execute all four stages of the 'Four stage model of the response process.' However, the stages are carried out less rigorously, which results in the output being satisfactory rather than highly accurate. An example of weak satisficing is a respondent who doesn't search their memory as deeply as possible. This would lead to acquiescence if the respondent only scanned their memory for information supporting the positive response option. [3]
  • Strong satisficing: Respondents implement a surface-level approach to answering the question by omitting the 'Retrieval' and 'Judgement' stages and only engaging in 'Comprehension' and 'Selection.' The respondent does not access any internal cognitive resources concerning the construct of interest for the question. A mechanism of selecting an appropriate answer may instead involve interpreting external cues such as question-wording. Following the social convention of agreeing is one alternative mechanism that would lead to Acquiescence bias. [3]

Solutions

Balanced scales

This approach involves the modification of scales to include an equal ratio of positively and negatively framed items. [1] In other words, a particular construct is assessed using conflicting stimulus statements. [2] For example, in trying to assess depression it would be a good idea to also include items assessing happiness and contentedness, etc. (reversed-keyed items), in addition to the usual depressive content. [8] The rationale is that such reverse-coded items force respondents to engage consciously and deliberately with survey questions, rather than automatically. [9] While this technique has been shown to minimise a construct's relationship with acquiescence bias, [2] it is imperfect in that respondents continue to provide responses biased by acquiescence. [1]

Douglas N. Jackson demonstrated acquiescence responding on the California F-scale (a measure of authoritarianism), which contains such truisms. He created a reverse-keyed version of the California F-scale where all the items were the opposite in meaning (see the two previous examples for a pair of such contradictory statements). He administered both the original and reverse-keyed versions of the California F-scale to the same group of respondents. One would expect that the correlation between these two scales to be negative, but there was a high, positive correlation. Jackson interpreted this as evidence of acquiescence responding. Respondents were merely being agreeable to the statements, regardless of the content.

In such cases, the only effect of the technique is in moving individuals from the positive pole of the scale to the midpoint which, it is argued, achieves nothing. [6] [2]

Statistical correctives

The use of two techniques has been proposed to separate out acquiescence bias from constructs of interest: Factor analysis, and Ipsatization. [2]

Jackson and Messick, using factor analysis, also demonstrated that the two main factors explaining the majority of response variation on the Minnesota Multiphasic Personality Inventory (MMPI) were for social desirability and acquiescence responding [10] (this would also hold true for the revised MMPI-2).

Question choice

Some researchers have denounced the use of 'agree/disagree' scales (including 'yes/no' and 'true/false' variations) and call for the use of question types that aren't as susceptible to acquiescence bias. [2]

One alternative is 'item-specific' (IS) questions. [2] Instead of providing a statement and 'agree/disagree' response option, the statement is transformed into a direct question and response options present a range that captures the extremities of an attitude or behaviour. For example, the statement 'I like the colour blue,' is transformed into 'do you like the colour blue?' with response options ranging from 'not at all' to 'very much.' Proponents of this solution reason that 'agree/disagree' scales demand excess cognitive resources. In many cases, a response to the direct question is a pre-requisite to providing an 'agree/disagree' choice; a person first responds to 'how often do I feel like starting a new hobby?' when presented with the statement 'I regularly feel like starting a new hobby.' There is, therefore, an additional process of translation onto the 'agree/disagree' scale. [6] [2]

See also

Related Research Articles

Satisficing is a decision-making strategy or cognitive heuristic that entails searching through the available alternatives until an acceptability threshold is met. The term satisficing, a portmanteau of satisfy and suffice, was introduced by Herbert A. Simon in 1956, although the concept was first posited in his 1947 book Administrative Behavior. Simon used satisficing to explain the behavior of decision makers under circumstances in which an optimal solution cannot be determined. He maintained that many natural problems are characterized by computational intractability or a lack of information, both of which preclude the use of mathematical optimization procedures. He observed in his Nobel Prize in Economics speech that "decision makers can satisfice either by finding optimum solutions for a simplified world, or by finding satisfactory solutions for a more realistic world. Neither approach, in general, dominates the other, and both have continued to co-exist in the world of management science".

Questionnaire construction refers to the design of a questionnaire to gather statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires can provide valuable data about any given subject.

<span class="mw-page-title-main">Likert scale</span> Psychometric measurement scale

A Likert scale is a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale, although there are other types of rating scales.

<span class="mw-page-title-main">Personality test</span> Method of assessing human personality constructs

A personality test is a method of assessing human personality constructs. Most personality assessment instruments are in fact introspective self-report questionnaire measures or reports from life records (L-data) such as rating scales. Attempts to construct actual performance tests of personality have been very limited even though Raymond Cattell with his colleague Frank Warburton compiled a list of over 2000 separate objective tests that could be used in constructing objective personality tests. One exception however, was the Objective-Analytic Test Battery, a performance test designed to quantitatively measure 10 factor-analytically discerned personality trait dimensions. A major problem with both L-data and Q-data methods is that because of item transparency, rating scales and self-report questionnaires are highly susceptible to motivational and response distortion ranging all the way from lack of adequate self-insight to downright dissimulation depending on the reason/motivation for the assessment being undertaken.

The anchoring effect is a cognitive bias whereby an individual's decisions are influenced by a particular reference point or 'anchor'. Both numeric and non-numeric anchoring have been reported in research. In numeric anchoring, once the value of the anchor is set, subsequent arguments, estimates, etc. made by an individual may change from what they would have otherwise been without the anchor. For example, an individual may be more likely to purchase a car if it is placed alongside a more expensive model. Prices discussed in negotiations that are lower than the anchor may seem reasonable, perhaps even cheap to the buyer, even if said prices are still relatively higher than the actual market value of the car. Another example may be when estimating the orbit of Mars, one might start with the Earth's orbit and then adjust upward until they reach a value that seems reasonable.

<span class="mw-page-title-main">Questionnaire</span> Series of questions for gathering information

A questionnaire is a research instrument that consists of a set of questions for the purpose of gathering information from respondents through survey or statistical study. A research questionnaire is typically a mix of close-ended questions and open-ended questions. Open-ended, long-term questions offer the respondent the ability to elaborate on their thoughts. The Research questionnaire was developed by the Statistical Society of London in 1838.

<span class="mw-page-title-main">Response bias</span> Type of bias

Response bias is a general term for a wide range of tendencies for participants to respond inaccurately or falsely to questions. These biases are prevalent in research involving participant self-report, such as structured interviews or surveys. Response biases can have a large impact on the validity of questionnaires or surveys.

<span class="mw-page-title-main">Social dominance orientation</span> Personality trait favoring social hierarchies

Social dominance orientation (SDO) is a personality trait measuring an individual's support for social hierarchy and the extent to which they desire their in-group be superior to out-groups. SDO is conceptualized under social dominance theory as a measure of individual differences in levels of group-based discrimination; that is, it is a measure of an individual's preference for hierarchy within any social system and the domination over lower-status groups. It is a predisposition toward anti-egalitarianism within and between groups.

<span class="mw-page-title-main">Right-wing authoritarian personality</span> Submissive personality type

In psychology, the right-wing authoritarian (RWA) is a personality type that describes somebody who is highly submissive to their authority figures, acts aggressively in the name of said authorities, and is conformist in thought and behavior. The prevalence of this personality type in a population varies from culture to culture, as a person's upbringing and education play a strong role in determining whether somebody develops this sort of personality.

In social science research, social-desirability bias is a type of response bias that is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting "good behavior" or under-reporting "bad", or undesirable behavior. The tendency poses a serious problem with conducting research with self-reports. This bias interferes with the interpretation of average tendencies as well as individual differences.

A self-report study is a type of survey, questionnaire, or poll in which respondents read the question and select a response by themselves without any outside interference. A self-report is any method which involves asking a participant about their feelings, attitudes, beliefs and so on. Examples of self-reports are questionnaires and interviews; self-reports are often used as a way of gaining participants' responses in observational studies and experiments.

<span class="mw-page-title-main">Self-report inventory</span> Type of psychological test

A self-report inventory is a type of psychological test in which a person fills out a survey or questionnaire with or without the help of an investigator. Self-report inventories often ask direct questions about personal interests, values, symptoms, behaviors, and traits or personality types. Inventories are different from tests in that there is no objectively correct answer; responses are based on opinions and subjective perceptions. Most self-report inventories are brief and can be taken or administered within five to 15 minutes, although some, such as the Minnesota Multiphasic Personality Inventory (MMPI), can take several hours to fully complete. They are popular because they can be inexpensive to give and to score, and their scores can often show good reliability.

<span class="mw-page-title-main">Situational judgement test</span>

A situational judgement test (SJT), or situational stress test (SStT) or inventory (SSI) is a type of psychological test which presents the test-taker with realistic, hypothetical scenarios and ask them to identify the most appropriate response or to rank the responses in the order they feel is most effective. SJTs can be presented to test-takers through a variety of modalities, such as booklets, films, or audio recordings. SJTs represent a distinct psychometric approach from the common knowledge-based multiple choice item. They are often used in industrial-organizational psychology applications such as personnel selection. Situational judgement tests tend to determine behavioral tendencies, assessing how an individual will behave in a certain situation, and knowledge instruction, which evaluates the effectiveness of possible responses. Situational judgement tests could also reinforce the status quo with an organization.

Psychological evaluation is a method to assess an individual's behavior, personality, cognitive abilities, and several other domains. A common reason for a psychological evaluation is to identify psychological factors that may be inhibiting a person's ability to think, behave, or regulate emotion functionally or constructively. It is the mental equivalent of physical examination. Other psychological evaluations seek to better understand the individual's unique characteristics or personality to predict things like workplace performance or customer relationship management.

Heuristics is the process by which humans use mental short cuts to arrive at decisions. Heuristics are simple strategies that humans, animals, organizations, and even machines use to quickly form judgments, make decisions, and find solutions to complex problems. Often this involves focusing on the most relevant aspects of a problem or situation to formulate a solution. While heuristic processes are used to find the answers and solutions that are most likely to work or be correct, they are not always right or the most accurate. Judgments and decisions based on heuristics are simply good enough to satisfy a pressing need in situations of uncertainty, where information is incomplete. In that sense they can differ from answers given by logic and probability.

Mode effect is a broad term referring to a phenomenon where a particular survey administration mode causes different data to be collected. For example, when asking a question using two different modes, responses to one mode may be significantly and substantially different from responses given in the other mode. Mode effects are a methodological artifact, limiting the ability to compare results from different modes of collection.

With the application of probability sampling in the 1930s, surveys became a standard tool for empirical research in social sciences, marketing, and official statistics. The methods involved in survey data collection are any of a number of ways in which data can be collected for a statistical survey. These are methods that are used to collect information from a sample of individuals in a systematic way. First there was the change from traditional paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). Now, face-to-face surveys (CAPI), telephone surveys (CATI), and mail surveys are increasingly replaced by web surveys.

Humor styles are a subject of research in the field of personality psychology that focuses on the ways in which individuals differ in their use of humor. People of all ages and cultures respond to humor, but their use of it can vary greatly. There are multiple factors, such as culture, age, and political orientation, that play a role in determining what people find humorous. Although humor styles can be somewhat variable depending on social context, they tend to be a relatively stable personality characteristic among individuals. Humor can play an instrumental role in the formation of social bonds, enabling people to relate to peers or to attract a mate, and can help to release tension during periods of stress. There is a lack of current, reliable research that explores the impact of humor usages on others because it is difficult to distinguish a healthy humor usage from one that is unhealthy. Justifications for harmful versus benign humor styles are subjective and lead to varying definitions of either usage.

The Levenson Self-Report Psychopathy scale (LSRP) is a 26-item, 4-point Likert scale, self-report inventory to measure primary and secondary psychopathy in non-institutionalised populations. It was developed in 1995 by Michael R. Levenson, Kent A. Kiehl and Cory M. Fitzpatrick. The scale was created for the purpose of conducting a psychological study examining antisocial disposition among a sample of 487 undergraduate students attending psychology classes at the University of California, Davis.

Best–worst scaling (BWS) techniques involve choice modelling and were invented by Jordan Louviere in 1987 while on the faculty at the University of Alberta. In general with BWS, survey respondents are shown a subset of items from a master list and are asked to indicate the best and worst items. The task is repeated a number of times, varying the particular subset of items in a systematic way, typically according to a statistical design. Analysis is typically conducted, as with DCEs more generally, assuming that respondents makes choices according to a random utility model (RUM). RUMs assume that an estimate of how much a respondent prefers item A over item B is provided by how often item A is chosen over item B in repeated choices. Thus, choice frequencies estimate the utilities on the relevant latent scale. BWS essentially aims to provide more choice information at the lower end of this scale without having to ask additional questions that are specific to lower ranked items.

References

  1. 1 2 3 4 5 6 Baron-Epel, Orna; Kaplan, Giora; Weinstein, Ruth; Green, Manfred S. (October 2010). "Extreme and acquiescence bias in a bi-ethnic population". European Journal of Public Health. 20 (5): 543–548. doi: 10.1093/eurpub/ckq052 . ISSN   1464-360X. PMID   20439322.
  2. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Kuru, Ozan; Pasek, Josh (2016-04-01). "Improving social media measurement in surveys: Avoiding acquiescence bias in Facebook research". Computers in Human Behavior. 57: 82–92. doi:10.1016/j.chb.2015.12.008. ISSN   0747-5632.
  3. 1 2 3 4 5 6 7 8 9 10 11 12 Krosnick, Jon A. (February 1999). "Survey Research". Annual Review of Psychology . 50 (1): 537–567. doi:10.1146/annurev.psych.50.1.537. ISSN   0066-4308. PMID   15012463.
  4. Watson, Dorothy (1992). "Correcting for Acquiescent Response Bias in the Absence of a Balanced Scale: An Application to Class Consciousness". Sociological Methods & Research. 21 (1): 52–88. doi:10.1177/0049124192021001003. S2CID   122977362.
  5. Moss, Simon (2008). "Acquiescence bias". Archived from the original on 2011-02-18. Retrieved 2010-07-19.
  6. 1 2 3 4 5 6 Pasek, Josh; Krosnick, Jon A. (2010-02-25). Leighley, Jan E. (ed.). "Optimizing Survey Questionnaire Design in Political Science". The Oxford Handbook of American Elections and Political Behavior. doi:10.1093/oxfordhb/9780199235476.001.0001. ISBN   9780199235476 . Retrieved 2020-04-14.
  7. 1 2 3 4 5 6 Tourangeau, Roger; Rips, Lance J.; Rasinski, Kenneth (March 2000). The Psychology of Survey Response. doi:10.1017/CBO9780511819322. ISBN   9780521572460.
  8. Erikson, Robert S.; Tedin, Kent L. (2015). American Public Opinion: Its Origins, Content and Impact. Routledge. ISBN   9781317350385.
  9. Podsakoff, Philip M.; MacKenzie, Scott B.; Lee, Jeong-Yeon; Podsakoff, Nathan P. (Oct 2003). "Common method biases in behavioral research: a critical review of the literature and recommended remedies". The Journal of Applied Psychology. 88 (5): 879–903. doi:10.1037/0021-9010.88.5.879. hdl: 2027.42/147112 . ISSN   0021-9010. PMID   14516251. S2CID   5281538.
  10. Messick, Samuel; Jackson, Douglas N. (1961). "Acquiescence and the factorial interpretation of the MMPI". Psychological Bulletin. 58 (4): 299–304. doi:10.1037/h0043979. PMID   13769793.