Common source bias

Last updated

Common source bias is a type of sampling bias, occurring when both dependent and independent variables are collected from the same group of people. This bias can occur in various forms of research, such as surveys, experiments, and observational studies. Common source bias is a significant concern for any study as it can lead to unreliable results. [1] It is most prevalent in the field of public administration research, where performance measures subject to common source bias can produce false positives when organisational performance is evaluated with explanatory and perceptual measures from the same source. [2]

Contents

Occurrence

Common source bias can be categorised into two types: common method bias, also known as common method variance, and common source bias. Common method bias occurs when the same method or instrument is used to collect data from multiple sources, which can lead to an over-representation of certain factors. Common source bias occurs when the information or data collected is influenced by a single source, such as a single individual, group, or organisation.

One of the major causes of common source bias is the influence of the source on the data collected. For example, if a survey is conducted by a single individual, their own beliefs, biases, and perspectives can influence the responses of the participants.

Common source bias is also present in participant selection. If participants are selected based on their association with the source, then their responses may be biased towards the source’s perspective. If participants are selected based on their willingness to participate, then their responses may not be representative of the population as a whole.

Remedies

Ex ante remedies

A recently proposed ex ante remedy for common source bias is the supplementation of survey data with administrative and/or archival data, however the majority of relevant studies seem to present the view that of the proposed statistical remedies for the bias, none appear to reliably counter the issue. [3] [4]

Related Research Articles

<span class="mw-page-title-main">Research</span> Systematic study undertaken to increase knowledge

Research is "creative and systematic work undertaken to increase the stock of knowledge". It involves the collection, organization and analysis of evidence to increase understanding of a topic, characterized by a particular attentiveness to controlling sources of bias and error. These activities are characterized by accounting and controlling for biases. A research project may be an expansion of past work in the field. To test the validity of instruments, procedures, or experiments, research may replicate elements of prior projects or the project as a whole.

Statistical bias, in the mathematical field of statistics, is a systematic tendency in which the methods used to gather data and generate statistics present an inaccurate, skewed or biased depiction of reality. Statistical bias exists in numerous stages of the data collection and analysis process, including: the source of the data, the methods used to collect the data, the estimator chosen, and the methods used to analyze the data. Data analysts can take various measures at each stage of the process to reduce the impact of statistical bias in their work. Understanding the source of statistical bias can help to assess whether the observed results are close to actuality. Issues of statistical bias has been argued to be closely linked to issues of statistical validity.

<span class="mw-page-title-main">Focus group</span> Group interviewed to analyse opinions

A focus group is a group interview involving a small number of demographically similar people or participants who have other traits/experiences in common depending on the research objective of the study. Their reactions to specific researcher/evaluator-posed questions are studied. Focus groups are used in market research to understand better people's reactions to products or services or participants' perceptions of shared experiences. The discussions can be guided or open. In market research, focus groups can explore a group's response to a new product or service. As a program evaluation tool, they can elicit lessons learned and recommendations for performance improvement. The idea is for the researcher to understand participants' reactions. If group members are representative of a larger population, those reactions may be expected to reflect the views of that larger population. Thus, focus groups constitute a research or evaluation method that researchers organize to collect qualitative data through interactive and directed discussions.

A performance appraisal, also referred to as a performance review, performance evaluation, (career) development discussion, or employee appraisal, sometimes shortened to "PA", is a periodic and systematic process whereby the job performance of an employee is documented and evaluated. This is done after employees are trained about work and settle into their jobs. Performance appraisals are a part of career development and consist of regular reviews of employee performance within organizations.

Survey methodology is "the study of survey methods". As a field of applied statistics concentrating on human-research surveys, survey methodology studies the sampling of individual units from a population and associated techniques of survey data collection, such as questionnaire construction and methods for improving the number and accuracy of responses to surveys. Survey methodology targets instruments or procedures that ask one or more questions that may or may not be answered.

In a blind or blinded experiment, information which may influence the participants of the experiment is withheld until after the experiment is complete. Good blinding can reduce or eliminate experimental biases that arise from a participants' expectations, observer's effect on the participants, observer bias, confirmation bias, and other sources. A blind can be imposed on any participant of an experiment, including subjects, researchers, technicians, data analysts, and evaluators. In some cases, while blinding would be useful, it is impossible or unethical. For example, it is not possible to blind a patient to their treatment in a physical therapy intervention. A good clinical protocol ensures that blinding is as effective as possible within ethical and practical constraints.

Participant observation is one type of data collection method by practitioner-scholars typically used in qualitative research and ethnography. This type of methodology is employed in many disciplines, particularly anthropology, sociology, communication studies, human geography, and social psychology. Its aim is to gain a close and intimate familiarity with a given group of individuals and their practices through an intensive involvement with people in their cultural environment, usually over an extended period of time.

Observer bias is one of the types of detection bias and is defined as any kind of systematic divergence from accurate facts during observation and the recording of data and information in studies. The definition can be further expanded upon to include the systematic difference between what is observed due to variation in observers, and what the true value is.

Policy analysis or public policy analysis is a technique used in the public administration sub-field of political science to enable civil servants, nonprofit organizations, and others to examine and evaluate the available options to implement the goals of laws and elected officials. People who regularly use policy analysis skills and techniques on the job, particularly those who use it as a major part of their job duties are generally known by the title Policy Analyst. The process is also used in the administration of large organizations with complex policies. It has been defined as the process of "determining which of various policies will achieve a given set of goals in light of the relations between the policies and the goals."

<span class="mw-page-title-main">Response bias</span> Type of bias

Response bias is a general term for a wide range of tendencies for participants to respond inaccurately or falsely to questions. These biases are prevalent in research involving participant self-report, such as structured interviews or surveys. Response biases can have a large impact on the validity of questionnaires or surveys.

In medical research, social science, and biology, a cross-sectional study is a type of observational study that analyzes data from a population, or a representative subset, at a specific point in time—that is, cross-sectional data.

The "ceiling effect" is one type of scale attenuation effect; the other scale attenuation effect is the "floor effect". The ceiling effect is observed when an independent variable no longer has an effect on a dependent variable, or the level above which variance in an independent variable is no longer measurable. The specific application varies slightly in differentiating between two areas of use for this term: pharmacological or statistical. An example of use in the first area, a ceiling effect in treatment, is pain relief by some kinds of analgesic drugs, which have no further effect on pain above a particular dosage level. An example of use in the second area, a ceiling effect in data-gathering, is a survey that groups all respondents into income categories, not distinguishing incomes of respondents above the highest level measured in the survey instrument. The maximum income level able to be reported creates a "ceiling" that results in measurement inaccuracy, as the dependent variable range is not inclusive of the true values above that point. The ceiling effect can occur any time a measure involves a set range in which a normal distribution predicts multiple scores at or above the maximum value for the dependent variable.

In social science research, social-desirability bias is a type of response bias that is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting "good behavior" or under-reporting "bad", or undesirable behavior. The tendency poses a serious problem with conducting research with self-reports. This bias interferes with the interpretation of average tendencies as well as individual differences.

Impact evaluation assesses the changes that can be attributed to a particular intervention, such as a project, program or policy, both the intended ones, as well as ideally the unintended ones. In contrast to outcome monitoring, which examines whether targets have been achieved, impact evaluation is structured to answer the question: how would outcomes such as participants' well-being have changed if the intervention had not been undertaken? This involves counterfactual analysis, that is, "a comparison between what actually happened and what would have happened in the absence of the intervention." Impact evaluations seek to answer cause-and-effect questions. In other words, they look for the changes in outcome that are directly attributable to a program.

Participation bias or non-response bias is a phenomenon in which the results of elections, studies, polls, etc. become non-representative because the participants disproportionately possess certain traits which affect the outcome. These traits mean the sample is systematically different from the target population, potentially resulting in biased estimates.

Quantitative methods provide the primary research methods for studying the distribution and causes of crime. Quantitative methods provide numerous ways to obtain data that are useful to many aspects of society. The use of quantitative methods such as survey research, field research, and evaluation research as well as others. The data can, and is often, used by criminologists and other social scientists in making causal statements about variables being researched.

Group decision-making is a situation faced when individuals collectively make a choice from the alternatives before them. The decision is then no longer attributable to any single individual who is a member of the group. This is because all the individuals and social group processes such as social influence contribute to the outcome. The decisions made by groups are often different from those made by individuals. In workplace settings, collaborative decision-making is one of the most successful models to generate buy-in from other stakeholders, build consensus, and encourage creativity. According to the idea of synergy, decisions made collectively also tend to be more effective than decisions made by a single individual. In this vein, certain collaborative arrangements have the potential to generate better net performance outcomes than individuals acting on their own. Under normal everyday conditions, collaborative or group decision-making would often be preferred and would generate more benefits than individual decision-making when there is the time for proper deliberation, discussion, and dialogue. This can be achieved through the use of committee, teams, groups, partnerships, or other collaborative social processes.

A glossary of terms used in clinical research.

With the application of probability sampling in the 1930s, surveys became a standard tool for empirical research in social sciences, marketing, and official statistics. The methods involved in survey data collection are any of a number of ways in which data can be collected for a statistical survey. These are methods that are used to collect information from a sample of individuals in a systematic way. First there was the change from traditional paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). Now, face-to-face surveys (CAPI), telephone surveys (CATI), and mail surveys are increasingly replaced by web surveys. In addition, remote interviewers could possibly keep the respondent engaged while reducing cost as compared to in-person interviewers.

References

  1. Favero, N.; Bullock, J. B. (2015). "How (Not) to Solve the Problem: An Evaluation of Scholarly Responses to Common Source Bias". Journal of Public Administration Research and Theory. pp. 285–308. doi:10.1093/jopart/muu020 . Retrieved 2023-02-07.
  2. Andersen, Lotte Bøgh; Heinesen, Eskil; Pedersen, Lene Holm (2016). "Individual Performance: From Common Source Bias to Institutionalized Assessment". Journal of Public Administration Research and Theory. 26 (1): 63–78. ISSN   1053-1858. JSTOR   44165112.
  3. Favero, Nathan; B. Bullock, Justin. "How (Not) to Solve the Problem: An Evaluation of Scholarly Responses to Common Source Bias".{{cite journal}}: Cite journal requires |journal= (help)
  4. Kim, Mirae; Daniel, Jamie Levine (2020-01-02). "Common Source Bias, Key Informants, and Survey-Administrative Linked Data for Nonprofit Management Research". Public Performance & Management Review. 43 (1): 232–256. doi:10.1080/15309576.2019.1657915. ISSN   1530-9576. S2CID   203468837.