Member check

Last updated January 07, 2024

In qualitative research, a member check, also known as informant feedback or respondent validation, is a technique used by researchers to help improve the accuracy, credibility, validity, and transferability (also known as applicability, internal validity,^[1] or fittingness) of a study.^[2] There are many subcategories of members checks, including: narrative accuracy checks, interpretive validity, descriptive validity, theoretical validity, and evaluative validity. In many member checks, the interpretation and report (or a portion of it) is given to members of the sample (informants) in order to check the authenticity of the work. Their comments serve as a check on the viability of the interpretation.^[2]

Member checking can be done during the interview process, at the conclusion of the study, or both to increase the credibility and validity (statistics) of a qualitative study. The interviewer should strive to build rapport with the interviewee in order to obtain honest and open responses. During an interview, the researcher will restate or summarize information and then question the participant to determine accuracy. Member checks completed after a study are completed by sharing all of the findings with the participants involved. This allows participants to critically analyze the findings and comment on them. The participants either affirm that the summaries reflect their views, feelings, and experiences, or that they do not reflect these experiences. If the participants affirm the accuracy and completeness, then the study is said to have credibility. These member checks are not without fault, but serve to decrease the incidence of incorrect data and the incorrect interpretation of data. The overall goal of this process is to provide findings that are authentic, original and reliable.

Positive aspects of member-checking

Member checking provides an opportunity to understand and determine what the author intended to do through their actions. It gives them the ability to correct errors and challenge what are perceived as wrong interpretations. Member checking provides the opportunity to volunteer additional information which may be stimulated by the playing back process. Gets respondent on the record with their reports. Provides people that respond the opportunity to assess adequacy of data and preliminary results as well as to confirm particular aspects of the data. The ability to summarize the preliminary findings. The major problem with member checking however is that it relies on the assumption that there is a fixed truth of reality that can be accounted for by a researcher and can be confirmed by a respondent, which may not be true. In fact, member checks are not consistent with the philosophy of non positivistic qualitative research methods.^[3] See positivism for more information on philosophy of approach.

Interviewing and member checking

There are two interview styles to promote discussion between interviewees and the interviewers during member checks. The first, Confrontation Style is only effective when there is a trusted, warm, and open relationship between the interviewer and the interviewee. This style promotes discussion empowering the interviewee to have confidence to "fight back" when his/her opinion is questioned. The second, Dialogue and Power, focuses on the inherent power struggle between the interviewer and the interviewee. Interviewers must be aware of this imbalance of power and promote discussion from the interviewee by valuing opinions and reinforcing equal collaboration between the two.^[4]

In an informal sense, member checks are carried out verbally throughout the conduct of fieldwork. The researcher constantly checks his or her understanding of the phenomenon by utilizing techniques such as paraphrasing and summarization for clarification. It is best to ask permission to visit the participant again at the time of the unstructured interview. Interviews are used as a way for the respondent to express their emotions and thoughts about their experiences and allow the interviewer to have a better understanding of a situation.^[5]

When the member check procedure is used in a sample of people who were not the original participants in the study, the procedure can be used to assess transferability. If the people who were not the original participants do not agree with the information, then the findings can be said to be biased or not transferable since transferability is the process of being able to apply the findings from one study to others in similar situations. A question to ask is whether the findings "ring true" in others' experiences.^[6]

It is critical to use member checking in qualitative research studies because these types of studies often involve interpretation.^[7] Thus, without allowing participants to validate the accuracy of their findings, one-sidedness will become a major concern. Since the most important issue in evaluating the rigor of qualitative research is trustworthiness, using the strategy of member checks, (along with other techniques such as independent audit, prolonged engagement, triangulation, peer debriefing), is critical to minimizing distortion.^[8] Prolonged engagement aids in the ability to detect and account for distortions that might be in the data. It also helps to become oriented to the situation so content is appreciated and the researcher builds trust.^[9] In general, member checks are a useful function in research, especially when there are questions about the adequacy of understanding based on limited time of exposure. Thus, a member check can serve to counterbalance concerns about whether engagement was sufficiently prolonged

Advantages and disadvantages of member checks

Advantages

The greatest advantage in a member check is that the researcher can verify the entirety and completeness of the findings, which is a measurable tool of the accuracy of the findings.

Other Advantages:^[10]

Provides an opportunity to understand and assess what the participant intended to do through his or her actions
Gives participants opportunity to correct errors and challenge what are perceived as wrong interpretations
Provides the opportunity to volunteer additional information which may be stimulated by the playing back process
Gets respondent on the record with his or her reports
Provides an opportunity to summarize preliminary findings
Member checks can be useful in action research projects, where researchers work with participants on a continuing basis to help with change.
It lessens the risk of participants reporting at a later date that the researcher misunderstood their contributions or claiming investigative error.
Enables an assessment of what the participant intended by making exact comments or taking specific actions
Shows high credibility and face validity ^[5]
Prevent personal biases from being included within the quantitative research study.
Prevent false information from being presented as reliable research.

Disadvantages

There are multiple disadvantages related to member checks that may occur throughout the process.

Research findings cannot be easily conveyed to the participant. This is most commonly the case in addiction and criminal research. Additionally, findings may be difficult to understand.
Member check exercises can make extensive demands on the participants' time.
Sometimes the content of transcripts and the research topic can be exploitative and/or distressing.
Research participants may refuse to participate in member check, especially if an extended period of time has elapsed since research study.
Participants may tell the researcher what they believe the researcher wants to hear.
Member checking relies on the assumption that there is a fixed truth of reality that can be accounted for by a researcher and confirmed by a respondent.
Members may relay accounts during an interview that they later regret or see differently. Members may deny such stories and want them removed from the data
Members may not be in the best position to check the data. They may forget what they said or the manner in which a story was told
Different members may have different views of the same data.
It requires skills and experience from the researcher and can be expensive and time-consuming.^[5]
Member checks (respondent validation) are part of a process of error reduction, which also generates further original data, which, in turn, require interpretation, rather than as a straightforward check on validity.^[11]
The very subjectivity of the inquiry leads to difficulties in establishing the reliability and validity of the approaches and information.
Its scope is limited due to the in-depth, comprehensive data gathering approaches required.^[12]

Objections to member checks

Many writers establish internal validity – truthfulness and representation of the reality of the participants – by showing that they have carried out a ‘member check’ as Lincoln and Guba (1985) suggest. However, some researchers disagree with the use of a member check. Many researchers have noted that when the essence of the participants' experiences are similar, their stories represent social reality. Accordingly, phenomenology (science) is defined as a philosophy or method of inquiry based on the premise that reality consists of objects and events as they are perceived or understood in human consciousness and not of anything independent of human consciousness.^[13]

In qualitative research, phenomenological methods are used to learn and construct the meaning of the human experience through intensive dialogue with persons who are living the experience. The researcher's goal is to explain the meaning of the experience to the participant. This is achieved through a dialogic process, which is more than a simple interview.^[14] Therefore, in their opinion, a member check can adversely transform the data through the process of analysis and writing. Phenomenologists believe each individual has his/her own unique perspective, inhabits a social world, and recognizes others’ reality to some extent. They believe that for an account to have validity, its readers will have grasped not only the essence of the phenomenon but also understood something of the human condition they have in common with the participants-–intersubjective understanding.^[15]

Member check considerations

Member checks by research participants are increasingly recommended in qualitative research. There are differences, however, in qualitative research that suggests member checks may need to be approached with caution.^[16] Although member checking has been used for verification of the results, it is not always a true verification strategy. Many methodologists caution against using member check as verification by defining what participants say to be correct because it may actually pose a threat to the validity instead. This is due to results of studies being combined, made neutral, and abstracted from other participants as well as investigators wanting to be more responsive to their participants and restrain some of their results. Ultimately this causes the researcher's study to be invalid.^[17] Many consider member checking to be the best method of establishing credibility, but one of the main drawbacks is the way in which the researcher views the research as being intended to generalize the findings. This may clash with the participant's view that their account is specific and solely their experience. Due to the different views regarding interpretation of data, member checks may be better suited as being identified as a tool for error reduction, rather than a verification protocol.^[18]

Member checks can be used as a technique to evaluate the problems with the study process such as practical, theoretical, representational, and moral flaws to ensure the honesty of the research procedures.^[19] The process of a member check also is important in revealing missing information that should be addressed before concluding the study. This is a step of reevaluation within a study that allows for researchers to implement changes and conduct further interviews in areas where a study is weak. However, the responses received from participants may not always be accurate and should be carefully reviewed by researchers.

It is important for researchers to review responses in order to avoid altering sound data. Participants in a study may have knowledge of incomplete or incorrect information that leads to misinformed responses. Likewise, they may also respond falsely in order to avoid social judgment or societal views on the subject, despite anonymity. Participants’ responses may also stem from myth-based knowledge or delusional thinking (Douglas, 1976). When hypotheses are formed, researchers often have predictions on the outcome; therefore, it is vital that researchers avoid their own biases to data. Avoidance of bias can be aided by having separate researchers review the member check responses rather than by those who conducted the interviews. There are no clear-cut means of clearly avoiding incorrect participant feedback or researcher bias from tainting gathered research. However, if researchers can minimize these factors they can strengthen the external validity of their research.^[20]

Although member checks are considered very beneficial to credibility of the study, there is not a lot of information or understanding on how member checks should be performed. In recent studies, two methods have been identified. The first is to send the transcripts or summaries are sent to the original participants to be confirmed. The second uses a member check group session with each group member being shown a summary of the analysis. Regardless of the method used to peer review or member check research it must be done to ensure quality research and improve upon the research being performed before it is submitted for publication and considered as a reliable study.

Notes

↑ Creswell, John W. (1994). Research Design Qualitative and Quantitative Approaches . CA: USA: Sage. p. 158. ISBN 9780803952553.
1 2 Yanow, Dvora; Schwartz-Shea, Peregrine (2006). Interpretation and Method: Empirical Research Methods and the Interpretive Turn. ISBN 978-0-7656-1463-6.
↑ "RWJF – Qualitative Research Guidelines Project | Member checking | Member Checks". www.qualres.org.
↑ Tanggaard, L. (2008). Objections in research interviewing. International Journal of Qualitative Methods, 7(3), p 15-29.
1 2 3 "Qualitative Interviews in Evaluation". Archived from the original on 2008-11-29. Retrieved 2008-11-25.
↑ LoBiondo-Wood, G & Haber, J. "Nursing Research." Mosby Elsevier (2006)
↑ Byrne, M.M. (2001).Evaluating the findings of qualitative research. AORN Journal. Retrieved November 23, 2008 from http://findarticles.com/p/articles
↑ Rubin, A., Babbie, E.R. (2008).Research methods for social work. Belmont, CA:Thomson Brooks/Cole.
↑ http://www.qualres.org/Home-Prol%5B%5D 3690.html.
↑ Cohen D, Crabtree B. "Qualitative Research Guidelines Project." July 2006. http://www.qualres.org/HomeMemb-3696.html
↑ Pope, C., & Mays, N. (2006). Qualitative Research in Health Care. Malden, MA: Wiley-Blackwell.
↑ Key, James Research Design in Occupational Education Module r14 Qualitative Research http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage21.htm
↑ "phenomenologists" – via The Free Dictionary.
↑ Evidence-Based Nursing#Methodology
↑ Holloway, I. "Qualitative Research in Healthcare." McGraw-Hill (2005)
↑ Reference World.com
↑ Morse, J.M., Barrett, M., Mayan, M., Olson, K., and Spiers, J. (2002).Verification Strategies for Establishing Reliability and Validity in Qualitative Research. International Journal of Qualitative Methods 1, (2). Retrieved on November 24, 2008 from https://www.ualberta.ca/~iiqm.
↑ Mays, N., Pope, C. (2000). Assessing quality in qualitative research. British Medical Journal, 320, 50-52.
↑ Sandelowski, Margarete PhD, RN, FAAN. December 1993. "Methods of Inquiry." Advances in Nursing Science 16(2):1-8. Retrieved November 30, 2008. http://www.advancesinnursingscience.com/pt/re/ans/abstract.
↑ Lincoln, Y. S., Guba, E. G., & Guba, E. (1985). Naturalistic inquiry: The paradigm revolution. London: SAGE.

Related Research Articles

Research is "creative and systematic work undertaken to increase the stock of knowledge". It involves the collection, organization and analysis of evidence to increase understanding of a topic, characterized by a particular attentiveness to controlling sources of bias and error. These activities are characterized by accounting and controlling for biases. A research project may be an expansion of past work in the field. To test the validity of instruments, procedures, or experiments, research may replicate elements of prior projects or the project as a whole.

In common usage, evaluation is a systematic determination and assessment of a subject's merit, worth and significance, using criteria governed by a set of standards. It can assist an organization, program, design, project or any other intervention or initiative to assess any aim, realisable concept/proposal, or any alternative, to help in decision-making; or to generate the degree of achievement or value in regard to the aim and objectives and results of any such action that has been completed.

A focus group is a group interview involving a small number of demographically similar participants. Their reactions to specific researcher/evaluator-posed questions are studied. Focus groups are used in market research to understand better people's reactions to products or services or participants' perceptions of shared experiences. The discussions can be guided or open. In market research, focus groups can explore a group's response to a new product or service. As a program evaluation tool, they can elicit lessons learned and recommendations for performance improvement. The idea is for the researcher to understand participants' reactions. If group members are representative of a larger population, those reactions may be expected to reflect the views of that larger population. Thus, focus groups constitute a research or evaluation method that researchers organize to collect qualitative data through interactive and directed discussions.

Cross-cultural communication is a field of study investigating how people from differing cultural backgrounds communicate, in similar and different ways among themselves, and how they endeavor to communicate across cultures. Intercultural communication is a related field of study.

An interview is a structured conversation where one participant asks questions, and the other provides answers. In common parlance, the word "interview" refers to a one-on-one conversation between an interviewer and an interviewee. The interviewer asks questions to which the interviewee responds, usually providing information. That information may be used or provided to other audiences immediately or later. This feature is common to many types of interviews – a job interview or interview with a witness to an event may have no other audience present at the time, but the answers will be later provided to others in the employment or investigative process. An interview may also transfer information in both directions.

Survey methodology is "the study of survey methods". As a field of applied statistics concentrating on human-research surveys, survey methodology studies the sampling of individual units from a population and associated techniques of survey data collection, such as questionnaire construction and methods for improving the number and accuracy of responses to surveys. Survey methodology targets instruments or procedures that ask one or more questions that may or may not be answered.

Qualitative marketing research involves a natural or observational examination of the philosophies that govern consumer behavior. The direction and framework of the research is often revised as new information is gained, allowing the researcher to evaluate issues and subjects in an in-depth manner. The quality of the research produced is heavily dependent on the skills of the researcher and is influenced by researcher bias.

Quantitative marketing research is the application of quantitative research techniques to the field of marketing research. It has roots in both the positivist view of the world, and the modern marketing viewpoint that marketing is an interactive process in which both the buyer and seller reach a satisfying agreement on the "four Ps" of marketing: Product, Price, Place (location) and Promotion.

Qualitative psychological research is psychological research that employs qualitative methods.

Participant observation is one type of data collection method by practitioner-scholars typically used in qualitative research and ethnography. This type of methodology is employed in many disciplines, particularly anthropology, sociology, communication studies, human geography, and social psychology. Its aim is to gain a close and intimate familiarity with a given group of individuals and their practices through an intensive involvement with people in their cultural environment, usually over an extended period of time.

Qualitative research is a type of research that aims to gather and analyse non-numerical (descriptive) data in order to gain an understanding of individuals' social reality, including understanding their attitudes, beliefs, and motivation. This type of research typically involves in-depth interviews, focus groups, or observations in order to collect data that is rich in detail and context. Qualitative research is often used to explore complex phenomena or to gain insight into people's experiences and perspectives on a particular topic. It is particularly useful when researchers want to understand the meaning that people attach to their experiences or when they want to uncover the underlying reasons for people's behavior. Qualitative methods include ethnography, grounded theory, discourse analysis, and interpretative phenomenological analysis. Qualitative research methods have been used in sociology, anthropology, political science, psychology, communication studies, social work, folklore, educational research, information science and software engineering research.

Autoethnography is a form of ethnographic research in which a researcher connects personal experiences to wider cultural, political, and social meanings and understandings. It is considered a form of qualitative and/or arts-based research.

Narrative inquiry or narrative analysis emerged as a discipline from within the broader field of qualitative research in the early 20th century, as evidence exists that this method was used in psychology and sociology. Narrative inquiry uses field texts, such as stories, autobiography, journals, field notes, letters, conversations, interviews, family stories, photos, and life experience, as the units of analysis to research and understand the way people create meaning in their lives as narratives.

A structured interview is a quantitative research method commonly employed in survey research. The aim of this approach is to ensure that each interview is presented with exactly the same questions in the same order. This ensures that answers can be reliably aggregated and that comparisons can be made with confidence between sample sub groups or between different survey periods.

Cognitive pretesting, or cognitive interviewing, is a field research method where data is collected on how the subject answers interview questions. It is the evaluation of a test or questionnaire before it's administered. It allows survey researchers to collect feedback regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey to the full sample of people.

Interpretative phenomenological analysis (IPA) is a qualitative form of psychology research. IPA has an idiographic focus, which means that instead of producing generalization findings, it aims to offer insights into how a given person, in a given context, makes sense of a given situation. Usually, these situations are of personal significance; examples might include a major life event, or the development of an important relationship. IPA has its theoretical origins in phenomenology and hermeneutics, and many of its key ideas are inspired by the work of Edmund Husserl, Martin Heidegger, and Maurice Merleau-Ponty. IPA's tendency to combine psychological, interpretative, and idiographic elements is what distinguishes it from other approaches to qualitative, phenomenological psychology.

An unstructured interview or non-directive interview is an interview in which questions are not prearranged. These non-directive interviews are considered to be the opposite of a structured interview which offers a set amount of standardized questions. The form of the unstructured interview varies widely, with some questions being prepared in advance in relation to a topic that the researcher or interviewer wishes to cover. They tend to be more informal and free flowing than a structured interview, much like an everyday conversation. Probing is seen to be the part of the research process that differentiates the in-depth, unstructured interview from an everyday conversation. This nature of conversation allows for spontaneity and for questions to develop during the course of the interview, which are based on the interviewees' responses. The chief feature of the unstructured interview is the idea of probe questions that are designed to be as open as possible. It is a qualitative research method and accordingly prioritizes validity and the depth of the interviewees' answers. One of the potential drawbacks is the loss of reliability, thereby making it more difficult to draw patterns among interviewees' responses in comparison to structured interviews. Unstructured interviews are used in a variety of fields and circumstances, ranging from research in social sciences, such as sociology, to college and job interviews. Fontana and Frey have identified three types of in depth, ethnographic, unstructured interviews - oral history, creative interviews, and post-modern interviews.

Evidence-based nursing (EBN) is an approach to making quality decisions and providing nursing care based upon personal clinical expertise in combination with the most current, relevant research available on the topic. This approach is using evidence-based practice (EBP) as a foundation. EBN implements the most up to date methods of providing care, which have been proven through appraisal of high quality studies and statistically significant research findings. The goal of EBN is to improve the health and safety of patients while also providing care in a cost-effective manner to improve the outcomes for both the patient and the healthcare system. EBN is a process founded on the collection, interpretation, appraisal, and integration of valid, clinically significant, and applicable research. The evidence used to change practice or make a clinical decision can be separated into seven levels of evidence that differ in type of study and level of quality. To properly implement EBN, the knowledge of the nurse, the patient's preferences, and multiple studies of evidence must all be collaborated and utilized in order to produce an appropriate solution to the task at hand. These skills are taught in modern nursing education and also as a part of professional training.

Thematic analysis is one of the most common forms of analysis within qualitative research. It emphasizes identifying, analysing and interpreting patterns of meaning within qualitative data. Thematic analysis is often understood as a method or technique in contrast to most other qualitative analytic approaches - such as grounded theory, discourse analysis, narrative analysis and interpretative phenomenological analysis - which can be described as methodologies or theoretically informed frameworks for research. Thematic analysis is best thought of as an umbrella term for a variety of different approaches, rather than a singular method. Different versions of thematic analysis are underpinned by different philosophical and conceptual assumptions and are divergent in terms of procedure. Leading thematic analysis proponents, psychologists Virginia Braun and Victoria Clarke distinguish between three main types of thematic analysis: coding reliability approaches, code book approaches and reflexive approaches. They describe their own widely used approach first outlined in 2006 in the journal Qualitative Research in Psychology as reflexive thematic analysis. Their 2006 paper has over 120,000 Google Scholar citations and according to Google Scholar is the most cited academic paper published in 2006. The popularity of this paper exemplifies the growing interest in thematic analysis as a distinct method.

An interview in qualitative research is a conversation where questions are asked to elicit information. The interviewer is usually a professional or paid researcher, sometimes trained, who poses questions to the interviewee, in an alternating series of usually brief questions and answers. They can be contrasted with focus groups in which an interviewer questions a group of people and observes the resulting conversation between interviewees, or surveys which are more anonymous and limit respondents to a range of predetermined answer choices. In addition, there are special considerations when interviewing children. In phenomenological or ethnographic research, interviews are used to uncover the meanings of central themes in the life world of the subjects from their own point of view.

References

Arminio, J., Jones, S., Torres, V. Negotiating the Complexities of Qualitative Research in Higher Education: Fundamental Elements and Issues.
Barbour, R.S. (2001).Checklists for improving rigour in qualitative research: a case of the tail wagging the dog. BMJ. 322, 1115–1117.
Britten, Nicky. BMJ 1995; 311: 251-253 (22 July).
Bryman, Alan. (Ed.). Member Validation. In Addressing social problems through qualitative research Loughborough: Reference World.
Bygstad, B., Munkvold B. (2007) "The Significance of Member Validation in Qualitative Analysis: Experiences From a Longitudinal Case Study." Proceedings of the 40th Hawaii International Conference on System Sciences.
Byrne, M.M. (2001). Evaluating the findings of qualitative research. AORN Journal.
Cohen, D., Crabtree, B. "Qualitative Research Guidelines Project." July 2006.
Doyle, S. (2007). Member Checking With Older Women: A Framework for Negotiating Meaning. Healthcare for Women International, 28, 888–908.
Flick, U. (2006). An Introduction to Qualitative Research. Sage.
Gordon, C., Prince, M.B., Benkendorf, J.L. (2002). Hamilton HE Journal of Genetic Counseling. 4, 245–63.
Grinnel R., Unrau Y., (2008) Social Work Research and Evaluation: Foundations of Evidence-based Practice. Oxford University Press US.
Holloway, I. (2005) Qualitative Research in Healthcare. McGraw-Hill.
Kirk, J., & Miller, M. L. (1986). Reliability and validity in qualitative research. Beverly Hills, CA: Sage.
Lincoln, Y., Guba, E. (1985) Naturalistic Inquiry. Sage Publications, Newbury Park, CA.
LoBiondo-Wood, G., & Haber, J. (2006). Nursing Research: Methods and Critical Appraisal for Evidence-Based Practice. St. Louis, Missouri: Mosby Elsevier.
Lofland, J., & Lofland, L. H. (1984). Analyzing social settings. Belmont, CA: Wadsworth Publishing Company, Inc.
McBrien, Barry. British Journal of Nursing, 2008 Nov 7-20
Morse, J.M., Barrett, M., Mayan, M., Olson, K., and Spiers, J. (2002). Verification Strategies for Establishing Reliability and Validity in Qualitative Research. International Journal of Qualitative Methods. 1, 2.
Ratcliff, D. (1995). Notes for Five Part Seminar on Qualitative Research.
Rubin, A., Babbie, E.R. (2008). Research methods for social work. Belmont, CA:Thomson Brooks/Cole.
Tanggaard, L. (2008). Objections in research interviewing. International Journal of Qualitative Methods, 7(3), p 15–29.
Walters, J. (2006, February). Doing Qualitative Research: A Practical Handbook.. Adult Education Quarterly, 56(2), 166–167. Retrieved November 29, 2008, doi : 10.1177/0741713605283440
Watson, R., Benner, P., & Ketefian, S. (2008). Nursing Research: Designs and Methods. Elsevier Health Sciences.