Realist Evaluation

Last updated

Realist evaluation or realist review (also realist synthesis) is a type of theory-driven evaluation method used in evaluating social programmes. [1] It is based on the epistemological foundations of critical realism, though one of the originators of realist evaluation, Ray Pawson, who was "initially impressed" by how critical realism explains generative causation in experimental science, later criticised its "philosophical grandstanding" and "explain-all Marxism". [2] Based on specific theories, realist evaluation provides an alternative lens to empiricist evaluation techniques for the study and understanding of programmes and policies. This technique assumes that knowledge is a social and historical product, thus the social and political context as well as theoretical mechanisms, need consideration in analysis of programme or policy effectiveness. [3] [4] [5]

Realist evaluation techniques recognise that there are many interwoven variables operative at different levels in society, thus this evaluation method suits complex social interventions, rather than traditional cause-effect, non-contextual methods of analysis. This realist technique acknowledges that intervention programmes and policy changes do not necessarily work for everyone, since people are different and are embedded in different contexts. [6]

Realist evaluation was popularised by the work of Ray Pawson and Nick Tilley in 1997. [7] They described the procedure followed in the implementation of realist evaluation techniques in programme evaluation and emphasise that once hypotheses have been generated and data collected, the outcomes of the programme are explored, focusing on the groups that the programme benefitted and those who did not benefit. Effectiveness of a programme is thus not dependent on the outcomes alone (cause–effect), rather there is a consideration of the theoretical mechanisms that are applied, and the socio-historical context in which the programmes were implemented. Thus, the final explanation of a programme considers context-mechanism-outcome. [8]

All research methods are applicable in realist evaluations: [9]

"... it is quite possible to carry out realistic evaluation using: strategies, quantitative and qualitative; timescales, contemporaneous or historical; viewpoints, cross-sectional or longitudinal; samples, large or small; goals, action-oriented or audit-centred; and so on and so forth."


Related Research Articles

In analytic philosophy, anti-realism is a position which encompasses many varieties such as metaphysical, mathematical, semantic, scientific, moral and epistemic. The term was first articulated by British philosopher Michael Dummett in an argument against a form of realism Dummett saw as 'colorless reductionism'.

International relations theory is the study of international relations (IR) from a theoretical perspective. It seeks to explain behaviors and outcomes in international politics. The four most prominent schools of thought are realism, liberalism, constructivism, and rational choice. Whereas realism and liberalism make broad and specific predictions about international relations, constructivism and rational choice are methodological approaches that focus on certain types of social explanation for phenomena.

Legal realism is a naturalistic approach to law; it is the view that jurisprudence should emulate the methods of natural science, that is, it should rely on empirical evidence. Hypotheses must be tested against observations of the world.

<span class="mw-page-title-main">Multimethodology</span>

Multimethodology or multimethod research includes the use of more than one method of data collection or research in a research study or set of related studies. Mixed methods research is more specific in that it includes the mixing of qualitative and quantitative data, methods, methodologies, and/or paradigms in a research study or set of related studies. One could argue that mixed methods research is a special case of multimethod research. Another applicable, but less often used label, for multi or mixed research is methodological pluralism. All of these approaches to professional and academic research emphasize that monomethod research can be improved through the use of multiple data sources, methods, research methodologies, perspectives, standpoints, and paradigms.

<span class="mw-page-title-main">Qualitative research</span> Form of research

Qualitative research is a type of research that aims to gather and analyse non-numerical (descriptive) data in order to gain an understanding of individuals' social reality, including understanding their attitudes, beliefs, and motivation. This type of research typically involves in-depth interviews, focus groups, or observations in order to collect data that is rich in detail and context. Qualitative research is often used to explore complex phenomena or to gain insight into people's experiences and perspectives on a particular topic. It is particularly useful when researchers want to understand the meaning that people attach to their experiences or when they want to uncover the underlying reasons for people's behavior. Qualitative methods include ethnography, grounded theory, discourse analysis, and interpretative phenomenological analysis. Qualitative research methods have been used in sociology, anthropology, political science, psychology, communication studies, social work, folklore, educational research, information science and software engineering research.

Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency.

<span class="mw-page-title-main">Methodology</span> Study of research methods

In its most common sense, methodology is the study of research methods. However, the term can also refer to the methods themselves or to the philosophical discussion of associated background assumptions. A method is a structured procedure for bringing about a certain goal, like acquiring knowledge or verifying knowledge claims. This normally involves various steps, like choosing a sample, collecting data from this sample, and interpreting the data. The study of methods concerns a detailed description and analysis of these processes. It includes evaluative aspects by comparing different methods. This way, it is assessed what advantages and disadvantages they have and for what research goals they may be used. These descriptions and evaluations depend on philosophical background assumptions. Examples are how to conceptualize the studied phenomena and what constitutes evidence for or against them. When understood in the widest sense, methodology also includes the discussion of these more abstract issues.

Ram Roy Bhaskar (1944–2014) was an English philosopher of science who is best known as the initiator of the philosophical movement of critical realism (CR). Bhaskar argued that the task of science is "the production of the knowledge of those enduring and continually active mechanisms of nature that produce the phenomena of the world", rather than the discovery of quantitative laws, and that experimental science makes sense only if such mechanisms exist and operate outside the lab as well as inside it. He went on to apply that realism about mechanisms and causal powers to the philosophy of social science, and he also elaborated a series of arguments to support the critical role of philosophy and the human sciences. According to Bhaskar, it is possible and desirable for the study of society to be scientific.

<span class="mw-page-title-main">Grounded theory</span> Qualitative research methodology

Grounded theory is a systematic methodology that has been largely applied to qualitative research conducted by social scientists. The methodology involves the construction of hypotheses and theories through the collecting and analysis of data. Grounded theory involves the application of inductive reasoning. The methodology contrasts with the hypothetico-deductive model used in traditional scientific research.

Evidence-based design (EBD) is the process of constructing a building or physical environment based on scientific research to achieve the best possible outcomes. Evidence-based design is especially important in evidence-based medicine, where research has shown that environment design can affect patient outcomes. It is also used in architecture, interior design, landscape architecture, facilities management, education, and urban planning. Evidence-based design is part of the larger movement towards evidence-based practices.

<span class="mw-page-title-main">Margaret Archer</span> English sociologist (1943–2023)

Margaret Scotford Archer was an English sociologist, who spent most of her academic career at the University of Warwick where she was for many years Professor of Sociology. She was also a professor at l'Ecole Polytechnique Fédérale de Lausanne, Switzerland. She is best known for coining the term elisionism in her 1995 book Realist Social Theory: The Morphogenetic Approach. On 14 April 2014, Archer was named by Pope Francis to succeed former Harvard law professor and US Ambassador to the Holy See Mary Ann Glendon as President of the Pontifical Academy of Social Sciences, and served in this position until her retirement on 27 March 2019.

<span class="mw-page-title-main">Systematic review</span> Comprehensive review of research literature using systematic methods

A systematic review is a scholarly synthesis of the evidence on a clearly presented topic using critical methods to identify, define and assess research on the topic. A systematic review extracts and interprets data from published studies on the topic, then analyzes, describes, critically appraises and summarizes interpretations into a refined evidence-based conclusion. For example, a systematic review of randomized controlled trials is a way of summarizing and implementing evidence-based medicine.

<span class="mw-page-title-main">Left realism</span>

Left realism emerged in criminology from critical criminology as a reaction against what was perceived to be the left's failure to take a practical interest in everyday crime, allowing right realism to monopolize the political agenda on law and order. Left realism argues that crime disproportionately affects working-class people, but that solutions that only increase repression serve to make the crime problem worse. Instead they argue that the root causes of crime lie in relative deprivation, although preventive measures and policing are necessary, but these should be democratically controlled.

<span class="mw-page-title-main">Logic model</span> Method of depicting causal relationships

Logic models are hypothesized descriptions of the chain of causes and effects leading to an outcome of interest. While they can be in a narrative form, logic model usually take form in a graphical depiction of the "if-then" (causal) relationships between the various elements leading to the outcome. However, the logic model is more than the graphical depiction: it is also the theories, scientific evidences, assumptions and beliefs that support it and the various processes behind it.

Ray Pawson is Professor of Social Research Methodology in the School of Sociology and Social Policy at the University of Leeds.

Empowerment evaluation (EE) is an evaluation approach designed to help communities monitor and evaluate their own performance. It is used in comprehensive community initiatives as well as small-scale settings and is designed to help groups accomplish their goals. According to David Fetterman, "Empowerment evaluation is the use of evaluation concepts, techniques, and findings to foster improvement and self-determination". An expanded definition is: "Empowerment evaluation is an evaluation approach that aims to increase the likelihood that programs will achieve results by increasing the capacity of program stakeholders to plan, implement, and evaluate their own programs."

Realism, Realistic, or Realists may refer to:

(R.) Andrew Sayer is Emeritus Professor of Social Theory and Political Economy at Lancaster University, UK. He is known for significant contributions to methodology and theory in the social sciences.

<span class="mw-page-title-main">Critical realism (philosophy of the social sciences)</span>

Critical realism is a philosophical approach to understanding science, and in particular social science, initially developed by Roy Bhaskar (1944–2014). It specifically opposes forms of empiricism and positivism by viewing science as concerned with identifying causal mechanisms. In the last decades of the twentieth century it also stood against various forms of postmodernism and poststructuralism by insisting on the reality of objective existence. In contrast to positivism's methodological foundation, and poststructuralism's epistemological foundation, critical realism insists that (social) science should be built from an explicit ontology. Critical realism is one of a range of types of philosophical realism, as well as forms of realism advocated within social science such as analytic realism and subtle realism.

Theory-driven evaluation is an umbrella term for any approach to program evaluation that develops a theory of change and uses it to design, implement, analyze, and interpret findings from an evaluation. More specifically, an evaluation is theory-driven if it:

  1. formulates a theory of change using some combination of social science, beneficiary lived experience, and program-related professionals' expertise;
  2. develops and prioritizes evaluation questions using the theory;
  3. uses the theory to guide the design and implementation of the evaluation;
  4. uses the theory to operationalize contextual, process, and outcome variables; and
  5. provides a causal explanation of how and why outcomes were achieved, including whether the program worked and/or had any unintended consequences, and what moderates outcomes.

References

  1. ↑ Marchal, B (2012). "Is realist evaluation keeping its promise?A review of published empirical studies in the field of health systems research". Evaluation. 18 (2): 192–212. doi:10.1177/1356389012442444. S2CID   145299417.
  2. ↑ Pawson, R. (2016). The ersatz realism of critical realism: A reply to Porter. Evaluation, 22, 49–57.
  3. ↑ Secord, P; Secord M (1983). "Implications for psychology of the new philosophy of science". American Psychologist. 38 (4): 399–413. doi:10.1037/0003-066x.38.4.399.
  4. ↑ Dalkin, SM; Jones, D; Lhussier, M; Cunningham, B (2012). "Understanding integrated care pathways in palliative care using realist evaluation:a mixed methods study protocol". BMJ Open . 3 (4): 1–6. doi:10.1136/bmjopen-2012-001533. PMC   3391371 . PMID   22761292. Open Access logo PLoS transparent.svg
  5. ↑ Dada, Sara; Dalkin, Sonia; Gilmore, Brynne; Hunter, Rebecca; Mukumbang, Ferdinand C. (2023-03-14). "Applying and reporting relevance, richness and rigour in realist evidence appraisals: Advancing key concepts in realist reviews". Research Synthesis Methods: jrsm.1630. doi: 10.1002/jrsm.1630 . ISSN   1759-2879. PMID   36872619. S2CID   257363167.
  6. ↑ Hewitt, G; S. Sims; R. Harris (2012). "The realist approach to evaluation research:an introduction". International Journal of Therapy and Rehabilitation. 19 (5): 205–259. doi:10.12968/ijtr.2012.19.5.250.
  7. ↑ Pawson, Ray (1997). Realistic evaluation. Nick Tilley. London: Sage. ISBN   0-7619-5008-7. OCLC   37255786.
  8. ↑ R Pawson, N. Tilley, "Realistic Evaluation" (Sage), London.
  9. ↑ Pawson, R., & Tilley, N. (1997, p. 85). Realistic Evaluation. SAGE Publications Ltd.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.