Most significant change technique

Last updated

The Most Significant Change Technique (MSC) is a monitoring and evaluation (M&E) method used for the monitoring and evaluating of complex development interventions. It was developed by Rick Davies as part of his PhD field work with the Christian Commission for Development in Bangladesh (CCDB) in 1994. [1] CCDB, a Bangladeshi NGO, subsequently continued and expanded the use of MSC to monitor the impact of its participatory rural development projects for the rest of the decade. [2]

Contents

Following publication of the CCDB experience on the internet in 1996, [3] MSC was progressively adopted for use by other NGOs in Africa, Asia, Latin America and Australasia. These experiences were then documented in the 2005 MSC Guide, co-authored by Davies and Dart, [4] which remains the most widely cited reference on how to use MSC. Jess Dart, the co-author of the Guide, carried out the first use of MSC in Australia as part of her PhD research. Her company Clear Horizon has since been the main provider of MSC training in Australia. [5]

MSC represents a shift away from more conventional quantitative and expert driven evaluation methods toward a more qualitative and participant driven approach, focusing on the human impact of interventions. [6] In summary: the MSC process typically involves the collection of qualitative information from the intended beneficiaries of an intervention, in the form of a description of a change each considers as the most significant within a given period of time; and then an explanation of why they see that change as most significant. This collection process is followed by the use of one or more selection panels, where those participants (or other stakeholders) review the set of collected MSC stories and identify the one which they agree (and explain) is most significant of all, as seen from their perspective. [4]

Steps involved in the Most Significant Change process

There are 10 steps involved in the Most Significant Change process [7]

  1. Starting and raising interest
  2. Defining the domains of change
  3. Defining the reporting period
  4. Collecting significant change stories
  5. Selecting the most significant of these stories
  6. Feeding back the results of the selection process
  7. Verification of stories
  8. Quantification
  9. Secondary analysis and meta-monitoring
  10. Revising the system

Steps 4 to 6 are the essential kernel of the process. Steps 1 to 4 are necessary preparatory steps. Steps 7 to 10 are optional follow up steps

Benefits and limitations of the MSC technique

Benefits

MSC enables the identification of the kinds of changes people value, why they value those changes and the extent to which they share those views with others. MSC stories, and discussions of these, can also shed light on the causal processes that generated the changes, and implications for actions that could be taken in response to those changes. [4]

The focus is on learning rather than accountability. Managers, field staff and evaluation team members can be asked to reflect on and openly question the intervention programme and their interactions with the community in which the intervention takes place. Its use can also support other M&E processes, for example by identifying the aspects of the intervention which could benefit from more quantitative investigations. In addition, the process can give the evaluators a greater sensitivity to the perspectives of beneficiaries, which it could be argued, may be conducive to more successful outcomes. [8]

Limitations

  1. MSC is a participatory process that emphasizes dialogue. This can be time consuming. Both participants and facilitators of an MSC exercise may have other pressing demands on their time
  2. The elicitation of individual MSC stories and the facilitation of discussions about multiple MSC stories both require some skill, in interviewing and small group facilitation
  3. Participants will often have concerns and reservations about the meaning of "significance", the acceptability of subjectivity and the appearance of competition, which need to be addressed
  4. The process as described by Davies and Dart in 2005 may be incompletely implemented. Changes may be adequately described but explanations of their significance may be cursory or absent (Step 4). MSC stories may be collected then not followed by a selection process (Step 5). Feedback of the choices made between a set of MSC stories, to the original providers of those stories, may be forgotten (Step 6)

These and other challenges are discussed in Chapter 3: Troubleshooting in the 2005 MSC Guide [4]

Usage

MSC is now widely used by development aid agencies, especially NGOs. The original MSC Guide has since been translated into 13 languages (Arabic, Bangla, French, Hindi, Bahasa Indonesian, Japanese, Malayalam, Russian, Sinhala, Tamil, Spanish and Urdu), typically by organisations working within those language groups. [9]

Since 2000 there has been an active and global "community of practice" that shares experiences with the use of MSC in different settings. The email list used by this group is currently hosted by Google Groups.

More recently, an online bibliography has accumulated information on the global use of MSC . References are now available on more than 290 papers and reports that have been published on the use of MSC. These cover both developed and developing countries and relate to interventions in a wide range of sectors – including health, education, agriculture, infrastructure, governance, and community development. [10]

Related Research Articles

<span class="mw-page-title-main">Non-governmental organization</span> Organization independent of any government, usually created to aid those in need

A non-governmental organization (NGO) or non-governmental organisation is an organization that generally is formed independent from government. They are typically nonprofit entities, and many of them are active in humanitarianism or the social sciences; they can also include clubs and associations that provide services to their members and others. Surveys indicate that NGOs have a high degree of public trust, which can make them a useful proxy for the concerns of society and stakeholders. However, NGOs can also be lobby groups for corporations, such as the World Economic Forum. NGOs are distinguished from international and intergovernmental organizations (IOs) in that the latter are more directly involved with sovereign states and their governments.

Evaluation is a systematic determination and assessment of a subject's merit, worth and significance, using criteria governed by a set of standards. It can assist an organization, program, design, project or any other intervention or initiative to assess any aim, realisable concept/proposal, or any alternative, to help in decision-making; or to ascertain the degree of achievement or value in regard to the aim and objectives and results of any such action that has been completed. The primary purpose of evaluation, in addition to gaining insight into prior or existing initiatives, is to enable reflection and assist in the identification of future change. Evaluation is often used to characterize and appraise subjects of interest in a wide range of human enterprises, including the arts, criminal justice, foundations, non-profit organizations, government, health care, and other human services. It is long term and done at the end of a period of time.

Scenario planning, scenario thinking, scenario analysis, scenario prediction and the scenario method all describe a strategic planning method that some organizations use to make flexible long-term plans. It is in large part an adaptation and generalization of classic methods used by military intelligence.

Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency. In both the public and private sectors, stakeholders often want to know whether the programs they are funding, implementing, voting for, receiving or objecting to are producing the intended effect. While program evaluation first focuses around this definition, important considerations often include how much the program costs per participant, how the program could be improved, whether the program is worthwhile, whether there are better alternatives, if there are unintended outcomes, and whether the program goals are appropriate and useful. Evaluators help to answer these questions, but the best way to answer the questions is for the evaluation to be a joint project between evaluators and stakeholders.

<span class="mw-page-title-main">BRAC (organisation)</span> International development organization based in Bangladesh

BRAC is an international development organisation based in Bangladesh. In order to receive foreign donations, BRAC was subsequently registered under the NGO Affairs Bureau of the Government of Bangladesh. BRAC is the largest non-governmental development organisation in the world, in terms of number of employees as of September 2016. Established by Sir Fazle Hasan Abed in 1972 after the independence of Bangladesh, BRAC is present in all 64 districts of Bangladesh as well as 11 other countries in Asia, Africa, and the Americas.

Danish Committee for Aid to Afghan Refugees (DACAAR)(Danish: Den danske komité for hjælp til afghanske flygtninge) is a non-political, non-governmental, non-profit humanitarian and development organization working to improve the lives of the Afghan people since 1984.

<span class="mw-page-title-main">Capacity building</span> Process by which individuals or organizations improve their capability to produce, perform or deploy

Capacity building is the improvement in an individual's or organization's facility "to produce, perform or deploy". The terms capacity building and capacity development have often been used interchangeably, although a publication by OECD-DAC stated in 2006 that capacity development was the preferable term. Since the 1950s, international organizations, governments, non-governmental organizations (NGOs) and communities use the concept of capacity building as part of "social and economic development" in national and subnational plans. The United Nations Development Programme defines itself by "capacity development" in the sense of "'how UNDP works" to fulfill its mission. The UN system applies it in almost every sector, including several of the Sustainable Development Goals to be achieved by 2030. For example, the Sustainable Development Goal 17 advocates for enhanced international support for capacity building in developing countries to support national plans to implement the 2030 Agenda. 

<span class="mw-page-title-main">Community-led total sanitation</span> Approach to improve sanitation and hygiene practices, mainly in developing countries

Community-led total sanitation (CLTS) is an approach used mainly in developing countries to improve sanitation and hygiene practices in a community. The approach tries to achieve behavior change in mainly rural people by a process of "triggering", leading to spontaneous and long-term abandonment of open defecation practices. It focuses on spontaneous and long-lasting behavior change of an entire community. The term "triggering" is central to the CLTS process: It refers to ways of igniting community interest in ending open defecation, usually by building simple toilets, such as pit latrines. CLTS involves actions leading to increased self-respect and pride in one's community. It also involves shame and disgust about one's own open defecation behaviors. CLTS takes an approach to rural sanitation that works without hardware subsidies and that facilitates communities to recognize the problem of open defecation and take collective action to clean up and become "open defecation free".

A parent education program is a course that can be followed to correct and improve a person's parenting skills. Such courses may be general, covering the most common issues parents may encounter, or specific, for infants, toddlers, children and teenagers. These courses may also be geared towards parents who are considering having a child, or adopting one, or are pregnant.

<span class="mw-page-title-main">Renewable Energy and Energy Efficiency Partnership</span> Organization for renewable energy

The Renewable Energy and Energy Efficiency Partnership (REEEP) is a Vienna-based international organisation that advances markets for renewable energy and energy efficiency with a particular emphasis on the emerging markets and developing countries. Its primary focus is in de-risking and scaling up clean energy business models.

Praxis intervention is a form of participatory action research that emphasizes working on the praxis potential, or phronesis, of its participants. This contrasts with other forms of participatory action research, which emphasize the collective modification of the external world. Praxis potential means the members' potential to reflexively work on their respective mentalities; participant here refers not just to the clientele beneficiaries of the praxis intervention project, but also the organisers and experts participating in such a project. Praxis intervention is intended to lead its members through a "participant objectivation". The method prioritizes unsettling the settled mentalities, especially where the settled mindsets prevalent in the social world or individuals is suspected to have sustained or contributed to their suffering or marginality.

<span class="mw-page-title-main">Participatory impact pathways analysis</span>

Participatory impact pathways analysis (PIPA) is a project management approach in which the participants in a project, including project staff, key stakeholders, and the ultimate beneficiaries, together co-construct their program theory.

Impact evaluation assesses the changes that can be attributed to a particular intervention, such as a project, program or policy, both the intended ones, as well as ideally the unintended ones. In contrast to outcome monitoring, which examines whether targets have been achieved, impact evaluation is structured to answer the question: how would outcomes such as participants' well-being have changed if the intervention had not been undertaken? This involves counterfactual analysis, that is, "a comparison between what actually happened and what would have happened in the absence of the intervention." Impact evaluations seek to answer cause-and-effect questions. In other words, they look for the changes in outcome that are directly attributable to a program.

<span class="mw-page-title-main">Strengthening Participatory Organization</span>

Strengthening Participatory Organization is the largest rights-based national support organization in Pakistan working since 1994 to strengthen and support community organizations and public interest institutions for promotion of democratic governance, social justice, peace and social harmony. SPO engages civil society networks, faith-based organisations and groups representing a wide range of stakeholders.

<span class="mw-page-title-main">Theory of change</span> Methodology for social impact

Theory of Change (ToC) is a methodology or a criteria for planning, participation, adaptive management, and evaluation that is used in companies, philanthropy, not-for-profit, international development, research, and government sectors to promote social change. Theory of Change defines long-term goals and then maps backward to identify necessary preconditions.

The PRECEDE–PROCEED model is a cost–benefit evaluation framework proposed in 1974 by Lawrence W. Green that can help health program planners, policy makers and other evaluators, analyze situations and design health programs efficiently. It provides a comprehensive structure for assessing health and quality of life needs, and for designing, implementing and evaluating health promotion and other public health programs to meet those needs. One purpose and guiding principle of the PRECEDE–PROCEED model is to direct initial attention to outcomes, rather than inputs. It guides planners through a process that starts with desired outcomes and then works backwards in the causal chain to identify a mix of strategies for achieving those objectives. A fundamental assumption of the model is the active participation of its intended audience — that is, that the participants ("consumers") will take an active part in defining their own problems, establishing their goals and developing their solutions.

The Emergency Capacity Building Project is a collaborative capacity-building project aimed at improving the speed, effectiveness and delivery of humanitarian response programs. The ECB Project is a partnership between seven non-governmental organizations (NGOs), and implements programs in one region and four countries known as consortia.

<span class="mw-page-title-main">Advocacy evaluation</span>

Advocacy evaluation, also called public policy advocacy design, monitoring, and evaluation, evaluates the progress or outcomes of advocacy, such as changes in public policy.

Constituent Voice is a performance management and measurement method designed as a continual improvement process developed by Keystone Accountability to enable organisations addressing social issues to improve their results by improving relationships with their constituents. Like customer satisfaction measurement, which it draws from, Constituent Voice treats measurement as an aspect of an intervention that not only provides metrics but contributes directly to outcomes by increasing the engagement of intended beneficiaries.

The Challenging the Frontiers of Poverty Reduction: Targeting the Ultra Poor (CFPR-TUP) project was initiated by BRAC, a Bangladesh-based development organisation in 2002. The ultra poor are a group of people who eat below 80% of their energy requirements despite spending at least 80% of income on food. In Bangladesh, they constitute the poorest 17.5 percent of the population. These people suffer from chronic hunger and malnutrition, have inadequate shelter, are more prone to disease, deprived of education and more vulnerable to recurring natural disasters. The CFPR-TUP programme is aimed at households which are too poor to access the benefits from development interventions such as microfinance and assists them to access mainstream development services. The program costs around US$35 million a year.

References

  1. ↑ Davies, R. (1998). Order and Diversity: Representing and Assisting Organisational Learning in Non-Government Aid Organisations.
  2. ↑ Monitoring Significant Changes in people’s Lives (2000). Planning, Monitoring and Evaluation (PME) section, CCDB. Dhaka, Bangladesh Retrieved 11 September 2021
  3. ↑ Davies, R. (1996). An evolutionary approach to organisational learning: An experiment by an NGO in Bangladesh.
  4. 1 2 3 4 Rick Davies,Jess Dart The Most Significant Change (MSC) Technique: A Guide to Its Use (2005)
  5. ↑ Dart, J., & Davies, R. (2003). A Dialogical, Story-Based Evaluation Tool: The Most Significant Change Technique. American Journal of Evaluation, 24(2), 137–155.
  6. ↑ "Most significant change". Government of Australia. 7 May 2010. Archived from the original on 10 May 2012. Retrieved 18 September 2012.
  7. ↑ McDonald, David; Gabrielle Bammer; Peter Deane (2009). Research Integration Using Dialogue Methods. Canberra: ANU E-Press. ISBN   978-1-921536-74-8.
  8. ↑ Willetts, Juliet; Paul Crawford (2007). "The most significant lessons about the most significant change technique". Development in Practice. 17 (3): 367–379. doi:10.1080/09614520701336907. hdl: 10453/4480 . S2CID   55718135.
  9. ↑ "Translations of the "Most Significant Changes" Guide" . Retrieved 6 November 2013.
  10. ↑ Tonkin, K., Silver, H., Pimentel, J., Chomat, A. M., Sarmiento, I., Belaid, L., Cockcroft, A., & Andersson, N. (2021). "How beneficiaries see complex health interventions: A practice review of the Most Significant Change in ten countries". Archives of Public Health, 79.
      This page is based on this Wikipedia article
      Text is available under the CC BY-SA 4.0 license; additional terms may apply.
      Images, videos and audio are available under their respective licenses.