Logic model

Example of a logic model for a school-based self-management educational interventions for asthma in children and adolescents.

Logic models are hypothesized descriptions of the chain of causes and effects leading to an outcome of interest (e.g. prevalence of cardiovascular diseases, annual traffic collision, etc). While they can be in a narrative form, logic model usually take form in a graphical depiction of the "if-then" (causal) relationships between the various elements leading to the outcome. However, the logic model is more than the graphical depiction: it is also the theories, scientific evidences, assumptions and beliefs that support it and the various processes behind it. ^{[1] [2]}

Logic models are used by planners, funders, managers and evaluators of programs and interventions to plan, communicate, implement and evaluate them. ^{[3] [2]} They are being employed as well by health scientific community to organize and conduct literature reviews such as systematic reviews. ^{[4] [5]} Domains of application are various, e.g. waste management, ^[6] poultry inspection, ^[7] business education, ^[8] heart disease and stroke prevention. ^[9] Since they are used in various contexts and for different purposes, their typical components and levels of complexity varies in literature (compare for example the W.K. Kellogg Foundation ^[10] presentation of logic model, mainly aimed for evaluation, and the numerous types of logic models in the intervention mapping framework ^[11] ). In addition, depending on the purpose of the logic model, elements depicted and the relationships between them is more or less detailed.

History of logic models

Citing Funnell and Rogers's account (2011), ^[12] Joy A. Frechtling's (2015) encyclopedia article ^[2] traces logic model underpinnings to the 1950s. Patricia J. Rogers's (2005) encyclopedia article ^[3] instead traces it back to Edward A. Suchman's (1967) book ^[13] about evaluative research. Both encyclopedia articles and LeCroy (2018) ^[14] mention increasing interest, usage and publications about the subject.

Uses of the logic model

Program planning

One of the most important uses of the logic model is for program planning. It is suggested to use the logic model to focus on the intended outcomes of a particular program. The guiding questions change from "what is being done?" to "what needs to be done"? McCawley suggests that by using this new reasoning, a logic model for a program can be built by asking the following questions in sequence:

1. What is the current situation that we intend to impact?
2. What will it look like when we achieve the desired situation or outcome?
3. What behaviors need to change for that outcome to be achieved?
4. What knowledge or skills do people need before the behavior will change?
5. What activities need to be performed to cause the necessary learning?
6. What resources will be required to achieve the desired outcome? ^[15]

By placing the focus on ultimate outcomes or results, planners can think backward through the logic model to identify how best to achieve the desired results. Here it helps managers to 'plan with the end in mind', rather than just consider inputs (e.g. budgets, employees) or the tasks that must be done.

Evaluation

The logic model is often used in government or not-for-profit organizations, where the mission and vision are not aimed at achieving a financial benefit. Traditionally, government programs were described only in terms of their budgets. It is easy to measure the amount of money spent on a program, but this is a poor indicator of outcomes. Likewise it is relatively easy to measure the amount of work done (e.g. number of workers or number of years spent), but the workers may have just been 'spinning their wheels' without getting very far in terms of ultimate results or outcomes.

However, nature of outcomes varies. To measure the progress toward outcomes, some initiatives may require an ad hoc measurement instrument. In addition, in programs such as in education or social programs, outcomes are usually in the long-term and may requires numerous intermediate changes (attitudes, social norm, industry practices, etc.) to advance progressively toward the outcomes.

By making clear the intended outcomes and the causal pathways leading to them, a program logic model provides the basis upon which planners and evaluators can develop a measurement plan and adequate instruments. Instead of only looking at the outcome progress, planners can open the "black box" and examine if the intermediate outcomes progress as planned. In addition, the pathways of numerous outcomes are still largely misunderstood due their complexity, their unpredictability and lack of scientific / practical evidences. Therefore, with proper research design, one may not only assess the progress of intermediate outcomes, but evaluate as well if the program theory of change is accurate, i.e. is successful change of an intermediate outcomes provokes the hypothesized subsequent effects in the causal pathway. Finally, outcomes may easily be achieved through processes independent of the program and an evaluation of those outcomes would suggest program success when in fact external outputs were responsible for the outcomes. ^[16]

Various types of logic models

The Inputs → Activities → Outputs → Outcomes template

Many authors and guides use the following template when speaking about logic model: ^{[2] [3] [10] [14] [17]}

Inputs	Activities	Outputs	Outcomes/impacts
what resources go into a program	what activities the program undertakes	what is produced through those activities	the changes or benefits that result from the program
e.g. money, staff, equipment	e.g. development of materials, training programs	e.g. number of booklets produced, workshops held, people trained	e.g. increased skills/ knowledge/ confidence, leading in longer-term to promotion, new job, etc.

Many refinements and variations^{[ which? ]} have been added to the basic template. For example, many versions of logic models set out a series of outcomes/impacts, explaining in more detail the logic of how an intervention contributes to intended or observed results. ^[18] Others often distinguish short-term, medium-term and long-term results, and between direct and indirect results.

Intervention mapping logic models

Main article: Intervention mapping

Logic Model of the Problem for Management information Decision Support Epilepsy Tool (MINDSET program) from Ruiter, DeSmet and Schneider (2007).

The intervention mapping approach of Bartholomew et al. ^[11] makes an extensive use of the logic model through the whole life-cycle of a health promotion program. Since this method can start from as far as a vague desired outcome (author's example is a city whose actors decide to address "health issues" of the city), planners go through various steps in order to develop effective interventions and properly evaluate them. There are distinguishable but closely interwoven logic models with different purposes that can be developed through the process:

• Logic model of the problem, which is a graphical depiction of at-risk population and its social environment behaviors (factors) leading to the health problem and their respective causal pathways (attitudes, beliefs, skills, etc.). This may include as well at-risk population physical environment related causes such as pollutants or lack of physical activity infrastructure and their respective causes, i.e. environmental agents behaviors leading to the physical environment causes and their respective causal pathways;
• Once the most relevant behaviors and causal pathways are identified, planners develop a logic model of change. This is a model of behavioral changes (performance objectives) that should happen and their corresponding necessary changes higher up in the cause-effects chain.
• Finally, a logic model of the intervention is developed. This model describes the various activities that will happen and the cascades of effects they are expected to cause toward the desired outcome.

Evaluators thereafter use the logic model of the intervention to design a proper evaluation plan to assess implementation, impact and efficiency.

Progressive Outcomes Scale Logic Models (POSLM)

Sample Progressive Outcomes Scale Logic Model (POSLM) (2021)

The Progressive Outcomes Scale Logic Model (POSLM) approach was developed by Quisha Brown in response to the racial wealth gap [exacerbated by the COVID-19 pandemic] to aid organizations in the immediate need to add a racial equity focus when developing program logic models. More testing and research is needed in order to verify the validity of this model.

The POSLM approach makes use of the logic model with a strong focus on tracking progressive improvement towards racial disparity outcomes. To measure the progress towards outcomes, this type of logic model states short, intermediate and long-term outcomes as "stage 1", "stage 2" and "stage 3. Each stage is uniquely defined and used to depict the percentage of KPIs achieved at each stage or the percentage of people who reach each stage as they progress on pre-identified Key Performance Indicators (KPI). These KPIs are specific to the racial disparity issues which the population served identifies with (i.e. low reading, financial literacy, unemployment, etc). In an effort to prevent the logic model itself from being cluttered with an overwhelming number of KPIs, the KPIs are arranged by category and only the category is displayed on the logic model. The extensive list of KPIs are an appendix to the logic model. Organizations identify the KPIs and corresponding outcomes by first conducting a needs assessment and/or community focus groups. This helps to ensure that the logic model remains focused on improving the real-time needs of people to remove racial barriers. The POSLM can help to make more clear the intended outcomes and the casual pathways leading to them; both of which help to connect and compose a logical companion "if, then" theory of change statement. Again, more research is needed and currently being conducted as more nonprofits, philanthropic and governments use this model.

Advantages

By describing work in this way, managers have an easier way to define the work and measure it. Performance measures can be drawn from any of the steps. One of the key insights of the logic model is the importance of measuring final outcomes or results, because it is quite possible to waste time and money (inputs), "spin the wheels" on work activities, or produce outputs without achieving desired outcomes. It is these outcomes (impacts, long-term results) that are the only justification for doing the work in the first place. For commercial organizations, outcomes relate to profit. For not-for-profit or governmental organizations, outcomes relate to successful achievement of mission or program goals.^{[ citation needed ]}

Disadvantages

There are some potential disadvantages of logic models due to tendencies toward oversimplification. ^[20] These include:

1. Program logic is no guarantee of actual logic in how the program may work. The world is complex, and some situations cannot be ascertained before they are implemented, so some programs may even progress against the "logic" of the model.
2. It is a partial representation of a complex system.
3. It is a representation of reality, not reality itself. Programs are not linear
4. Normally, it does not include effects besides those initially expected.
5. They do not necessarily establish causality. Many factors exert influence upon the effects.

See also

• Theory of change
• Backcasting
• Critical theory
• Scenario planning
• Thought experiment

References

1. Renger R (2002). "A Three-Step Approach to Teaching Logic Models". The American Journal of Evaluation. 23 (4): 493–503. doi:10.1016/s1098-2140(02)00230-8.
2. Frechtling JA (2015). "Logic Models". International Encyclopedia of the Social & Behavioral Sciences. Elsevier. pp. 299–305. doi:10.1016/b978-0-08-097086-8.10549-5. ISBN   978-0-08-097087-5.
3. "Logic Model". Encyclopedia of Evaluation. Sage Publications, Inc. 2005. doi:10.4135/9781412950558.n321. ISBN   978-0-7619-2609-2.
4. Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P (March 2011). "Using logic models to capture complexity in systematic reviews". Research Synthesis Methods. 2 (1): 33–42. doi:10.1002/jrsm.32. PMID   26061598. S2CID   34282960.
5. Kneale D, Thomas J, Harris K (2015-11-17). "Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews". PLOS ONE. 10 (11): e0142187. Bibcode:2015PLoSO..1042187K. doi: 10.1371/journal.pone.0142187 . PMC   4648510 . PMID   26575182.
6. Industrial Economics, Incorporated (IEc) Evaluation Team (2010). Evaluation of the WasteWise Program (PDF). EPA's Office of Policy, Economics, and Innovation.
7. Development of a logic model and an evaluation framework of the Canadian Food Inspection Agency's Modernized Poultry Inspection Program. Canada. Health Canada. Food Safety Assessment Program. [Ottawa]. 2003. ISBN   978-0-662-35161-0. OCLC   905371520.
8. Hense J, Kriz WC, Wolfe J (February 2009). "Putting theory-oriented evaluation into practice: A logic model approach for evaluating SIMGAME" (PDF). Simulation & Gaming. 40 (1): 110–33. doi:10.1177/1046878107308078. S2CID   61673390.
9. Sitaker M, Jernigan J, Ladd S, Patanian M (April 2008). "Adapting logic models over time: the Washington State Heart Disease and Stroke Prevention Program experience". Preventing Chronic Disease. 5 (2): A60. PMC   2396971 . PMID   18341795.
10. W.K. Kellogg Foundation (1998). W.K. Kellogg Foundation Logic Model Development Guide. Battle Creek: W.K. Kellogg Foundation.
11. Eldredge LK, Markham CM, Ruiter RA, Kok G, Parcel GS (2016). Planning health promotion programs: an intervention mapping approach (Fourth ed.). San Francisco, CA: John Wiley & Sons. ISBN   978-1-119-03556-5. OCLC   914256995.
12. Funnell SC, Rogers PJ (February 2011). Purposeful program theory: Effective use of theories of change and logic models (1st ed.). San Francisco, CA: John Wiley & Sons. ISBN   978-0-470-47857-8. OCLC   660161852.
13. Suchman E (December 1968). Evaluative Research: Principles and Practice in Public Service and Social Action Progr. New York: Russell Sage Foundation. ISBN   978-0-87154-863-4. OCLC   712569.
14. LeCroy CW (2018-06-25). "Logic Models". Encyclopedia of Social Work. doi:10.1093/acrefore/9780199975839.013.1273. ISBN   9780199975839.
15. McCawley PF (1995). The logic model for program planning and evaluation (PDF). University of Idaho Extension.
16. Rossi PH, Lipsey MW, Freeman HE (2004). Evaluation : a systematic approach (7th ed.). Thousand Oaks, CA: Sage. ISBN   978-0-7619-0894-4. OCLC   52706526.
17. McLaughlin JA, Jordan G (2015-10-14). Using Logic Models. John Wiley & Sons, Inc. pp. 62–87. doi:10.1002/9781119171386.ch3. ISBN   978-1-119-17138-6.{{cite book}}: |work= ignored (help)
18. Weiss CH (1972). Evaluation Research. Methods for Assessing Program Effectiveness . Englewood Cliffs, New Jersey: Prentice-Hall, Inc. ISBN   9780132921930.
19. Shegog R, Begley CE (2017). "Clinic-Based Mobile Health Decision Support to Enhance Adult Epilepsy Self-Management: An Intervention Mapping Approach". Frontiers in Public Health. 5: 256. doi: 10.3389/fpubh.2017.00256 . PMC   5632356 . PMID   29043247. The full intervention mapping based protocol is available in the full article
20. Knowlton LW, Phillips CC, Phillips C (2013). The logic model guidebook : better strategies for great results (2nd ed.). Los Angeles: SAGE C. ISBN   978-1-4522-1675-1. OCLC   791492618.

Further reading

• Hernandez M, Hodges S (July 2003). "Crafting logic models for systems of care: Ideas into action.". National Evaluation of the Comprehensive Community Mental Health Services and their Families Program.
• Alter C, Murty S (1997). "Logic modeling: A tool for teaching practice evaluation". Journal of Social Work Education. 33 (1): 103–117. doi:10.1080/10437797.1997.10778856.
• Conrad KJ, Randolph FL (1999). "Creating and using logic models: Four perspectives". Alcoholism Treatment Quarterly . 17 (1–2): 17–32. doi:10.1300/J020v17n01_02.
• Hernandez M (2000). "Using logic models and program theory to build outcome accountability". Education and Treatment of Children. 23 (1): 24–41.
• Julian DA (1997). "The utilization of the logic model as a system level planning and evaluation device". Evaluation and Program Planning. 20 (3): 251–257. doi:10.1016/S0149-7189(97)00002-5.
• McLaughlin JA, Jordan GB (1999). "Logic models: A tool for telling your program's performance story". Evaluation and Program Planning. 22 (1): 65–72. doi:10.1016/S0149-7189(98)00042-1.
• Stinchcomb JB (2001). "Using logic modeling to focus evaluation efforts: Translating operational theories into practical measures". Journal of Offender Rehabilitation. 33 (2): 47–65. doi:10.1300/J076v33n02_04. S2CID   141946096.
• McCawley PF (2001). "The Logic Model for Program planning and Evaluation" (PDF). University of Idaho Extension. Archived from the original (PDF) on 17 November 2010.
• Savitz S, Matthews M, Weilant S (2017). "Assessing Impact to Inform Decisions: A Toolkit on Measures for Policymakers". RAND.
• "A set of three online evaluation tools that includes a Logic Model Builder" . Innovation Network's Point K Logic Model Builder. 2006.
• Unrau YA (2001). "Using client exit interviews to illuminate outcomes in program logic models: A case example". Evaluation and Program Planning. 24 (4m): 353–361. doi:10.1016/S0149-7189(01)00029-5.