Using theory of change to design and evaluate public health interventions: A systematic review

Implementation Science

Volumn 11, Issue 1, 2016, Pages

  Breuer, Erica ^a   Lee, Lucy ^b   De Silva, Mary ^b   Lund, Crick ^a  

^a UNIVERSITY OF CAPE TOWN   (South Africa)

^b King's College London   (United Kingdom)

Author keywords

Evaluation; Intervention development; Programme theory; Public health; Systematic review; Theory of change

Indexed keywords

CHECKLIST; HUMAN; LANGUAGE; MEDLINE; PSYCINFO; PUBLIC HEALTH; SCOPUS; STUDY DESIGN; SYSTEMATIC REVIEW; THEORETICAL MODEL; HEALTH CARE PLANNING; METHODOLOGY; PROCEDURES;

HEALTH PLAN IMPLEMENTATION; HUMANS; PUBLIC HEALTH; RESEARCH DESIGN;

EID: 84965032074     PISSN: None     EISSN: 17485908     Source Type: Journal    
DOI: 10.1186/s13012-016-0422-6     Document Type: Article

References (87)

1. Mackenzie M, O'Donnell C, Halliday E, Sridharan S, Platt S. Do health improvement programmes fit with MRC guidance on evaluating complex interventions? BMJ. 2010;340:c185. doi: 10.1136/bmj.c185.
2. Ling T. Evaluating complex and unfolding interventions in real time. Evaluation. 2012;18:79-91. doi: 10.1177/1356389011429629.
3. De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, et al. Theory of Change: a theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials. 2014;15:267. doi: 10.1186/1745-6215-15-267.
4. Coryn CLS, Noakes LA, Westine CD, Schroter DC. A systematic review of theory-driven evaluation practice from 1990 to 2009. Am J Eval. 2011;32:199-226. doi: 10.1177/1098214010389321.
5. Funnell SC, Rogers PJ. Purposeful program theory: effective use of theories of change and logic models. 1st ed. San Francisco, CA: Jossey-Bass; 2011.
6. US Department of Health and Human Services, Centres for Disease Conrol and Prevention, Office of the Director, Office of Strategy and Innovation. Introduction to program evaluation for public health programs: a self study guide. Atlanta, GA: Centres for Disease Control and Prevention; 2011.
7. Weiss C. Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community initiatives for children and families. In: Connell JP, editor. New approaches to evaluating community initiatives: concepts, methods, and contexts. Washington, DC: Aspen Institute; 1995. p. 65-92.
8. Vogel I. Review of the use of 'Theory of Change' in international development. UK: Department for International Development (DFID); 2012.
9. Andersen A. A community builder's approach to theory of change: a practical guide to theory development. New York: The Aspen Insitute; 2004.
10. Sullivan H, Stewart M. Who owns the theory of change? Evaluation. 2006;12(2):179-99. doi: 10.1177/1356389006066971.
11. Breuer E, De Silva M, Fekadu A, Luitel N, Murhar V, Nakku J, et al. Using workshops to develop theories of change in five low and middle income countries: lessons from the programme for improving mental health care (PRIME). Int J Ment Health Syst. 2014;8(1):15.
12. Hernandez M, Hodges S. Applying a theory of change approach to interagency planning in child mental health. Am J Community Psychol. 2006;38(3-4):165-73. doi: 10.1007/s10464-006-9079-7.
13. Weitzman BC, Silver D, Dillman KN. Integrating a comparison group design into a theory of change evaluation: the case of the urban health initiative. Am J Eval. 2002;23(4):371-85. doi: 10.1177/109821400202300402.
14. Knowlton LW, Phillips C. Corporate giving gets smarter: ConAgra Foods Foundation fights childhood hunger. Found Rev. 2012;4:2. doi: 10.4087/FOUNDATiONREVIEW-D-11-00034.1.
15. Connell JP, Kubisch AC. Applying a theory of change approach to the evaluation of comprehensive community initiatives: progress, prospects, and problems. In: Fulbright-Andersen K, Kubisch A, Connell JP, editors. New Approaches to Evaluating Community Initiatives, Vol. 2: Theory, Measurement, and Analysis United States of America: The Aspen Institute; 1998. p. 15-44.
16. Bauer GF. Developing community health indicators to support comprehensive community building initiatives: a case study of a participatory action research project. University of California, Berkely, US: ProQuest Information & Learning; 1999
17. Prinsen G, Nijhof S. Between logframes and theory of change: reviewing debates and a practical experience. Dev Pract. 2015;25(2):234-46. doi: 10.1080/09614524.2015.1003532.
18. Marchal B, van Belle S, van Olmen J, Hoeree T, Kegels G. Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation. 2012;18:192-212. doi: 10.1177/1356389012442444.
19. James C. Theory of change review: a report comissioned by Comic Relief; 2011. http://mande.co.uk/2012/uncategorized/comic-relief-theory-of-change-review/.
20. Blamey A, Mackenzie M. Theories of change and realistic evaluation. Evaluation. 2007;13(4):439-55. doi: 10.1177/1356389007082129.
21. Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. 2002;56:119-27.
22. Yin RK. Case study research: design and methods. Fourth ed. Applied Social Research Methods Series. Sage Publications; 2009
23. Endnote. Endnote. X7 ed. San Fransisco: Thomson Reuters; 2015.
24. Evernote Cooperation. California: Evernote Corporation; 2015.
25. Statcorp. STATA. 13th ed. College Station: Statcorp; 2013.
26. Andersen R, Nesman T, Hodges S. Expressing your ideas: using a logic model to create system change USF logic model team, Tampa, Florida. University of South Florida: Tools for Creating and Sustaining Human Services: Theory of Change Logic Models; 2012.
27. Andreas D, Ja DY, Wilson S. Peers reach out supporting peers to embrace recovery (PROSPER): a center for substance abuse treatment recovery community services program. Alcohol Treat Q. 2010;28(3):326-38. doi: 10.1080/07347324.2010.488538.
28. AusAID. AUusAID's mulitsectoral contribution to the national response in Papua New Guinea. Annex 10 Theory of Change 2010. http://www.ode.ausaid.gov.au/publications/documents/pnghiv/annex10-theoryofchange.pdf. Accessed date 16 Jan 2014.
29. Bacchus LJ, Bewley S, Vitolas CT, Aston G, Jordan P, Murray SF. Evaluation of a domestic violence intervention in the maternity and sexual health services of a UK hospital. Reprod Health Matters. 2010;18(36):147-57. doi: 10.1016/s0968-8080(10)36526-810.1136/.
30. Barton WH, Powers GT, Morris ES, Harrison A. Evaluating a comprehensive community initiative for children, youth, and families. Adolesc Family Health. 2001;2(1):27-36.
31. Basson HA, Roets L. Workplace wellness for HIV/AIDS affected nurses in South Africa. Br J Nurs. 2013;22(1):38-44.
32. Bauld L, Judge K, Barnes M, Benzeval M, MacKenzie M, Sullivan H. Promoting social change: the experience of health action zones in England. J Soc Policy. 2005;34(3):427-45. doi: 10.1017/S0047279405008858.
33. Beeston C, Robinson M, Craig N. Monitoring and evaluating Scotland's alcohol strategy. Setting the scene: theory of change and baseline picture. Edinburgh: NHS Health Scotland; 2011.
34. Bhattacharjee P. Use of theory of change to guide the design and monitoring of an intervention intended to reduce partner violence and increase protected sex among sex workers. STRIVE Ann Meet. 2013.
35. Birkby BW. The design and implementation of an evaluation for a comprehensive community-wide initiative on child maltreatment (Kentucky). Kentucky University, US: ProQuest Information & Learning; 2001.
36. Bonner L. Using theory-based evaluation to build evidence-based health and social care policy and practice. Crit Public Health. 2003;13(1):77-92. doi: 10.1080/0958159031000100224.
37. Brown EC, Hawkins JD, Rhew IC, Shapiro VB, Abbott RD, Oesterle S et al. Prevention system mediation of communities that care effects on youth outcomes. Prev Sci. 2013:1-10. doi: 10.1007/s11121-013-0413-7
38. Carr SM, Lhussier M, Wilkinson J, Gleadhill S. Empowerment evaluation applied to public health practice. Crit Public Health. 2008;18(2):161-74. doi: 10.1080/09581590701499327.
39. Carroll M, David M, Jacobs B, Judge K, Wilkes B. A realistic/theory of change approach to the evaluation of health promotion in small- and medium-sized enterprises in Sandwell. Soc Policy Soc. 2005;4(4):393.
40. Chandani Y, Noel M, Pomeroy A, Andersson S, Pahl MK, Williams T. Factors affecting availability of essential medicines among community health workers in Ethiopia, Malawi, and Rwanda: solving the last mile puzzle. Am J Trop Med Hyg. 2012;87(5):120-6. doi: 10.4269/ajtmh.2012.11-0781.
41. Cole DC, Hogg-Johnson S, Manno M, Ibrahim S, Wells RP, Ferrier SE, et al. Reducing musculoskeletal burden through ergonomic program implementation in a large newspaper. Int Arch Occup Environ Health. 2006;80(2):98-108. doi: 10.1007/s00420-006-0107-6.
42. De La Rosa IA, Perry J, Dalton LE, Johnson V. Strengthening families with first-born children: exploratory story of the outcomes of a home visiting intervention. Res Soc Work Pract. 2005;15(5):323-38. doi: 10.1177/1049731505277004.
43. De La Rosa IA, Perry J, Johnson V. Benefits of increased home-visitation services: exploring a case management model. Family Community Health. 2009;32(1):58-75.
44. Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167-205. doi: 10.1111/j.1468-0009.2011.00625.x.
45. Dixon-Woods M, Tarrant C, Willars J, Suokas A. How will it work? A qualitative study of strategic stakeholders' accounts of a patient safety initiative. Qual Saf Health Care. 2010;19(1):74-8. doi: 10.1136/qshc.2008.029504.
46. Goss-Power PJ. Evaluation of a school-based program for youth with emotional and behavioral disabilities. New England Graduate School, US: ProQuest Information & Learning; 2005.
47. Gray P, Seddon T. Prevention work with children disaffected from school: findings from the evaluation of two innovative community-based projects. Health Educ. 2005;105(1):62-72. doi: 10.1108/09654280510572312.
48. Gregor M. TB alert change story map: charities evaluation services: National Performance Programme. London: Charities Evaluation Services; 2009. http://www.ces-vol.org.uk/Resources/CharitiesEvaluationServices/Documents/tbalertchangestorymaptoc-802-810.pdf. Accessed date 16 Jan 2014.
49. Henderson PM. Program evaluation of a long-term substance abuse treatment program for homeless men. US: ProQuest Information & Learning; 2004.
50. Illinois Caucus for Adolescent Health. Strategic Plan. 2013. http://icah.org/about/#strategic-plan. Accessed 16th January 2014.
51. Kemp L, Harris E, McMahon C, Matthey S, Vimpani G, Anderson T, et al. Benefits of psychosocial intervention and continuity of care by child and family health nurses in the pre- and postnatal period: process evaluation. J Adv Nurs. 2013;69(8):1850-61. doi: 10.1111/jan.12052.
52. Kreger M, Sargent K, Arons A, Standish M, Brindis CD. Creating an environmental justice framework for policy change in childhood asthma: a grassroots to treetops approach. Am J Public Health. 2011;101 Suppl 1:S208-16. doi: 10.2105/ajph.2011.300188.
53. Levinson-Johnson J. Monroe Country ACCESS Logic Model. Tools for creating and sustaining human services: theory of change logic models: University of South Florida 2012
54. Levison-Johnson J, Wenz-Gross M. From complexity to reality: providing useful frameworks for defining systems of care. Eval Program Plann. 2010;33(1):56-8. doi: 10.1016/j.evalprogplan.2009.05.005.
55. Lund C, Tomlinson M, de Silva M, Fekadu A, Shidhaye R, Jordans M, et al. PRIME: a programme to reduce the treatment gap for mental disorders in five low- and middle-income countries. PLoS Med. 2012;9(12):e1001359. doi: 10.1371/journal.pmed.1001359.
56. Macfarlane F, Greenhalgh T, Humphrey C, Hughes J, Butler C, Pawson R. A new workforce in the making?: a case study of strategic human resource management in a whole-system change effort in healthcare. J Health, Organ Manag. 2011;25(1):55-72. doi: 10.1108/14777261111116824.
57. Mackenzie M. Benefit or burden: introducing paraprofessional support staff to health visiting teams: the case of Starting Well. Health Soc Care Community. 2006;14(6):523-31. doi: 10.1111/j.1365-2524.2006.00640.x.
58. Mackenzie M, Blamey A. The practice and the theory: lessons from the application of a theories of change approach. Evaluation. 2005;11(2):151-68. doi: 10.1177/1356389005055538.
59. Mackenzie M, Blamey A, Halliday E, Maxwell M, McCollam A, McDaid D, et al. Measuring the tail of the dog that doesn't bark in the night: the case of the national evaluation of Choose Life (the national strategy and action plan to prevent suicide in Scotland). BMC Public Health. 2007;7:146. doi: 10.1186/1471-2458-7-146.
60. Mackenzie M, Reid M, Turner F, Wang YY, Clarke J, Sridharan S, et al. Reaching the hard-to-reach: conceptual puzzles and challenges for policy and practice. J Soc Policy. 2012;41(03):511-32. doi: 10.1017/S0047279412000074.
61. Maselli B. Thrive community youth logic model: working together for a brighter tomorrow Lewiston, Maine. Tools for Creating and Sustaining Human Services: Theory of Change Logic Models: University of South Florida 2012
62. Mason P. Visual data in applied qualitative research: lessons from experience. Qual Res. 2005;5(3):325-46. doi: 10.1177/1468794105054458.
63. McQuiston C, Choi-Hevel S, Clawson M. Protegiendo nuestra comunidad: empowerment participatory education for HIV prevention. J Transcult Nurs. 2001;12(4):275-83.
64. Mookherji S, LaFond A. Strategies to maximize generalization from multiple case studies: lessons from the Africa Routine Immunization System Essentials (ARISE) project. Evaluation. 2013;19(3):284-303. doi: 10.1177/1356389013495212.
65. Morilus-Black M, McCarthy V, Kernan JB. Using a logic model as a tool to apply a theory of change approach to support implementation, evaluation and strategic planning. Tools for creating and sustaining human services: theory of change logic models: University of South Florida 2012
66. Reid M, Botma Y. A framework to expand public services to children with biomedical healthcare needs related to HIV in the Free State, South Africa. Afr J Aids Res. 2012;11(2):91-8. doi: 10.2989/16085906.2012.698054.
67. Riley AJ, Byng R, White C, Smith S. Utilising theories of change to understand the engagement of general practitioners in service improvement: a formative evaluation of the Lewisham Depression Programme. Qual Prim Care. 2008;16(1):17-26.
68. Rivera JA, Martorell R, Gonzalez W, Lutter C, Cossio TG, Flores-Ayala R, et al. Prevención de la desnutrición de la madre y el niño: el componente de nutrición de la Iniciativa Salud Mesoamérica 2015. Salud Publica Mex. 2011;53 Suppl 3:S303-11.
69. Rodriguez MH, Betanzos-Reyes AF. Grupo de Trabajo de Malaria del Sistema Mesoamericano de Salud P. Plan de mejoramiento del control de la malaria hacia su eliminación en Mesoamérica. Salud Publica Mex. 2011;53 Suppl 3:S333-48.
70. Scanlon DP, Beich J, Alexander JA, Christianson JB, Hasnain-Wynia R, McHugh MC, et al. The aligning forces for quality initiative: background and evolution from 2005 to 2012. Am J Manag Care. 2012;18:S115-25.
71. Schierhout G, Hains J, Si D, Kennedy C, Cox R, Kwedza R, et al. Evaluating the effectiveness of a multifaceted, multilevel continuous quality improvement program in primary health care: developing a realist theory of change. Implementation Sci. 2013;8(1):119. doi: 10.1186/1748-5908-8-119.
72. Secker J, Bowers H, Webb D, Llanes M. Theories of change: what works in improving health in mid-life? Health Educ Res. 2005;20(4):392-401. doi: 10.1093/her/cyg133.
73. Smith N, Barnes M. New jobs old roles-working for prevention in a whole-system model of health and social care for older people. Health Soc Care Community. 2013;21(1):79-87. doi: 10.1111/j.1365-2524.2012.01089.x.
74. Suarez-Balcazar Y. Empowerment and participatory evaluation of a community health intervention: Implications for occupational therapy. OTJR Occup Participation Health. 2005;25(4):133-42.
75. Tran L. A theory of change for mental health advocacy: a case study Charities evaluation services; 2009. http://www.cesvol.org.uk/Resources/CharitiesEvaluationServices/Documents/kingsfundtoccasestudy-801-809.pdf. Accessed date 16 Jan 2014.
76. Tucker P, Liao Y, Giles WH, Liburd L. The REACH 2010 logic model: an illustration of expected performance. Prev Chronic Dis. 2006;3(1):A21.
77. Vander Stoep A, Williams M, Jones R, Green L, Trupin E. Families as full research partners: what's in it for us? J Behav Health Serv Res. 1999;26(3):329-44.
78. Veerman JW, De Kemp RAT, Ten Brink LT, Slot NW, Scholte EM. The implementation of families first in the Netherlands: a one year follow-up. Child Psychiatry Hum Dev. 2003;33(3):227-44. doi: 10.1023/a:1021456630670.
79. von dem Knesebeck O, Joksimovic L, Badura B, Siegrist J. Evaluation of a community-level health policy intervention. Health Policy. 2002;61(1):111-22.
80. Walker JS, Matarese M. Using a theory of change to drive human resource development for wraparound. J Child Family Studies. 2011;20(6):791-803. doi: 10.1007/s10826-011-9532-6.
81. Weitzman BC, Mijanovich T, Silver D, Brecher C. Finding the impact in a messy intervention: using an integrated design to evaluate a comprehensive citywide health initiative. Am J Eval. 2009;30(4):495-514. doi: 10.1177/1098214009347555.
82. Wenz-Gross W, DuBrino T. Central Massachusetts Communities of Care: applying a theory of change approach to support implementation, evaluation, and strategic planning. Tools for Creating and Sustaining Human Services: Theory of Change Logic Models: University of South Florida 2012.
83. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6, e1000097. doi: 10.1371/journal.pmed.1000097.
84. Clarke DJ, Godfrey M, Hawkins R, Sadler E, Harding G, Forster A, et al. Implementing a training intervention to support caregivers after stroke: a process evaluation examining the initiation and embedding of programme change. Implementation Sci. 2013;8(1):96. doi: 10.1186/1748-5908-8-96.
85. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8(1):18.
86. Vandenbroucke JP, Von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Ann Intern Med. 2007;147(8):W-163-94.
87. Möhler R, Köpke S, Meyer G. Criteria for Reporting the Development and Evaluation of Complex Interventions in healthcare: revised guideline (CReDECI 2). Trials. 2015;16(1):204.