Applying complexity theory: A review to inform evaluation design

Evaluation and Program Planning

Volumn 45, Issue , 2014, Pages 119-126

  Walton, Mat ^a  

^a MASSEY UNIVERSITY   (New Zealand)

Author keywords

Complexity theory; Evaluation design; Methods

Indexed keywords

COMPLEXITY; DESIGN; PLANNING METHOD; THEORETICAL STUDY;

HUMAN; METHODOLOGY; PROCEDURES; PROGRAM EVALUATION; SYSTEMS THEORY;

HUMANS; PROGRAM EVALUATION; RESEARCH DESIGN; SYSTEMS THEORY;

EID: 84899841727     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.evalprogplan.2014.04.002     Document Type: Article

References (78)

1. Adam T., Hsu J., De Savigny D., Lavis J.N., Rottingen J.A., Bennett S. Evaluating health systems strengthening interventions in low-income and middle-income countries: Are we asking the right questions?. Health Policy and Planning 2012, 27(Suppl. 4):iv9-iv19.
2. Anderson R.A., Crabtree B.F., Steele D.J., McDaniel R.R. Case study research: The view from complexity science. Qualitative Health Research 2005, 15(5):669-685. 10.1177/1049732305275208.
3. Barnes M., Matka E., Sullivan H. Evidence understanding and complexity: Evaluation in non-linear systems. Evaluation 2003, 9(3):265-284.
4. Blackman T., Hunter D., Marks L., Harrington B., Elliott E., Williams G., et al. Wicked comparisons: Reflections on cross-national research about health inequalities in the UK. Evaluation 2010, 16(1):43-57.
5. Blackman T., Wistow J., Byrne D. A qualitative comparative analysis of factors associated with trends in narrowing health inequalities in England. Social Science and Medicine 2011, 72(12):1965-1974.
6. Blackman T., Wistow J., Byrne D. Using qualitative comparative analysis to understand complex policy problems. Evaluation 2013, 19(2):126-140.
7. Boustani M.A., Munger S., Gulati R., Vogel M., Beck R.A., Callahan C.M. Selecting a change and evaluating its impact on the performance of a complex adaptive health care delivery system. Clinical Interventions in Aging 2010, 5:141-148.
8. Bristow G., Farrington J., Shaw J., Richardson T. Developing an evaluation framework for crosscutting policy goals: The accessibility policy assessment tool. Environment and Planning A 2009, 41(1):48-62.
9. Brousselle A., Lessard C. Economic evaluation to inform health care decision-making: Promise, pitfalls and a proposal for an alternative path. Social Science and Medicine 2011, 72(6):832-839.
10. Buijs J.-M., Eshuis J., Byrne D. Approaches to researching complexity in public management. Managing complex governance systems 2009, 37-55. Routledge, New York. G.A. Teisman, V. Buuren, L. Gerrits (Eds.).
11. Burgoyne J.G. Evaluating action learning: A critical realist complex network theory approach. Action Learning: Research and Practice 2010, 7(3):239-251.
12. Burns D. Evaluation in complex governance arenas: The potential of large system action research. Systems concepts in evaluation: An expert anthology 2006, 181-196. American Evaluation Association, Point Reyes, CA. B. Williams, I. Imam (Eds.).
13. Byrne D. Complexity configurations and cases. Theory Culture Society 2005, 22(5):95-111.
14. Byrne D. Applying social science: The role of social research in politics, policy and practice 2011, The Policy Press, Bristol.
15. Byrne D. Evaluating complex social interventions in a complex world. Evaluation 2013, 19(3):217-228. 10.1177/1356389013495617.
16. Byrne D., Callaghan G. Complexity theory and the social sciences: The state of the art 2014, Routledge, Oxon.
17. Cabrera D., Colosi L., Lobdell C. Systems thinking. Evaluation and Program Planning 2008, 31(3):299-310.
18. Callaghan G. Evaluation and negotiated order: Developing the application of complexity theory. Evaluation 2008, 14(4):399-411.
19. Capra F. Complexity and life. Theory Culture Society 2005, 22(5):33-44. 10.1177/0263276405057046.
20. Dattée B., Barlow J. Complexity and whole-system change programmes. Journal of Health Services Research and Policy 2010, 15(Suppl. 2):19-25.
21. Dickinson H. The evaluation of health and social care partnerships: An analysis of approaches and synthesis for the future. Health and Social Care in the Community 2006, 14(5):375-383.
22. Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: A review of possible methods. Journal of Health Services Research and Policy 2005, 10(1):45-53.
23. Dyson A., Todd L. Dealing with complexity: Theory of change evaluation and the full service extended schools initiative. International Journal of Research and Method in Education 2010, 33(2):119-134.
24. Eppel E., Matheson A., Walton M. Applying complexity theory to New Zealand public policy: Principles for practice. Policy Quarterly 2011, 7(1):48-55.
25. Fleischhacker S. Food fight: The battle over redefining competitive foods. The Journal of School Health 2007, 77(3):147.
26. Evaluating the complex: Attribution, contribution, and beyond 2011, Vol. 18. Transaction Publishers, New Brunswick.
27. Gare A. Systems theory and complexity introduction. Democracy and Nature 2000, 6(3):327-339.
28. Goldstein J. Emergence as a construct: History and issues. Emergence 1999, 1(1):49-72.
29. Hawe P., Bond L., Butler H. Knowledge theories can inform evaluation practice: What can a complexity lens add?. New Directions for Evaluation 2009, 2009(124):89-100. 10.1002/ev.316.
30. Hawe P., Shiell A., Riley T., Gold L. Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. Journal of Epidemiology and Community Health 2004, 58(9):788-793.
31. Haynes P. Complexity theory and evaluation in public management. Public Management Review 2008, 10(3):401-419.
32. Hoffer L.D., Bobashev G., Morris R.J. Researching a local heroin market as a complex adaptive system. American Journal of Community Psychology 2009, 44(3):273-286.
33. Israel N., Wolf-Branigin M. Nonlinearity in social service evaluation: A primer on agent-based modeling. Social Work Research 2011, 35(1):20-24.
34. Kania A., Patel A.B., Roy A., Yelland G.S., Nguyen D.T.K., Verhoef M.J. Capturing the complexity of evaluations of health promotion interventions: A scoping review. Canadian Journal of Program Evaluation 2013, 27(1):65-91.
35. Lessard C. Complexity and reflexivity: Two important issues for economic evaluation in health care. Social Science and Medicine 2007, 64(8):1754-1765.
36. Levy D.T., Cho S.I., Kim Y.M., Park S., Suh M.K., Kam S. SimSmoke model evaluation of the effect of tobacco control policies in Korea: The unknown success story. American Journal of Public Health 2010, 100(7):1267-1273.
37. Maani K.E., Cavana R.Y. Systems thinking and modelling: Understanding change and complexity 2000, Pearson Education New Zealand Limited, Auckland.
38. Marchal B., Van Belle S., De Brouwere V., Witter S. Studying complex interventions: Reflections from the FEMHealth project on evaluating fee exemption policies in West Africa and Morocco. BMC Health Services Research 2013, 469.
39. Mason P., Barnes M. Constructing theories of change: Methods and sources. Evaluation 2007, 13(2):151-170.
40. Matheson A., Dew K., Cumming J. Complexity, evaluation and the effectiveness of community-based interventions to reduce health inequalities. Health Promotion Journal of Australia 2009, 20(3):221-226.
41. Mays N., Pope C., Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of Health Services Research & Policy 2005, 10:S6.
42. McLean R.M., Hoek J.A., Buckley S., Croxson B., Cumming J., Ehau T.H., et al. "Healthy eating - Healthy action": Evaluating New Zealand's obesity prevention strategy. BMC Public Health 2009, 9. http://www.biomedcentral.com/1471-2458/9/452.
43. Midgley G. Systems thinking, complexity and the philosophy of science. E:CO Emergence: Complexity and Organization 2008, 10(4):55-73.
44. Morçöl G. A complexity theory for public policy 2012, Routledge, New York.
45. Morell J.A. Evaluation in the face of uncertainty: Anticipating surprise and responding to the inevitable 2010, Guildford Press, New York.
46. Morell J.A., Hilscher R., Magura S., Ford J. Integrating evaluation and agent-based modeling: Rationale and an example for adopting evidence-based practices. Journal of MultiDisciplinary Evaluation 2010, 6(14):32-57.
47. Munda G. Social multi-criteria evaluation: Methodological foundations and operational consequences. European Journal of Operational Research 2004, 158(3):662-677.
48. Nordtveit B.H. Development as a complex process of change: Conception and analysis of projects, programs and policies. International Journal of Educational Development 2010, 30(1):110-117.
49. Olney C.A. Using evaluation to adapt health information outreach to the complex environments of community-based organizations. Journal of the Medical Library Association 2005, 93(4 Suppl.):S57-S67.
50. O'Sullivan D. Complexity science and human geography. Transactions of the Institute of British Geographers 2004, 29(1):282-295.
51. Parsons B.A. The state of methods and tools for social systems change. American Journal of Community Psychology 2007, 39(3-4):405-409.
52. Patton M.Q. Developmental evaluation: Applying complexity concepts to enhance innovation and use 2011, The Guilford Press, New York.
53. Plsek P.E., Greenhalgh T. Complexity science: The challenge of complexity in health care. BMJ 2001, 323(7313):625-628. 10.1136/bmj.323.7313.625.
54. Radej B. Synthesis in policy impact assessment. Evaluation 2011, 17(2):133-150.
55. Rametsteiner E., Weiss G. Assessing policies from a systems perspecitve - Experiences with applied innovation systems analysis and implications for policy evaluation. Forest Policy and Economics 2006, 8(5):564-576.
56. Richardson K.A., Cilliers P. What Is complexity science? A view from different directions. Emergence 2001, 3(1):5-23.
57. Richardson K.A., Gregory W.J., Midgley G. Editorial introduction to the special double issue on complexity thinking and systems theory. Emergence Complexity & Organization 2007, 9(1/2):vi-viii.
58. Rickles D., Hawe P., Shiell A. A simple guide to chaos and complexity. Journal of Epidemiology & Community Health 2007, 61:933-937.
59. Rog D.J. When background becomes foreground: Toward context-sensitive evaluation practice. New Directions for Evaluation 2012, 2012(135):25-40.
60. Rogers P.J. Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation 2008, 14(1):29-48.
61. Room G. Complexity. Institutions and public policy 2011, Edward Elgar Publishing, Cheltenham.
62. Rothwell H., Shepherd M., Murphy S., Burgess S., Townsend N., Pimm C. Implementing a social-ecological model of health in Wales. Health Education 2010, 110(6):471-489.
63. Sanderson I. Evaluation in complex policy systems. Evaluation 2000, 6(4):433.
64. Sanderson I. Intelligent policy making for a complex world: Pragmatism evidence and learning. Political Studies 2009, 57(4):699-719.
65. Schensul J.J. Community, culture and sustainability in multilevel dynamic systems intervention science. American Journal of Community Psychology 2009, 43(3-4):241-256.
66. Schwartz R., Garcia J. Intervention Path Contribution Analysis (IPCA) for complex strategy evaluation: Evaluating the smoke-free Ontario strategy. Evaluating the complex: Attribution, contribution and beyond 2011, 187-207. Transaction Publishers, New Brunswick. K. Forss, M. Marra, R. Schwartz (Eds.).
67. Shiell A., Hawe P., Gold L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ 2008, 336(7656):1281-1283. 10.1136/bmj.39569.510521.AD.
68. Simpson K.M., Porter K., McConnell E.S., Colón-Emeric C., Daily K.A., Stalzer A., et al. Tool for evaluating research implementation challenges: A sense-making protocol for addressing implementation challenges in complex research settings. Implementation Science 2013, 8(1).
69. Ssengooba F., McPake B., Palmer N. Why performance-based contracting failed in Uganda - An "open-box" evaluation of a complex health system intervention. Social Science and Medicine 2012, 75(2):377-383.
70. Stern E., Stame N., Mayne J., Forss K., Davies R., Befani B. Broadening the range of designs and methods for impact evaluations 2012, (Working Paper No. 38), Department for International Development, London, Retrieved from http://www.dfid.gov.uk/Documents/publications1/design-method-impact-eval.pdf.
71. Stewart J., Ayres R. Systems theory and policy practice: An exploration. Policy Sciences 2001, 34:94-97.
72. Trenholm S., Ferlie E. Using complexity theory to analyse the organisational response to resurgent tuberculosis across London. Social Science and Medicine 2013, 93:229-237.
73. Verweij S., Gerrits L.M. Understanding and researching complexity with qualitative comparative analysis: Evaluating transportation infrastructure projects. Evaluation 2013, 19(1):40-55. 10.1177/1356389012470682.
74. Vincent R. Insights from complexity theory for evaluation of development action: Recognising the two faces of complexity 2012, PANOS/IKM Emergent Research Programme, London, Retrieved from http://wiki.ikmemergent.net/files/1203-IKM_Emergent_Working_Paper_14-Complexity_Theory-March_2012.pdf.
75. Walker R. Entropy and the evaluation of labour market interventions. Evaluation 2007, 13(2):193-219.
76. Walton M., Signal L., Thomson G. Public policy to promote healthy nutrition in schools: Views of policymakers. Health Education Journal 2013, 72(3):283-291. 10.1177/0017896912442950.
77. Weiss C.H. Evaluation 1998, Prentice Hall, Upper Saddle River. 2nd ed.
78. Westhorp G. Using complexity-consistent theory for evaluating complex systems. Evaluation 2012, 18(4):405-420. 10.1177/1356389012460963.