Qualitative comparative analysis as a method for evaluating complex cases: An overview of literature and a stepwise guide with empirical application;Qualitative Comparative Analysis als Verfahren zur Bewertung komplexer Fälle.: Eine Literaturübersicht und ein schrittweiser Leitfaden mit empirischen Beispielen

Zeitschrift fur Evaluation

Volumn 15, Issue 1, 2016, Pages 7-22

  Gerrits, Lasse M ^a   Verweij, Stefan ^b  

^a UNIVERSITY OF BAMBERG   (Germany)

^b ERASMUS UNIVERSITY ROTTERDAM   (Netherlands)

Author keywords

Complexity; Confi gurational method; Evaluation; Qualitative comparative analysis

Indexed keywords

EID: 84963997538     PISSN: 16195515     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (103)

1. Amenta, Edwin/Poulsen, Jane D. (1994): Where to Begin: A Survey of Five Approaches to Selecting Independent Variables for Qualitative Comparative Analysis. In: Sociological Methods and Research, 23 (1), pp. 22-53.
2. Balthasar, Andreas (2006): The Effects of Institutional Design on the Utilization of Evaluation: Evidence Using Qualitative Comparative Analysis (QCA). In: Evaluation, 12 (3), pp. 353-371.
3. Bamberger, Michael (Ed.) (2000): Directions in Development: Integrating Quantitative and Qualitative Research in Development Projects. Washington: The World Bank.
4. Bamberger, Michael/Rao, Vijayendra/Woolcock, Michael (2010): Using Mixed Methods in Monitoring and Evaluation: Experiences from International Development. Washington: The World Bank.
5. Barnes, Marian/Matka, Elizabeth/Sullivan, Helen (2003): Evidence, Understanding and Complexity: Evaluation in Non-Linear Systems. In: Evaluation, 9 (3), pp. 265-284.
6. Befani, Barbara (2013): Between Complexity and Generalization: Addressing Evaluation Challenges with QCA. In: Evaluation, 19 (3), pp. 269-283.
7. Befani, Barbara/Ledermann, Simone/Sager, Fritz (2007): Realistic Evaluation and QCA: Conceptual Parallels and an Empirical Application. In: Evaluation, 13 (2), pp. 171-192.
8. Befani, Barabara/Sager, Fritz (2006): QCA as a Tool for Realistic Evaluations: The Case of the Swiss Environmental Impact Assessment. In: Rihoux, Benoit/Grimm, Heike (Eds.): Innovative Comparative Methods for Policy Analysis: Beyond the Quantitative-Qualitative Divide. New York: Springer, pp. 263-284.
9. Blackman, Tim/Wistow, Jonathan/Byrne, Dave S. (2013): Using Qualitative Comparative Analysis to Understand Complex Policy Problems. In: Evaluation, 19 (2), pp. 126-140.
10. Bressers, Nanny E. W./Gerrits, Lasse M. (2013): A Complexity-Informed Approach to Evaluating National Knowledge and Innovation Programmes. In: Systems Research and Behavioral Science, 32 (1), pp. 50-63.
11. Burton, Paul/Goodlad, Robina/Croft, Jacqui (2006): How Would We Know What Works? Context and Complexity in the Evaluation of Community Involvement. In: Evaluation, 12 (3), pp. 294-312.
12. Byrne, David S. (1998): Complexity Theory and the Social Sciences: An Introduction. New York: Routledge.
13. Byrne, David S. (2004): Complexity Theory and Social Research. In: Social Research Update. Verfügbar unter: http://sru.soc.surrey.ac.uk/SRU18.html [17.02.2016].
14. Byrne, David S. (2005): Complexity, Confi gurations and Cases. In: Theory, Culture & Society, 22 (5), pp. 95-111.
15. Byrne, David S. (2009): Complex Realist and Confi gurational Approaches to Cases: A Radical Synthesis. In: David S. Byrne/Charles C. Ragin (Eds.): The Sage Handbook of Case-Based Methods. London: Sage, pp. 101-119.
16. Byrne, David S. (2011a): Applying Social Science: The Role of Social Research in Politics, Policy and Practice. Bristol: The Policy Press.
17. Byrne, David S. (2011b): Exploring Organizational Effectiveness: The Value of Complex Realism as a Frame of Reference and Systematic Comparison as a Method. In: Allen, Peter/Maguire, Steve/McKelvey, Bill (Eds.): The Sage Handbook of Complexity and Management. London: Sage, pp. 131-141.
18. Byrne, David S. (2013): Evaluating Complex Social Interventions in a Complex World. In: Evaluation, 19 (3), pp. 217-228.
19. Byrne, David S./Callaghan, Gillian (2013): Complexity Theory and the Social Sciences: The State of the Art. Abingdon: Routledge.
20. Cabrera, Derek/Colosi, Laura (2008): Distinctions, Systems, Relationships, and Perspectives (DSRP): A Theory of Thinking and of Things. In: Evaluation and Program Planning, 31 (3), pp. 311-317.
21. Cabrera, Derek/Colosi, Laura/Lobdell, Claire (2008): Systems Thinking. In: Evaluation and Program Planning, 31 (3), pp. 299-310.
22. Caren, Neal/Panofsky, Aaaron (2005): TQCA: A Technique for Adding Temporality to Qualitative Comparative Analysis. In: Sociological Methods and Research, 34 (2), pp. 147-172.
23. Checkland, Peter (1981): Systems Thinking, Systems Practice. Chichester: John Wiley & Sons.
24. Cilliers, Paul (1998): Complexity and Postmodernism: Understanding Complex Systems. New York: Routledge.
25. Cilliers, Paul (2001): Boundaries, Hierarchies and Networks in Complex Systems. In: International Journal of Innovation Management, 5 (2), pp. 135-147.
26. Cilliers, Paul (2002): Why We Cannot Know Complex Things Completely. In: Emergence, 4 (1-2), pp. 77-84.
27. Cilliers, Paul (2005): Knowledge, Limits and Boundaries. In: Futures, 37 (7), pp. 605-613.
28. Da Costa, António Firminio/Pegado, Elsa/Ávila, Patrícia/Coelho, Ana R. (2013): Mixed-Methods Evaluation in Complex Programmes: The National Reading Plan in Portugal. In: Evaluation and Program Planning, 39, pp. 1-9.
29. Dyer, Wendy (2011): Mapping Pathways. In: Williams, Malcolm/Vogt, Paul (Eds.): The SAGE Handbook of Innovation in Social Research Methods. New York: Sage Publications.
30. Elder-Vass, Dave (2005): Emergence and the Realist Account of Cause. In: Journal of Critical Realism, 4 (2), pp. 315-338.
31. Flood, Robert L. (1999): Rethinking the Fifth Discipline: Learning within the Unknowable. London: Routledge.
32. Forss, Kim/Marra, Mita/Schwartz, Robert (Eds.) (2011): Evaluating the Complex: Attribution, Contribution, and beyond. New Brunswick: Transaction Publishers.
33. Gell-Mann, Murray (1995): What is Complexity? Remarks on Simplicity and Complexity by the Nobel Prize-Winning Author of the Quark and the Jaguar. In: Complexity, 1 (1), pp. 16-19.
34. Gerrits, Lasse M. (2012): Punching Clouds: An Introduction to the Complexity of Public Decision-Making. Litchfi eld Park: Emergent Publications.
35. Gerrits, Lasse M./Verweij, Stefan (2013): Critical Realism as a Meta-Framework for Understanding the Relationships Between Complexity and Qualitative Comparative Analysis. In: Journal of Critical Realism, 12 (2), pp. 166-182.
36. Glouberman, Sholom/Zimmerman, Brenda (2002): Complicated and Complex Systems: What Would Successful Reform of Medicare Look Like?. Toronto: Commission on the Future of Health Care in Canada.
37. Goldstein, Jeffrey (1999): Emergence as a Construct: History and Issues. In: Emergence, 1 (1), pp. 49-72.
38. Hak, Tony/Jaspers, Ferdinand/Dul, Jan (2013): The Analysis of Temporally Ordered Confi gurations: Challenges and Solutions. In: Fiss, Peer C./Cambré, Bart/Marx, Axel (Eds.): Confi gurational Theory and Methods in Organizational Research. Bingley: Emerald, pp. 107-127.
39. Harvey, David L. (2009): Complexity and Case. In: Byrne, David S./Ragin, Charles C. (Eds.): The Sage Handbook of Case-Based Methods. London: Sage, pp. 15-38.
40. Hino, Airo (2009): Time-Series QCA: Studying Temporal Change through Boolean Analysis. In: Sociological Theory and Methods, 24 (2), pp. 247-265.
41. Holland, John H. (1995): Hidden Order: How Adaptation Builds Complexity. New York: Basic Books.
42. Holland, John H. (2006): Studying Complex Adaptive Systems. In: Journal of Systems Science and Complexity, 19 (1), pp. 1-8.
43. Holvoet, Nathalie/Dewachter, Sara (2013): Multiple Paths to Effective National Evaluation Societies: Evidence from 37 Low-and Middle-Income Countries. In: American Journal of Evaluation, 34 (4), pp. 519-544.
44. Jackson, Suzanne F./Kolla, Gillian (2012): A new Realistic Evaluation Analysis Method: Linking Coding of Context, Mechanism, and Outcome Relationships. In: American Journal of Evaluation, 33 (3), pp. 339-349.
45. Kiel, L. Douglas (1989): Nonequilibrium Theory and its Implications for Public Administration. In: Public Administration Review, 49 (6), pp. 544-551.
46. Kiel, L. Douglas/Elliott, Euel (Eds.) (1996): Chaos Theory in the Social Sciences: Foundations and Applications. Ann Arbor: The University of Michigan Press.
47. Laszlo, Ervin (1972): The Systems View of the World: The Natural Philosophy of the New Developments in the Sciences. New York: George Braziller.
48. Ledermann, Simone (2012): Exploring the Necessary Conditions for Evaluation Use in Program Change. In: American Journal of Evaluation, 33 (2), pp. 159-178.
49. Leeuw, Frans L. (2003): Reconstructing Program Theories: Methods Available and Problems to Be Solved. In: American Journal of Evaluation, 24 (1), pp. 5-20.
50. Marion, Russ (1999): The Edge of Organization: Chaos and Complexity Theories of Formal Social Systems. Thousand Oaks: Sage.
51. Marx, Axel (2005): Systematisch comparatief case onderzoek van evaluatieonderzoek. In: Tijdschrift Voor Sociologie, 26 (1), pp. 95-113.
52. Mathews, K. Micheal/White, Michael C./Long, Rebecca G. (1999): Why Study the Complexity Sciences in the Social Sciences?. In: Human Relations, 52 (4), pp. 439-462.
53. Meadows, Donella H. (2008): Thinking in Systems: A Primer. White River Junction: Chelsea Green.
54. Medd, Will (2001): Making (Dis)Connections: Complexity and the Policy Process. In: Social Issues, 1 (2).
55. Midgley, Gerald/Munlo, Isaac/Brown, Mandy (1998): The Theory and Practice of Boundary Critique: Developing Housing Services for Older People. In: The Journal of the Operational Research Society, 49 (5), pp. 467-478.
56. Mitchell, Sandra (2009): Complexity and Explanation in the Social Sciences. In: Mantzavinos, Chrysostomos (Ed.): Philosophy of the Social Sciences: Philosophical Theory and Scientifi c Practice. Cambridge: Cambridge University Press, pp. 130-145.
57. Mowles, Chris (2014): Complex, but Not Quite Complex Enough: The Turn to the Complexity Sciences in Evaluation Scholarship. In: Evaluation, 20 (2), pp. 160-175.
58. Page, Scott E. (2008): Uncertainty, Diffi culty, and Complexity. In: Journal of Theoretical Politics, 20 (2), pp. 115-149.
59. Parsons, Talcott (1951): The Social System. New York: The Free Press.
60. Patton, Michael Q. (2011): Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: The Guilford Press.
61. Pattyn, Valérie/Brans, Marleen (2013): Outsource versus In-House? An Identifi cation of Organizational Conditions Infl uencing the Choice for Internal or External Evaluators. In: The Canadian Journal of Program Evaluation, 28 (2), pp. 43-63.
62. Pawson, Ray/Tilley, Nick (1997): Realistic Evaluation. London: Sage.
63. Prigogine, Ilya (1997): The End of Certainty: Time, Chaos, and the New Laws of Nature. New York: The Free Press.
64. Ragin, Charles C. (1987): The Comparative Method: Moving beyond Qualitative and Quantitative Strategies. Los Angeles: University of California Press.
65. Ragin, Charles C. (2000): Fuzzy-Set Social Science. Chicago: University of Chicago Press.
66. Ragin, Charles C. (2006): Set Relations in Social Research: Evaluating their Consistency and Coverage. In: Political Analysis, 14 (3), pp. 291-310.
67. Ragin, Charles C. (2008): Redesigning Social Inquiry: Fuzzy Sets and beyond. Chicago: University of Chicago Press.
68. Ragin, Charles C./Becker, Howard S. (Eds.) (1992): What is a Case? Exploring the Foundations of Social Inquiry. Cambridge: Cambridge University Press.
69. Ragin, Charles C./Shulman, David/Weinberg, Adam/Gran, Brian (2003): Complexity, Generality, and Qualitative Comparative Analysis. In: Field Methods, 15 (4), pp. 323-340.
70. Ragin, Charles C./Strand, Sarah I. (2008): Using Qualitative Comparative Analysis to Study Causal Order: Comment on Caren and Panofsky (2005). In: Sociological Methods and Research, 36 (4), pp. 431-441.
71. Rantala, Kati/Hellström, Eeva (2001): Qualitative Comparative Analysis and a Hermeneutic Approach to Interview Data. In: International Journal of Social Research Methodology, 4 (2), pp. 87-100.
72. Reed, Michael/Harvey, David L. (1992): The New Science and the Old: Complexity and Realism in the Social Sciences. In: Journal for the Theory of Social Behaviour, 22 (4), pp. 353-380.
73. Rescher, Nicholas (1998): Complexity: A Philosophical Overview. New Brunswick: Transaction Publishers.
74. Richardson, Kurt A./Lissack, Michael R. (2001): On the Status of Boundaries, Both Natural and Organizational: A Complex Systems Perspective. In: Emergence, 3 (4), pp. 32-49.
75. Rihoux, Benoît/Álamos-Concha, Priscilla/Bol, Damien/Marx, Axel/Rezsöhazy, Ilona (2013): From Niche to Mainstream Method? A Comprehensive Mapping of QCA Applications in Journal Articles from 1984 to 2011. In: Political Research Quarterly, 66 (1), pp. 175-184.
76. Rihoux, Benoît/Ragin, Charles C. (Eds.) (2009): Confi gurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques. London: Sage.
77. Rihoux, Benoît/Rezsöhazy, Ilona/Bol, Damien (2011): Qualitative Comparative Analysis (QCA) in Public Policy Analysis: An Extensive Review. In: German Policy Studies, 7 (3), pp. 9-82.
78. Rogers, Patricia J. (2008): Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions. In: Evaluation, 14 (1), pp. 29-48.
79. Rogers, Patricia J. (2011): Implications of Complicated and Complex Characteristics for Key Tasks in Evaluation. In: Forss, Kim/Marra, Mita/Schwartz, Robert (Eds.): Evaluating the Complex: Attribution, Contribution, and Beyond. New Brunswick: Transaction Publishers, pp. 33-52.
80. Sager, Fritz/Andereggen, Céline (2012): Dealing with Complex Causality in Realist Synthesis: The Promise of Qualitative Comparative Analysis. In: American Journal of Evaluation, 33 (1), pp. 60-78.
81. Sanderson, Ian (2000): Evaluation in Complex Policy Systems. In: Evaluation, 6 (4), pp. 433-454.
82. Sanderson, Ian (2002): Evaluation, Policy Learning and Evidence-Based Policy Making. In: Public Administration, 80 (1), pp. 1-22.
83. Schelling, Thomas C. (1978): Micromotives and Macrobehavior. New York: W.W. Norton & Company.
84. Schneider, Carsten Q./Wagemann, Claudius (2010): Standards of Good Practice in Qualitative Comparative Analysis (QCA) and Fuzzy Sets. In: Comparative Sociology, 9 (3), pp. 397-418.
85. Schneider, Claudius Q./Wagemann, Carsten (2012): Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis. Cambridge: Cambridge University Press.
86. Sibeon, Roger (1999): Anti-Reductionist Sociology. In: Sociology, 33 (2), pp. 317-334.
87. Smith, Thomas S./Stevens, Gregory T. (1996): Emergence, Self-Organisation, and Social Interaction: Arousal Dependent Structure in Social Systems. In: Sociological Theory, 14 (2), pp. 131-153.
88. Stame, Nicoletta (2004): Theory-Based Evaluation and Types of Complexity. In: Evaluation, 10 (1), pp. 58-76.
89. Stern, Elliot (2014): Editorial. In: Evaluation, 20 (2), pp. 157-159.
90. Stern, Elliot et al. (2012): Broadening the Range of Designs and Methods for Impact Evaluations: Report of a Study Commissioned by the Department for International Development. London: Department for International Development.
91. Thiem, Alrik (2013): Clearly Crisp, and Not Fuzzy: A Reassessment of the (Putative) Pitfalls of Multi-Value QCA. In: Field Methods, 25 (2), pp. 197-207.
92. Ton, Giel (2012): The Mixing of Methods: A Three-Step Process for Improving Rigour in Impact Evaluations. In: Evaluation, 18 (1), pp. 5-25.
93. Uprichard, Emma/Byrne, David S. (2006): Representing Complex Places: A Narrative Approach. In: Environment and Planning A, 38 (4), pp. 665-676.
94. Verweij, Stefan/Gerrits, Lasse M. (2012): Assessing the Applicability of Qualitative Comparative Analysis for the Evaluation of Complex Projects. In: Lasse M. Gerrits/Peter K. Marks (Eds.): Compact 1: Public Administration in Complexity. Litchfi eld Park: Emergent Publications, pp. 93-117.
95. Verweij, Stefan/Gerrits, Lasse M. (2013): Understanding and Researching Complexity with Qualitative Comparative Analysis: Evaluating Transportation Infrastructure Projects. In: Evaluation, 19 (1), pp. 40-55.
96. Verweij, S./Gerrits, L.M. (2015): How Satisfaction is Achieved in the Implementation Phase of Large Transportation Infrastructure Projects: A Qualitative Comparative Analysis into the A2 Tunnel Project. In: Public Works Management & Policy, 20 (1), pp. 5-28.
97. Vink, Maarten P./Van Vliet, Olaf (2009): Not Quite Crisp, Not yet Fuzzy? Assessing the Potentials and Pitfalls of Multi-Value QCA. In: Field Methods, 21 (3), pp. 265-289.
98. Vink, Maarten P./Van Vliet, Olaf (2013): Potentials and Pitfalls of Multi-Value QCA: Response to Thiem. In: Field Methods, 25 (2), pp. 208-213.
99. Wagenaar, Hendrik (2007): Governance, Complexity, and Democratic Participation: How Citizens and Public Offi cials Harness the Complexities of Neighborhood Decline. In: The American Review of Public Administration, 37 (1), pp. 17-50.
100. Walton, Mat (2014): Applying Complexity Theory: A Review to Inform Evaluation Design. In: Evaluation and Program Planning, 45, pp. 119-126.
101. Westhorp, Gill (2012): Using Complexity-Consistent Theory for Evaluating Complex Systems. In: Evaluation, 18 (4), pp. 405-420.
102. Wolf-Branigin, Michael (2013): Using Complexity Theory for Research and Program Evaluation. Oxford: Oxford University Press.
103. Yin, Robert K. (2013): Validity and Generalization in Future Case Study Evaluations. In: Evaluation, 19 (3), pp. 321-332.