Linking theory-based evaluation and contribution analysis: Three problems and a few solutions

Evaluation

Volumn 18, Issue 3, 2012, Pages 348-363

  Leeuw, Frans L ^a  

^a MAASTRICHT UNIVERSITY   (Netherlands)

Author keywords

contribution analysis; counterfactual history; impact evaluation; implementation science; mechanisms; theory based evaluation

Indexed keywords

EID: 84863663700     PISSN: 13563890     EISSN: 14617153     Source Type: Journal    
DOI: 10.1177/1356389012452051     Document Type: Article

References (58)

1. Argyris C,Schön DAOrganizational Learning: A Theory of Action Perspective. Reading, MA: Addison-Wesley; 1978:.
2. Astbury B,Leeuw FL.Unpacking black boxes: mechanisms and theory-building in evaluation.American Journal of Evaluation. 2010;31 (3): 363-81.
3. Barnoski ROutcome Evaluation of Washington State's Research-Based Programs for Juvenile Offenders. Olympia, WA: Washington State Institute for Public Policy; 2004:.
4. Carroll C,Patterson M,Wood S,Booth A,Rick J,Balain S.A conceptual framework for implementation fidelity.Implementation Science. 2007;2 (1): 40.
5. Chen HT,Rossi PH.The multi-goal, theory-driven approach to evaluation: a model linking basic and applied social science.Social Forces. 1980;59:106-22.
6. Coleman JFoundations of Social Theory. Cambridge, MA: Belknap Press; 1990:.
7. Coryn C,Noakes LA,Westine CD,Schröter DC.A systematic review of theory-driven evaluation practice from 1990 to 2009.American Journal of Evaluation. 2010;32 (1): 1-28.
8. Dolan P,Hallsworth M,Halpern D,King D,Vlaev ILondon: Cabinet Office and Institute for Government; 2010:.
9. Donaldson SIProgram Theory-Driven Evaluation Science. New York: Lawrence; 2007:.
10. Durlak JA,DuPre E.Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation.American Journal of Community Psychology. 2008;41:327-50.
11. Managerial and Organizational Cognition: Theory, Methods and Research. Eden CSpender J-C, ed. London: SAGE; 1998:.
12. Elster JNuts and Bolts for the Social Sciences. Cambridge: Cambridge University Press; 1989:.
13. Elster JExplaining Social Behaviour: More Nuts and Bolts for the Social Sciences. Cambridge: Cambridge University Press; 2007:.
14. Festinger LA Theory of Cognitive Dissonance. Stanford, CA: Stanford University Press; 1957:.
15. Fogel RRailroads and American Economic Growth: Essays in Econometric History. Baltimore, MD: Johns Hopkins University Press; 1964:.
16. Foy R,Ovretveit J,Shekelle PG,Pronovost PJ,Taylor SL,Dy S,Hempel S,McDonald KM,Rubenstein LV,Wachter RM.The role of theory in research to develop and evaluate the implementation of patient safety practices.BMJ Quality & Safety. 2011;20 (5): 453-9.
17. Funnell S,Rogers PPurposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco, CA: Jossey-Bass; 2011:.
18. Greenhalgh T,Robert G,MacFarlane F,Bate P,Kyriakidou O.Diffusion of innovations in service organizations: systematic review and recommendations.The Milbank Quarterly. 2004;82 (4): 581-629.
19. Hansen MB,Vedung E.Theory-based stakeholder evaluation.American Journal of Evaluation. 2010;31 (3): 295-313.
20. Harmelink M,Nilsson L,Harmsen R.Theory-based policy evaluation of 20 energy efficiency instruments.Energy Efficiency. 2008;1:131-48.
21. Hedström PDissecting the Social. On the Principles of Analytical Sociology. Cambridge: Cambridge University Press; 2005:.
22. Hedström P,Swedberg RSocial Mechanisms. An Analytical Approach to Social Theory. Cambridge: Cambridge University Press; 1998:.
23. Kautto P,Similä J.Recently introduced policy instruments and intervention theories.Evaluation. 2005;11 (1): 55-68.
24. Keizer K,Lindenberg S,Steg L.The spreading of disorder.Science. 2008;322:1681-5.
25. Kruisbergen EW.Voorlichting: doen of laten? Theorie van afschrikwekkende voorlichtingscampagnes toegepast op de casus van bolletjesslikkers.Beleidswetenschap. 2007;19 (3): 38-51.
26. Leeuw FL.Policy theories, knowledge utilization, and evaluation.Knowledge and Policy. 1991;4:73-92.
27. Leeuw FL.Reconstructing program theories: methods available and problems to be solved.American Journal of Evaluation. 2003;24 (1): 5-20.
28. Leeuw FLReport on Theory-Based Evaluation (Tbe), Prepared for the Evalsed Handbook on Impact Evaluation. Brussels: DG Regional Policy.
29. Leeuw FL,Leeuw B.Cyber society and digital policies: challenges to evaluation?.Evaluation. 2012;18 (1): 111-27.
30. Leeuw FL,Vaessen JImpact Evaluation and Development. Washington, DC: NONIE & World Bank; 2009:.
31. Lipsey MW.The primary factors that characterize effective interventions with juvenile offenders: a meta-analytic overview.Victims and Offenders. 2009;4:124-47.
32. Ludwig J,Kling JR,Mullainathan S.Mechanism experiments and policy evaluations.Journal of Economic Perspectives. 2011;25 (3): 17-38.
33. McLaughlin JA,Jordan GB.Logic models: a tool for telling your program's performance story.Evaluation and Program Planning. 2004;22:65-72.
34. Mannheim KMan and Society in an Age of Reconstruction: Studies in Modern Social Structure. New York: Harcourt, Brace & World; 1967 [1935]:.
35. Mason I,Mitroff IChallenging Strategic Planning Assumptions. New York: Wiley; 1981:.
36. Mayne JEvaluating the Complex. Schwartz RForss KMarra M, ed. New Brunswick, NJ: Transaction Publishers; 2011:53-96.
37. Nas C,Van Ooyen-Houben M,Wieman JInterventies in uitvoering: wat er mis kan gaan bij de uitvoering van justitiële (gedrags)interventies en hoe dat komt. Memorandum 2011-12. The Hague: WODC; 2011:.
38. Pawson R.Evidence-based policy: the promise of 'realist synthesis'.Evaluation. 2002a;8 (3): 340-58.
39. Pawson R.Evidence and policy and naming and shaming.Policy Studies. 2002b;23 (3): 211-30.
40. Pawson REvidence Based Policy: A Realist Perspective. London: SAGE; 2006:.
41. Pawson R,Tilley NRealistic Evaluation. Thousand Oaks, CA: SAGE; 1997:.
42. Pawson R,Sridharan SEvidence-Based Public Health Effectiveness and Efficiency. Killoran AKelly M, ed. Oxford: Oxford University Press; 2010:42-62.
43. Pawson R,Wong G,Owen L.Evidence-based policy known knowns, known unknowns, unknown unknowns: the predicament of of evidence-based policy.American Journal of Evaluation. 2011;32 (4): 518-46.
44. Popper KRConjectures and Refutations: The Growth of Scientific Knowledge. London: Routledge; 1963:.
45. Rozendal P,Moors H,Leeuw FHet bevolkingsvraagstuk in de jaren 80; opvattingen over overheidsbeleid. The Hague: NIDI; 1985:.
46. Suchman EEvaluative Research. New York: Russell Sage Foundation; 1967:.
47. Tetlock PE,Belkin ACounterfactual Reasoning, Counterfactual Thought Experiments in Global Politics: Logical, Methodological and Psychological Perspectives. Tetlock PBelkin A, ed. Princeton, NJ: Princeton University Press; 1996:3-38.
48. Thompson VD,Appelbaum MPopulation Policy Acceptance: Psychological Determinants. Chapel Hill, NC: Carolina Population Center Monograph Series; 1974:.
49. Prospective Evaluations Methods: The Prospective Evaluation Synthesis. Washington, DC: US General Accounting Office; 1995:.
50. Twardy CR.Argument maps improve critical thinking.Teaching Philosophy. 2004;27 (2): 95-116.
51. Van den BraakS (2010) Sensemaking software. Unpublished PhD thesis, Utrecht University.
52. Van den Braak SW,Van Oostendorp H,Prakken H,Vreeswijk GAW.A critical review of argument visualization tools: do users become better reasoners?. Workshop Notes of the ECAI-2006 Workshop on Computational Models of Natural Argument (CMNA VI); 2006Riva del Garda, Italy; 2006. 67-75.
53. Verheij BVirtual Arguments. On the Design of Argument Assistants for Lawyers and Other Arguers. The Hague: TMC Asser Press; 2005:.
54. Weick KE,Bougon MGThe Thinking Organization: Dynamics of Organizational and Social Cognition. Sims HPGioia DA, ed. San Francisco, CA: Jossey-Bass; 1986:.
55. Weiss CHNew Approaches to Evaluating Community Initiatives: Volume 1, Concepts, Methods, and Contexts. Connell JKubisch ASchorr LBWeiss CH, ed. New York: Aspen Institute; 1995:65-92.
56. Weiss CH.Theory-based evaluation: Past, present and future.New Directions for Evaluation. 1997;76:41-55.
57. Wholey JEvaluation Research Methods. Rutman L, ed. Beverly Hills, CA: SAGE; 1977:.
58. Zumbach J.The role of graphical and text based argumentation tools in hypermedia learning.Computers in Human Behavior. 2009;25 (4): 811-17.