The mixing of methods: A three-step process for improving rigour in impact evaluations

Evaluation

Volumn 18, Issue 1, 2012, Pages 5-25

  Ton, Giel ^a  

^a WAGENINGEN UNIVERSITY   (Netherlands)

Author keywords

applied research; design; rural development; validity; value chain development

Indexed keywords

EID: 84856152249     PISSN: 13563890     EISSN: 14617153     Source Type: Journal    
DOI: 10.1177/1356389011431506     Document Type: Article

References (48)

1. Armytage L.Evaluating aid: an adolescent domain of practice.Evaluation. 2011;17 (3): 261-76.
2. Bamberger M,Rugh J.A framework for assessing the quality, conclusion validity and utility of evaluations: experience from international development and lessons for developed countries. Paper presented at the American Evaluation Association conference; 2008 Baltimore; 2008.
3. Bamberger M,Rao V,Woolcock MUsing Mixed Methods in Monitoring and Evaluation: Experiences from International Development. Washington, DC: World Bank; 2010:.
4. Bamberger M,Rugh J,Mabry LRealWorld Evaluation: Working under Budget, Time, Data, and Political Constraints. Thousand Oaks, CA: SAGE; 2006:.
5. Bhaskar RA Realist Theory of Science. Abingdon: Taylor & Francis; 2008 [1978]:.
6. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Brady HECollier D, ed. Lanham, MD: Rowman and Littlefield; 2004:.
7. Brady HE,Collier D,Seawright J.Toward a pluralistic vision of methodology.Political Analysis. 2006;14 (3): 353-68.
8. Byrne D,Ragin CThe SAGE Handbook of Case-Based Methods. London: SAGE; 2009:.
9. Campbell DT,Stanley JC,Gage NLExperimental and Quasi-Experimental Designs for Research. Chicago, IL: Rand McNally; 1963:.
10. Chen HTTheory-Driven Evaluations. Thousand Oaks, CA: SAGE; 1994:.
11. Chen HT.The bottom-up approach to integrative validity: a new perspective for program evaluation.Evaluation and Program Planning. 2010;33 (3): 205-14.
12. 1er Censo Nacional de Organizaciones Económicas Campesinas, Indígenas y Originarias. La Paz: CIOEC-Bolivia; 2009:.
13. The DCED Standard for Measuring Achievements in Private Sector Development: Control Points and Compliance Criteria. Version V. 2010;:.
14. Donaldson S,Christie C,Mark MWhat Counts as Credible Evidence in Applied Research and Evaluation Practice?. Thousand Oaks, CA: SAGE; 2008:.
15. Easton GRealist Perspectives on Management and Organisations. Ackroyd SFleetwood S, ed. London/New York: Routledge; 2000:.
16. Garbarino S HollandJ (2009) Quantitative and qualitative methods in impact evaluation and measuring results. Governance and Social Development Resource Centre Issues Paper. URL: http://www.gsdrc.org/docs/open/EIRS4.pdf.
17. Guijt I,Woodhill JManaging for Impact in Rural Development: A Guide for Project M&E. Rome: IFAD; 2002:.
18. Guijt I,Brouwers J,Kusters C,Prins E,Zeynalova B. Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity - Conference Report; 2011; 2011.
19. Heckman JJ,Smith JA.Assessing the case for social experiments.The Journal of Economic Perspectives. 1995;9 (2): 85-110.
20. PRISM Guidelines: A Toolbox for Developing and Implementing Poverty Reduction through Irrigation and Smallholder Markets (PRISM) Programs. Denver, CO: International Development Enterprises; 2005:.
21. A Path Out of Poverty: Connecting Dollar-a-Day Farmers to Affordable Small Plot Irrigation and Markets. Denver, CO: International Development Enterprises; 2006:.
22. Toolkit for Developing and Implementing Poverty Reduction through Irrigation and Smallholder Markets (PRISM) Programs. Denver, CO: International Development Enterprises; 2007:.
23. Khandker S,Koolwal B,Samad HHandbook on Impact Evaluation. Washington, DC: World Bank; 2009:.
24. Kusters CMaking Evaluations Matter: A Practical Guide for Evaluators. Wageningen: Centre for Development Innovation, Wageningen UR; 2011:.
25. Leeuw F,Vaessen JImpact Evaluation and Development. Washington, DC: Nonie/Worldbank; 2009:.
26. McKenzie D.Impact assessments in finance and private sector development: what have we learned and what should we learn?.World Bank Research Observer. 2010;25 (2): 209-33.
27. Maredia MKImproving the Proof: Evolution Of and Emerging Trends in Impact Assessment Methods and Approaches in Agricultural Development. Washington, DC: IFPRI; 2009:.
28. Mayne J.Addressing attribution through contribution analysis: using performance measures sensibly.Canadian Journal of Program Evaluation. 2001;16 (1): 1-24.
29. Morell JEvaluation in the Face of Uncertainty: Anticipating Surprise and Responding to the Inevitable. New York: The Guilford Press; 2010:.
30. The DAC Guidelines: Poverty Reduction. Paris: OECD; 2001:.
31. Evaluating Development Co-operation: Summary of Key Norms and Standards. Paris: OECD; 2008.
32. Patton MQUtilization-Focused Evaluation. 2008) Utilization-Focused Evaluation, 4th edn.Thousand Oaks, CA: SAGE.Thousand Oaks, CA: SAGE; 2008:.
33. Pawson REvidence-Based Policy: A Realist Perspective. London: SAGE; 2006:.
34. Pawson R,Tilley NRealistic Evaluation. London: SAGE; 1997:.
35. Ravallion M.Should the randomistas rule?.The Economists' Voice. 2009;6 (2):.
36. Rihoux B,Ragin CConfigurational Comparative Methods. Qualitative Comparative Analysis (QCA) and related techniques (Applied Social Research Methods). Thousand Oaks, CA: SAGE; 2008:.
37. Rogers PJ.Using programme theory to evaluate complicated and complex aspects of interventions.Evaluation. 2008;14 (1): 29-48.
38. Rogers PJ.Matching impact evaluation design to the nature of the intervention and the purpose of the evaluation.Journal of Development Effectiveness. 2009;1 (3): 217-26.
39. Sayer RAMethod in Social Science: A Realist Approach. London: Routledge; 1992:.
40. Sayer RARealism and Social Science. London: SAGE; 2000:.
41. Shadish WR,Cook TD,Campbell DTExperimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin Co; 2002:.
42. Stame N.Theory-based evaluation and types of complexity.Evaluation. 2004;10 (1): 58-76.
43. Ton G.Resolving the challenges of collective marketing: incentive structures that reduce the tensions between members and their group.ESFIM Policy Brief #4. 2010;:.
44. Ton G,Van der Mheen JContract Farming Checklist: A Tool for Reflection on Critical Issues in Contract Farming Arrangements in Developing Countries. Den Haag: EVD / LEI-WUR / ICCO; 2009:.
45. Ton G,Vellema S,De Ruyter De Wildt M.Development impacts of value chain interventions: how to collect credible evidence and draw valid conclusions in impact evaluations?.Journal on Chain and Network Studies. 2011;11 (1): 69-84.
46. Vellema SMaking Contract Farming Work? Society and Technology in Philippine Transnational Agribusiness. Maastricht: Shaker Publishing; 2002:.
47. Weiss CH.How can theory-based evaluation make greater headway?.Evaluation Review. 1997;21 (4): 501-24.
48. White H.A contribution to current debates in impact evaluation.Evaluation. 2010;16 (2): 153-64.