Purity or pragmatism? Reflecting on the use of systematic review methodology in development

Journal of Development Effectiveness

Volumn 4, Issue 3, 2012, Pages 430-444

  Stewart, Ruth ^a,b   van Rooyen, Carina ^b   de Wet, Thea ^b  

^a UNIVERSITY OF LONDON   (United Kingdom)

^b UNIVERSITY OF JOHANNESBURG   (South Africa)

Author keywords

development; microfinance; multidisciplinary teamwork; sub Saharan Africa; systematic review

Indexed keywords

ASSESSMENT METHOD; DEVELOPMENT ECONOMICS; ECONOMIC DEVELOPMENT; MICROFINANCE;

SUB-SAHARAN AFRICA;

EID: 84866651641     PISSN: 19439342     EISSN: 19439407     Source Type: Journal    
DOI: 10.1080/19439342.2012.711341     Document Type: Article

References (54)

1. Abadie, A. and Imbens, G. 2009. Matching on the estimated propensity score, Cambridge, MA: National Bureau of Economic Research. NBER Working Paper No. w15301
2. Algoso, D., 2011. Limitations of RCTs [online]. Blog on 1, 3, 13 and 24 April at Find what works. http://findwhatworks.wordpress.com/2011/04/01/limitations-of-rcts-politics-and-context/(Accessed: 24 May 2011).
3. Alkire, S. and Santos, M.E., 2010. Multidimensional poverty index: 2010 data [online]. Oxford: Oxford Poverty and Human Development Initiative. www.ophi.org.uk/policy/multidimensional-poverty-index/(Accessed: 24 May 2011).
4. Bamberger, M., Rao, W. and Woolcock, M. 2010. Using mixed methods in monitoring and evaluation: experiences from international development, Washington, DC: Development Research Group, World Bank. Policy Research Working Paper No. 5245
5. Bamberger, M., Rugh, J. and Mabry, L. 2006. Realworld evaluation: working under budget, time, data and political constraints, Thousand Oaks, CA: Sage Publications.
6. Banerjee, A. and Duflo, E. 2011. Poor economics: a radical rethinking of the way we fight global poverty, New York, NY: Public Affairs.
7. Barrett, C. and Carter, M. 2010. The power and pitfalls of experiments in development economics: some non-random reflections. Applied economic perspectives and policy, 32(4): 515-548.
8. Bendavid, E., 2011. Those who fail to learn from medical history. Article for small changes, big results forum of the Boston review, March/April [online]. http://www.bostonreview.net/BR36.2/eran_bendavid_behavioral_economics_global_development.php (Accessed: 18 June 2011).
9. Bhargava, A. 2008. Randomised controlled experiments in health and social sciences: some conceptual issues. Economics and human biology, 6: 293-298.
10. Blattman, C., 2011. Behavioural economics and randomised trials: trumpeted, attacked and parried [online]. Blog on 24 March at Chris Blattman. http://chrisblattman.com/2011/03/24/behavioral-economics-and-randomized-trials-trumpeted-attacked-and-parried/ (Accessed: 24 May 2011).
11. Brunton, V. 2005. Promoting physical activity amongst children outside of physical education classes: a systematic review integrating intervention studies and qualitative studies. Health education journal, 64: 323-338.
12. Buckley, C., 2010. To test housing program, some are denied aid [online]. The New York Times, 8 December. http://www.nytimes.com/2010/12/09/nyregion/09placebo.html?_r=2&hpAlways=&pagewanted=all(Accessed: 24 May 2011).
13. Campbell Collaboration, 2011. [online]. http://www.campbellcollaboration.org/(Accessed: 24 May 2011).
14. Cartwright, N. 2007. Are RCTs the gold standard?. BioSocieties, 2: 11-20.
15. Chambers, R. 2009. Designing impact evaluations: different perspectives, New Delhi: 3ie. Working Paper 4
16. Clemens, M.A. and Demombynes, G. 2010. When does rigorous impact evaluation make a difference? The case of the Millennium Villages, Washington, DC: Centre for Global Development. CGD Working Paper No. 225
17. Cochrane Collaboration, 2011. [online]. http://www.cochrane.org/cochrane-reviews(Accessed: 24 May 2011).
18. Cochrane Consumer Network, homepage. [online]. http://consumers.cochrane.org/ (Accessed: 18 June 2011).
19. Copestake, J., Golberg, N. and Karlan, D. 2009. Randomised control trials are the best way to measure impact of microfinance programmes and improve microfinance product designs. Enterprise development and microfinance, 20(3): 167-176.
20. Court, J., Hovland, I. and Young, J., eds. 2005. Bridging research and policy in development: evidence and the change process, London: ITDG Publishing.
21. Deaton, A. 2009. "Instruments of development: randomisation in the tropics, and the search for the elusive keys to economic development". In Proceedings of the British Academy Vol. 162, 123-160.
22. Devarajan, S., 2011. Can randomised control trials reduce poverty? [online]. Blog on 23 March at Africa can end poverty. http://blogs.worldbank.org/africacan/can-randomized-control-trials-reduce-poverty (http://blogs.worldbank.org/africacan/can-randomized-control-trials-reduce-poverty)
23. Duflo, E., Glennerster, R., and Kremer, M., 2004. Randomised evaluations of interventions in social service delivery. Development Outreach, March (special report) [online]. http://stuff.mit.edu:8001/afs/athena/course/other/d-lab/DlabIII06/random-eval.pdf (http://stuff.mit.edu:8001/afs/athena/course/other/d-lab/DlabIII06/random-eval.pdf) (Accessed: 18 June 2011).
24. El-Zoghbi, M. and Martinez, M. 2011. Measuring changes in clients lives through microfinance: contributions of different approaches, Washington, DC: Consultative Group to Assist the Poor. CGAP Brief, May
25. EPPI-Centre, 2011. [online]. http://eppi.ioe.ac.uk/ (Accessed: 24 May 2011).
26. Glennerster, R. and Kremer, M., 2011. Behavioural economics at work in poor countries. Article for Small Changes, Big Results, a forum of the Boston Review, March/April [online]. http://www.bostonreview.net/BR36.2/glennerster_kremer_behavioral_economics_global_development.php(Accessed: 18 June 2011).
27. Goldacre, B., 2011. How can you tell if a policy is working? Run a trial [online]. The Guardian, 14 May. http://www.guardian.co.uk/commentisfree/2011/may/14/bad-science-ben-goldacre-randomised-trials (Accessed: 18 June 2011).
28. GSR, 2009. Rapid evidence assessment toolkit [online]. London: Government Social Research Service. http://www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-evidence-assessment/how-to-do-a-rea (Accessed: 24 May 2011).
29. Haddad, L., 2011. A new harvest of RCTs? [online]. Blog on 15 May at Development horizons. http://www.developmenthorizons.com/2011/05/new-harvest-of-rcts.html(Accessed: 24 May 2011).
30. Harden, A. 2001. "Finding research evidence: systematic searching". In Using research for effective health promotion, Edited by: Oliver, S. and Peersman, G. 47-68. Buckingham: Open University Press.
31. Honohan, P. and Beck, T. 2007. Making finance work for Africa, Washington, DC: World Bank.
32. Hulme, D. 2000. Impact assessment methodologies for microfinance: theory, experience and better practice. World development, 28(1): 79-98.
33. Jones, H. 2009. The 'gold standard' is not a silver bullet for evaluation. ODI Opinion 127, London: Overseas Development Institute.
34. Karlan, D. and Apple, J. 2011. More than good intentions: how a new economics is helping to solve global poverty, New York: Penguin.
35. Kermeliotis, T., 2011. Experts warn Africa must learn from India's microfinance problems [online]. CNN Marketplace Africa. http://edition.cnn.com/2011/BUSINESS/03/23/microfinance.africa.lessons/index.html(Accessed: 23 March 2011).
36. King, E., Samii, C. and Snilstveit, B. 2010. Interventions to promote social cohesion in sub-Saharan Africa (Synthetic review 002), New Delhi: 3ie.
37. Korth, M. 2012. Microfinance: development intervention or just another bank?. Journal of agrarian change, forthcoming.,
38. Kristof, N.D., 2011. Getting smart on aid [online]. The New York Times, 18 May. http://www.nytimes.com/2011/05/19/opinion/19kristof.html?_r=1&src=tptw (http://www.nytimes.com/2011/05/19/opinion/19kristof.html?_r=1&src=tptw) (Accessed: 18 June 2011).
39. Lindley, H., 2011. Monique and the mango rains: qualitative research in action [online]. Blog on 10 May at Pas à pas. http://pasapasblog.wordpress.com/2011/05/10/monique-and-the-mango-rains-qualitative-research-in-action/#more-189(Accessed: 24 May 2011).
40. Makina, D. and Malobola, L.M. 2004. Impact assessment of microfinance programmes, including lessons from Khula Enterprise Finance. Development Southern Africa, 21(5): 799-814.
41. Morduch, J., 2011. Is microfinance research as bad as medical research? [online]. Blog on 5 January at CGAP Microfinance Blog. http://microfinance.cgap.org/2011/01/05/is-microfinance-research-as-bad-as-medical-research/(Accessed: 24 May 2011).
42. Nanor, M.A. 2008. Microfinance and its impact on selected districts in eastern region of Ghana. Master of Philosophy Dissertation, Kwame Nkrumah University of Science and Technology.
43. Odell, K. 2010. Measuring the impact of microfinance: taking another look, Washington, DC: Grameen Foundation.
44. Ravallion, M. 2009. Should the randomistas rule?. The economists' voice, 6(2) Article 6
45. Roodman, D., 2011. Literature review on microfinance impacts in Africa [online]. Blog on 9 February at David Roodman's microfinance open book blog. http://blogs.cgdev.org/open_book/2011/02/literature-review-on-microfinance-impacts-in-africa.php (Accessed: 24 May 2011).
46. Stewart, R., 2011. What is being excluded in systematic reviews? [online]. Blog on 1 April at Ruth Stewart. http://www.ruthstewartresearch.org/what-is-being-excluded-in-systematic-reviews/(Accessed: 1 February 2012).
47. Stewart, R. 2010. What is the impact of microfinance on poor people? A systematic review of evidence from sub-Saharan Africa, London: EPPI-Centre, Social Science Research Unit, University of London. Technical Report
48. Subramanian, A., 2011. Nicholas Kristof and aid [online]. Blog post on 20 May at Global Development: Views from the Center. http://blogs.cgdev.org/globaldevelopment/2011/05/nicholas-kristof-and-aid.php (Accessed: 24 May 2011).
49. Sumner, A. 2010. Global poverty and the new bottom billion: what if three-quarters of the world's poor live in middle-income countries?, Brighton: Institute for Development Studies. Working Paper
50. Sutcliffe, S. and Court, J. 2005. Evidence-based policymaking: what is it? How does it work? What relevance for developing countries?, London: Overseas Development Institute.
51. van Rooyen, C., Stewart, R., and de Wet, T., 2012. The impact of microfinance in sub-Saharan Africa: a systematic review of the evidence [online]. World Development. http://www.sciencedirect.com/science/article/pii/S0305750X12000496 (Accessed: 12 March 2012).
52. Waddington, H. 2009. Water, sanitation and hygiene interventions to combat childhood diarrhoea in developing countries (Synthetic review 001), New Delhi: 3ie.
53. White, H., 2011a. The 3ie policy window [online]. Presented at the 5th NONIE meeting, Paris, March 2011. http://nonie2011.org/?q=content/conference-programme (Accessed: 24 May 2011).
54. White, H. 2011b. An introduction to the use of randomised control trials to evaluate development interventions, New Delhi: 3ie. Working Paper No. 9