Systematic reviews: From 'bare bones' reviews to policy relevance

Journal of Development Effectiveness

Volumn 4, Issue 3, 2012, Pages 388-408

  Snilstveit, Birte ^a  

^a London School of Hygiene and Tropical Medicine   (United Kingdom)

Author keywords

development effectiveness; evidence based policy; systematic review

Indexed keywords

ASSESSMENT METHOD; DEVELOPMENT ECONOMICS; ECONOMIC DEVELOPMENT; POLICY ANALYSIS;

EID: 84866652315     PISSN: 19439342     EISSN: 19439407     Source Type: Journal    
DOI: 10.1080/19439342.2012.709875     Document Type: Article

References (89)

1. 3ie, 2008. Principles for impact evaluation [online]. http://www.3ieimpact.org/media/filer/2012/04/20/principles-for-impact-evaluation.pdf(Accessed: 23 July 2012).
2. 3ie. 2009. Building on what works: 2009 annual report, New Delhi: International Initiative for Impact Evaluation.
3. 3ie. n.d. 2010-2013 Improving lives through impact evaluation: a strategy for 3ie, New Delhi: International Initiative for Impact Evaluation.
4. Anderson, L.M. 2011. Using logic models to capture complexity in systematic reviews. Research synthesis methods, 2(1): 32-42.
5. Anderson, R. and Shemilt, I. 2010. "The role of economic perspectives and evidence in systematic review". In Evidence-based decisions and economics-health care, social welfare, education and criminal justice, 2nd, Edited by: Shemilt, I., Mugford, M., Vale, L., Marsh, K. and Donaldson, C. Oxford: Wiley-Blackwell.
6. Armstrong, R. and Waters, E., 2007. Systematic reviews of health promotion and public health interventions guidelines [online]. http://ph.cochrane.org/sites/ph.cochrane.org/files/uploads/Guidelines%20HP_PH%20reviews.pdf(Accessed: 9 November 2011).
7. Atkins, S. 2008. Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC medical research methodology, 8(21) doi:10.1186/1471-2288-8-21
8. Bamberger, M, Rao, V and Woolcock, M. 2010. Using mixed methods in monitoring and evaluation-experiences from international development, Washington, DC: World Bank. Policy Research Working Paper No. 5245
9. Barnett-Page, E. and Thomas, J. 2009. Methods for the synthesis of qualitative research: a critical review. BMC medical research methodology, 9(59) doi:10.1186/1471-2288-9-59
10. Barroso, J. 2003. The challenges of searching for and retrieving qualitative studies. Western journal of nursing research, 25(2): 153-178.
11. Baxter, S. 2010. Synthezising diverse evidence: the use of primary qualitative data analysis methods and logic models in health reviews. Public health I, 24: 99-106.
12. Birckmayer, J.D. and Weiss, C.H. 2000. Theory-based evaluation in practice: what do we learn?. Evaluation review, 24(4): 407-431.
13. Boot, A., 2011. Searching for studies. In: J. Noyes, et al., eds. Supplementary guidance for inclusion of qualitative research in Cochrane systematic reviews of interventions [online]. Version 1 (updated August 2011). The Cochrane Collaboration Qualitative Methods Group. http://cqrmg.cochrane.org/supplemental-handbook-guidance(Accessed: 5 February 2012).
14. Bravata, D.M. 2005. Challenges in systematic reviews: synthesis of topics related to the delivery, organization, and financing of health care. Annals of internal medicine, 142(12): 1057-1065.
15. Campbell, M. 2002. Framework for design and evaluation of complex interventions to improve health. British medical journal, 321(7262): 694-696.
16. Campbell, R. 2003. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Social science and medicine, 56: 671-684.
17. Cartwright, N., 2010. Will this policy work for you? Predicting effectiveness better: how philosophy helps [online]. Presidential Address Philosophy of Science Association. http://www2.lse.ac.uk/CPNSS/projects/orderProject/documents/Publications/CartwrightPSA.pdf(Accessed: 23 July 2012).
18. CASP (Critical Appraisal Skills Programme), 2006. 10 Questions to help you make sense of qualitative research [online]. http://www.phru.nhs.uk/Doc_Links/Qualitative%20Appraisal%20Tool.pdf(Accessed: 21 August 2009).
19. Centre for Global Development. 2006. When will we ever learn?, Washington, DC: Centre for Global Development.
20. Centre for Reviews and Dissemination. 2008. Systematic reviews: CRD's guidance for undertaking reviews in health care, York: Centre for Reviews and Dissemination, University of York.
21. Chen, H.T., Donaldson, S.I. and Mark, M.M. 2011. "Validity frameworks for outcome evaluation". In Advancing validity in outcome evaluation: theory and practice 5-16. New directions for evaluation, 2011 (130)
22. Cochrane Collaboration, 2011. Cochrane Collaboration logo. http://www.cochrane.org/about-us/history/our-logo (Accessed: 23 July 2012).
23. Cook, T.D. and Campbell, D.T. 1979. Quasi-experimentation: design and analysis issues for field settings, Chicago, IL: Russell Sage Foundation.
24. Dane, A.V. and Schneider, B.H. 1998. Program integrity in primary and early secondary prevention: are implementation effects out of control?. Clinical psychology review, 18(1): 23-45.
25. Davies, P. 2006. "What is needed from research synthesis from a policy-making perspective?". In Moving beyond effectiveness in evidence synthesis-methodological issues in the synthesis of diverse sources of evidence, Edited by: Popay, J. London: National Institute for Health and Clinical Excellence.
26. Denison, E., et al., 2011. Interventions to reduce the prevalence of female genital mutilation/cutting in African countries [online] (Protocol), Campbell Library. http://campbellcollaboration.org/lib/download/1049/ (Accessed: 9 November 2011).
27. Dixon-Woods, M. 2005. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of health service research and policy, 10(1): 45-53.
28. Dixon-Woods, M. 2006. How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative research, 2(27): 27-44.
29. Dobson, D. and Cook, T.J. 1980. Avoiding type II error in program evaluation: results from a field experiment. Evaluation and program planning, 3: 269-276.
30. Fisher, M. 2006. Using qualitative research in systematic reviews: older people's views of hospital discharge, Bristol: The Policy Press. How knowledge work in social care report 9, Social Care Institute for Excellence.
31. Gaarder, M.M., Glassman, A. and Todd, J. 2010. Conditional cash transfers and health: unpacking the causal chain. Journal of development effectiveness, 2(1): 6-50.
32. Gardner, B. 2010. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Social science and medicine, 70: 1618-1625.
33. Glasziou, P. 2010. Taking healthcare interventions from trial to practice. British medical journal, 341: 384-387.
34. Greenhalgh, T. and Peacock, R. 2005. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. British medical journal, 331: 1064-1065.
35. Greenhalgh, T., Kristiansson, E. and Robinson, V. 2007. Realist review to understand the efficacy of school feeding programmes. British medical journal, 335: 858-861.
36. Hagen-Zanker, J., et al., 2012. Making systematic reviews work for international development research [online]. Secure Livelihoods Research Consortium Briefing Paper 1. http://www.odi.org.uk/resources/docs/7546.pdf (Accessed: 27 April 2012).
37. Hannes, K., 2011. Critical appraisal of qualitative research. In: J. Noyes, et al., eds. Supplementary guidance for inclusion of qualitative research in Cochrane systematic reviews of interventions [online]. Version 1. The Cochrane Collaboration Qualitative Methods Group. http://cqrmg.cochrane.org/supplemental-handbook-guidance(Accessed: 9 November 2011).
38. Harden, A. 2009. Teenage pregnancy and social disadvantage: systematic review integrating controlled trials and qualitative studies. British medical journal, 339: b4254 doi:10.1136/bmj.b4254
39. Higgins, J.P.T. and Green, S., eds., 2011. Cochrane handbook for systematic reviews of interventions version 5.1.0 [online] (updated March 2011). The Cochrane Collaboration. http://www.cochrane-handbook.org/(Accessed: 5 February 2012).
40. Herbert, R.D. and Bø, K. 2005. Analysis of quality of interventions in systematic reviews. British medical journal, 331: 507-509.
41. King, E., Samii, C. and Snilstveit, B. 2010. Interventions to promote social cohesion in sub-Saharan Africa. Journal of development effectiveness, 2(3): 336-370.
42. Leiknes, K.A. 2011. Electroconvulsive therapy for depression (protocol). Cochrane database of systematic reviews,: CD009105 doi:10.1002/14651858.CD009105
43. Lins, S. 2011. Efficacy and experiences of telephone counselling for informal carers of people with dementia (protocol). Cochrane database of systematic reviews,: CD009126 doi:10.1002/14651858.CD009126
44. Mayo-Wilson, E. 2007. Reporting implementation in randomized trials: proposed additions to the consolidated standards of reporting trials statement. American journal of public health, 97(4): 630-633.
45. Mays, N., Pope, C. and Popay, J. 2005. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of health service research and policy, 10(Suppl. 1): 6-20.
46. Montgomery, P. forthcoming. The Oxford implementation index: a new tool for incorporating implementation data into systematic reviews and meta-analyses,
47. Munro, S.A. 2007. Patient adherence to tuberculosis treatment: a systematic review of qualitative research. PLoS medicine, 4(7): 1230-1245.
48. NICE. 2009. Methods for the development of NICE public health guidance, 2nd, London: NICE.
49. Noyes, J. 2010. Never mind the qualitative feel the depth! The evolving role of qualitative research in Cochrane intervention reviews. Journal of research in nursing, 15(6): 525-534.
50. Noyes, J., et al., 2011. Qualitative research and Cochrane reviews. In: J.P.T. Higgins and S. Green, eds. Cochrane handbook for systematic reviews of interventions [online]. Version 5.0.1. Oslo: The Cochrane Collaboration. http://www.cochrane-handbook.org(Accessed: 9 November 2011).
51. Noyes, J. and Lewin, S., 2011. Supplemental guidance on selecting a method of qualitative evidence synthesis, and integrating qualitative evidence with Cochrane intervention reviews. In: J. Noyes, et al., eds. Supplementary guidance for inclusion of qualitative research in Cochrane systematic reviews of interventions [online]. Version 1. The Cochrane Collaboration Qualitative Methods Group. http://cqrmg.cochrane.org/supplemental-handbook-guidance(Accessed: 9 November 2011).
52. Oliver, S. 2005. An emerging framework for including different types of evidence in systematic reviews for public policy. Evaluation, 11(4): 428-446.
53. Pawson, R. 2005. Realist review-a new method of systematic review designed for complex policy interventions. Journal of health policy and research, 10(Suppl. 1): 21-34.
54. Pawson, R. 2006. Evidence based policy: a realist perspective, London: Sage.
55. Petrosino, A. 2001. Meeting the challenges of evidence-based policy: the Campbell collaboration. The ANNALS of the American Academy of Political and Social Science, 578(1): 14-34.
56. Petrosino, A., Turpin-Petrosino, C., and Buehler, J., 2004. 'Scared Straight' and other juvenile awareness programs for preventing juvenile delinquency [online]. Campbell Systematic Reviews. http://campbellcollaboration.org/lib/download/13/(Accessed: 3 August 2011).
57. Petticrew, M. 2003. Why certain systematic reviews reach uncertain conclusions. British medical journal, 326: 756-758.
58. Petticrew, M. 2009. Cochrane update: better evidence about wicked issues in tackling inequities. Journal of public health, 31(3): 453-456.
59. Petticrew, M. and Roberts, H. 2003. Evidence, hierarchies, and typologies: horses for courses. Journal of epidemiology and community health, 57: 527-529.
60. Petticrew, M. and Roberts, H. 2006. Systematic reviews in the social sciences-a practical guide, Oxford: Blackwell Publishing.
61. Popay, J. 2005. Commentary-moving beyond floccinaucinihilipilification: enhancing the utility of systematic reviews. Journal of clinical epidemiology, 58: 1079-1080.
62. Popay, J. and Roberts, H. 2006. "Introduction: methodological issues in the synthesis of diverse sources of evidence". In Moving beyond effectiveness in evidence synthesis-methodological issues in the synthesis of diverse sources of evidence, Edited by: Popay, J. London: National Institute for Health and Clinical Excellence.
63. Popay, J., et al., 2006. Guidance on the conduct of narrative synthesis in systematic reviews: a comparison of guidance-led narrative synthesis versus meta-analysis. Version 1: April 2006, 14th Cochrane Colloquium, Dublin, Ireland. ESRC Methods Programme, available on request from: j.cox@lancaster.ac.uk
64. Pope, C., Mays, N. and Popay, J. 2007. Synthesising qualitative and quantitative health evidence: a guide to methods, 79-89. Maidenhead: Open University Press.
65. Pound, P. 2005. Resisting medicines: a synthesis of qualitative studies of medicine taking. Social science and medicine, 61: 133-155.
66. Raynor, D.K. 2007. A systematic review of quantitative and qualitative research on the role and effectiveness of written information available to patients about individual medicines. Health technology assessment, 11(5): 1-160.
67. Rodgers, M. 2009. Testing methodological guidance on the conduct of narrative synthesis in systematic reviews effectiveness of interventions to promote smoke alarm ownership and function. Evaluation, 15(1): 47-71.
68. Roen, K. 2006. Extending systematic reviews to include evidence on implementation. Social science and medicine, 63: 1060-1071.
69. Roen, K. 2007. Extending systematic reviews to include evidence on implementation: methodological work on a review of community-based initiatives to prevent injuries. Social science and medicine, 63: 1060-1071.
70. Rubenstein, L.V. 2009. Determining key features of effective depression interventions, Los Angeles, CA: Greater Los Angeles Veterans Affairs Healthcare System/Southern California/RAND Evidence-based Practice Center.
71. Ryan, R. 2011. Notification and support for people exposed to the risk of Creutzfeldt-Jakob disease (CJD) (or other prion diseases) through medical treatment (iatrogenically). Cochrane database of systematic reviews, 16(3): 7578 doi:10.1002/14651858.CD007578.pub2
72. Saini, M. and Shlonsky, A. 2012. Systematic synthesis of qualitative research, Oxford: Oxford University Press.
73. Scriven, M. 2008. A summative evaluation of RCT methodology: an alternative approach to causal research. Journal of multi-disciplinary evaluation, 5(9): 11-19.
74. Shadish, W.R., Cook, T.D. and Campbell, D.T. 2002. Experimental and quasi-experimental designs for generalized causal inference, Wadsworth: Cengage Learning.
75. Shaw, R.L. 2004. Finding qualitative research: an evaluation of search strategies. BMC medical research methodology, 4(5): 1-5.
76. Spencer, L. 2003. Quality in qualitative evaluation: a framework for assessing research evidence, London: Government Chief Social Researcher's Office.
77. Stewart, R. 2010. What is the impact of microfinance on poor people? A systematic review of evidence from sub-Saharan Africa, London: EPPI-Centre, Social Science Research Unit, University of London.
78. Thomas, J. and Harden, A. 2008. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC medical research methodology, 8(45): 1-10.
79. Thomas, J., Harden, A. and Newman, M. 2012. "Synthesis: combining results systematically and appropriately". In An introduction to systematic reviews, Edited by: Gough, D., Oliver, S. and Thomas, J. 179-226. London: Sage.
80. Thomas, J. 2003. Children and healthy eating: a systematic review of barriers and facilitators, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.
81. Thomas, J. 2004. Integrating qualitative research with trials in systematic reviews. British medical journal, 328: 1010-1012.
82. Ton, G. 2012. The mixing of methods: a three-step process for improving rigour in impact evaluations. Evaluation, 18(1): 5-25.
83. Tugwell, P. 2010. Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. British medical journal, 341: 873-877.
84. Van der Knapp, L.M. 2008. Combining Campbell standards and the realist evaluation approach-the best of two worlds?. American journal of evaluation, 29(1): 48-57.
85. Voils, C.I. 2008. Making sense of qualitative and quantitative findings in mixed research synthesis studies. Field methods, 20(1): 3-25.
86. Waddington, H., et al., 2011. Farmer Field Schools for improving farming practices and farmer outcomes in low- and middle-income countries: a systematic review [online]. Campbell Collaboration, Protocol. http://campbellcollaboration.org/lib/download/1775/(Accessed: 5 February 2012).
87. White, H. 2009. Theory-based impact evaluation: principles and practice, New Delhi: International Initiative for Impact Evaluation (3ie).
88. Whitlock, E.P. 2005. Screening and interventions for childhood overweight: a summary of evidence for the US preventive services task force. Pediatrics, 116(1): e125-e144.
89. Williamson, L.M. 2009. Limits to modern contraceptive use among women in developing countries: a systematic review of qualitative research. Reproductive health, 6(3): 1-12.