Risk of bias in overviews of reviews: a scoping review of methodological guidance and four-item checklist

Research Synthesis Methods

Volumn 8, Issue 1, 2017, Pages 92-108

  Ballard, Madeleine ^a   Montgomery, Paul ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

meta review; overview of reviews; scoping review; synthesis methods; umbrella review

Indexed keywords

ALGORITHM; CHECKLIST; CLINICAL TRIAL (TOPIC); COST BENEFIT ANALYSIS; EVIDENCE BASED MEDICINE; FACTUAL DATABASE; HUMAN; LITERATURE; METHODOLOGY; PUBLICATION; PUBLISHING; STANDARDS;

ALGORITHMS; CHECKLIST; CLINICAL TRIALS AS TOPIC; COST-BENEFIT ANALYSIS; DATABASES, FACTUAL; EVIDENCE-BASED MEDICINE; HUMANS; PUBLICATION BIAS; PUBLICATIONS; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC;

EID: 85014410156     PISSN: None     EISSN: 17592887     Source Type: Journal    
DOI: 10.1002/jrsm.1229     Document Type: Article

References (50)

1. Aromataris E, Fernandez R, Godfrey C, Holly C, Khalil H, Tungpunkom P 2014. Methodology for JBI umbrella reviews. The Joanna Briggs Institute Reviewers Manual (Suppl): 5–34.
2. Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P 2015. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare 13: 132–40.
3. Baker PR, Costello JT, Dobbins M, Waters EB 2014. The benefits and challenges of conducting an overview of systematic reviews in public health: a focus on physical activity. Journal of Public Health 36: 517–21.
4. Becker L, Oxman A 2011. Chapter 22: Overviews of reviews. In Higgins J, Green S (eds.). Cochrane Handbook for Systematic Reviews of Interventions www.cochrane-handbook.org. The Cochrane Collaboration.
5. Caird J, Sutcliffe K, Kwan I, Dickson K, Thomas J 2015. Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evidence and Policy 11: 81–97.
6. Caldwell DM, Welton NJ, Ades AE 2010. Mixed treatment comparison analysis provides internally coherent treatment effect estimates based on overviews of reviews and can reveal inconsistency. Journal of Clinical Epidemiology 63: 875–82.
7. Cochrane Comparing Multiple Interventions Methods Group. 2012 Review type & methodological considerations: background paper for the first part of the Paris CMIMG discussion. Paris CMIMG Discussion, Paris, France.
8. Cochrane Comparing Multiple Interventions Methods Group 2013. Undertaking, publishing and maintaining Cochrane Reviews that compare multiple interventions.
9. Cochrane Empty Reviews Project. 2013. The Cochrane empty reviews project [online]. Available from: https://empty-reviews.org [Accessed May 10 2016].
10. Conn VS, Coon Sells TG 2014. WJNR welcomes umbrella reviews. Western Journal of Nursing Research 36: 147–151.
11. Cooper H, Koenka AC 2012. The overview of reviews: unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. The American Psychologist 67: 446–462.
12. Flodgren G, Eccles Martin P, Shepperd S, Scott A, Parmelli E, Beyer Fiona R 2011. An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes. Cochrane Database of Systematic Reviews[Online]. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009255/abstract.
13. Grant SP, Mayo-Wilson E, Melendez-Torres G, Montgomery P 2013. Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications. PLoS One 8: e65442.
14. Hartling L, Chisholm A, Thomson D, Dryden DM 2012. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS ONE 7: e49667.
15. Hartling L, Vandermeer B, Fernandes RM 2014. Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. Evidence-Based Child Health a Cochrane Review Journal 9: 486–94.
16. Higgins JPT, Green S 2011. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from: http://handbook.cochrane.org/0_2_how_to_cite_the_handbook.htm
17. Hill S, Ryan R 2014. Cochrane review—interventions to improve safe and effective medicines use by consumers: an overview of systematic review. Journal of Evidence-Based Medicine 7: 231–231.
18. Ioannidis JP 2009. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ 181: 488–93.
19. Jadad AR, Cook DJ, Browman GP 1997. A guide to interpreting discordant systematic reviews. CMAJ 156: 1411–6.
20. Jones L, Othman M, Dowswell T, Alfirevic Z, Gates S, Newburn M, Jordan S, Lavender T, Neilson James P 2012. Pain management for women in labour: an overview of systematic reviews. Cochrane Database of Systematic Reviews[Online]. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009234.pub2/abstract.
21. Jørgensen AW, Hilden J, Gøtzsche PC 2006. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ 333: 782.
22. Lamming L, Mason D, Morton K, Pears S, Bijker M, Sutton S, Hardeman W 2014. What do we know about brief interventions to promote physical activity? Systematic review of reviews. European Health Psychologist 16: 808.
23. Li L, Tian J, Tian H, Sun R, Liu Y, Yang K 2012. Quality and transparency of overviews of systematic reviews. Journal of Evidence-Based Medicine 5: 166–173.
24. Martínez-González NA, Berchtold P, Ullman K, Busato A, Egger M 2014. Integrated care programmes for adults with chronic conditions: a meta-review. International Journal for Quality in Health Care 26: 561–570.
25. Meltzer H 1972. Review of reviews in industrial psychology, 1950–1969. Personnel Psychology 25: 201–222.
26. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, Group Q 1999. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. The Lancet 354: 1896–1900.
27. Moher D, Schulz KF, Simera I, Altman DG 2010. Guidance for developers of health research reporting guidelines. PLoS Medicine 7: e1000217.
28. Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C 2009. Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. Journal of Clinical Epidemiology 62: 1021–1030.
29. O'donnell A, Anderson P, Newbury-Birch D, Schulte B, Schmidt C, Reimer J, Kaner E 2014. The impact of brief alcohol interventions in primary healthcare: a systematic review of reviews. Alcohol and Alcoholism 49: 66–78.
30. Olsen O, Middleton P, Ezzo J, Gøtzsche PC, Hadhazy V, Herxheimer A, Kleijnen J, Mcintosh H 2001. Quality of Cochrane reviews: assessment of sample from 1998. BMJ 323: 829–832.
31. Oxman AD, Guyatt GH 1991. Validation of an index of the quality of review articles. Journal of Clinical Epidemiology 44: 1271–8.
32. Pieper D, Antoine S-L, Mathes T, Neugebauer EAM, Eikermann M 2014a. Systematic review finds overlapping reviews were not mentioned in every other overview. Journal of Clinical Epidemiology 67: 368–375.
33. Pieper D, Antoine SL, Morfeld JC, Mathes T, Eikermann M 2014b. Methodological approaches in conducting overviews: current state in HTA agencies. Research Synthesis Methods 5: 187–99.
34. Pieper D, Antoine SL, Neugebauer EAM, Eikermann M 2014c. Up-to-dateness of reviews is often neglected in overviews: a systematic review. Journal of Clinical Epidemiology 67: 1302–1308.
35. Pieper D, Buechter R, Jerinic P, Eikermann M 2012. Overviews of reviews often have limited rigor: a systematic review. Journal of Clinical Epidemiology 65: 1267–73.
36. Piso B, Semlitsch T, Reinsperger I, Breuer J, Kaminski-Hartenthaler A, Kien C, Thaler K, Siebenhofer A 2015. Practical experience with overviews of reviews—valuable decision aid or academic exercise? Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 109: 300–8.
37. Ryan RE, Kaufman CA, Hill SJ 2009. Building blocks for meta-synthesis: data integration tables for summarising, mapping, and synthesising evidence on interventions for communicating with health consumers. BMC Medical Research Methodology 9: 16.
38. Santaguida PL, Keshavarz H, Carlesso LC, Lomotan M, Gross A, Macdermid JC, Walton DM, Group IW 2013. A description of the methodology used in an overview of reviews to evaluate evidence on the treatment, harms, diagnosis/classification, prognosis and outcomes used in the management of neck pain. The Open Orthopaedics Journal 7: 461–72.
39. Shea B, Moher D, Graham I, Pham B, Tugwell P 2002. A comparison of the quality of Cochrane reviews and systematic reviews published in paper-based journals. Evaluation & the Health Professions 25: 116–129.
40. Shea BJ, Bouter LM, Peterson J, Boers M, Andersson N, Ortiz Z, Ramsay T, Bai A, Shukla VK, Grimshaw JM 2007. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS ONE 2: e1350.
41. Shojania K, Sampson MJ, Ansari MT, Ji J, Doucette S, Moher D 2007. How quickly do systematic reviews go out of date? A survival analysis. Annals of Internal Medicine 147: 224–233.
42. Singh Jasvinder A, Wells George A, Christensen R, Tanjong Ghogomu E, Maxwell L, Macdonald John K, Filippini G, Skoetz N, Francis Damian K, Lopes Luciane C, Guyatt Gordon H, Schmitt J, La Mantia L, Weberschock T, Roos Juliana F, Siebert H, Hershan S, Cameron C, Lunn Michael PT, Tugwell P, Buchbinder R 2011. Adverse effects of biologics: a network meta-analysis and Cochrane overview. Cochrane Database of Systematic Reviews[Online]. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008794.pub2/abstract.
43. Smith V, Devane D, Begley CM, Clarke M 2011. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Medical Research Methodology 11: 15.
44. Steultjens EM, Dekker J, Bouter LM, Leemrijse CJ, Van Den Ende CH 2005. Evidence of the efficacy of occupational therapy in different conditions: an overview of systematic reviews. Clinical Rehabilitation 19: 247–254.
45. Theodoratou E, Tzoulaki I, Zgaga L, Ioannidis JP 2014. Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials. BMJ 348: g2035.
46. Thomson D 2014. Evidence synthesis in child health: overviews of reviews. Evidence-Based Child Health 9: 1–2.
47. Thomson D, Foisy M, Oleszczuk M, Wingert A, Chisholm A, Hartling L 2013. Overview of reviews in child health: evidence synthesis and the knowledge base for a specific population. Evidence-Based Child Health: A Cochrane Review Journal 8: 3–10.
48. Thomson D, Russell K, Becker L, Klassen T, Hartling L 2010. The evolution of a new publication type: steps and challenges of producing overviews of reviews. Research Synthesis Methods 1: 198–211.
49. Wang, X., Lindsley, K. & Li, T. 2012. Is there agreement in outcomes among Cochrane reviews to support ‘Overviews’ of reviews? A case study within the Cochrane Eyes and Vision Group (CEVG). The Cochrane Library. 2012 Supplement.
50. Yuan JQ, Liu YL, Yang KH, Chen YL, Yang M 2012. Survey of quality assessment methods of included studies in overviews of reviews. Chinese Journal of Evidence-Based Medicine 12: 238–242.