Preferred reporting items for overviews of systematic reviews including harms checklist: a pilot tool to be used for balanced reporting of benefits and harms

Journal of Clinical Epidemiology

Volumn 93, Issue , 2018, Pages 9-24

  Bougioukas, Konstantinos I ^a   Liakos, Aris ^a   Tsapas, Apostolos ^a,b   Ntzani, Evangelia ^c,d   Haidich, Anna Bettina ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF IOANNINA   (Greece)

^d BROWN UNIVERSITY   (United States)

Author keywords

Adverse events; Checklist; Harms; Overview of systematic reviews; Reporting; Umbrella reviews

Indexed keywords

CHECKLIST; HEALTH CARE; HUMAN; INFORMATION PROCESSING; INTERRATER RELIABILITY; PRIORITY JOURNAL; PUBLICATION; REVIEW; SYSTEMATIC REVIEW (TOPIC); EVIDENCE BASED MEDICINE; PERSONNEL; PILOT STUDY; PRACTICE GUIDELINE; RESEARCH;

CHECKLIST; EVIDENCE-BASED MEDICINE; HUMANS; PILOT PROJECTS; PRACTICE GUIDELINES AS TOPIC; RESEARCH PERSONNEL; RESEARCH REPORT; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85040228195     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.10.002     Document Type: Review

References (72)

1. Becker, A.L., Oxman, D.A., Chapter 22: overviews of reviews. Higgins, J., Green, S., (eds.) Cochrane Handb. Syst. Rev. Interv. Version 5.1.0 (updated March 2011), 2011, The Cochrane Collaboration Available at www.handbook.cochrane.org. Accessed November 6, 2017.
2. Lavis, J.N., How can we support the use of systematic reviews in policymaking?. PLoS Med, 6, 2009, e1000141.
3. Thomson, D., Russell, K., Becker, L., Klassen, T., Hartling, L., The evolution of a new publication type: steps and challenges of producing overviews of reviews. Res Synth Methods 1 (2010), 198–211.
4. Murad, M.H., Asi, N., Alsawas, M., Alahdab, F., New evidence pyramid. Evid Based Med 21 (2016), 125–127.
5. Lemeshow, A.R., Blum, R.E., Berlin, J.A., Stoto, M.A., Colditz, G.A., Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol 58 (2005), 867–873.
6. Golder, S., Loke, Y., McIntosh, H.M., Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol 61 (2008), 440–448.
7. Pieper, D., Buechter, R., Jerinic, P., Eikermann, M., Overviews of reviews often have limited rigor: a systematic review. J Clin Epidemiol 65 (2012), 1267–1273.
8. Hartling, L., Vandermeer, B., Fernandes, R.M., Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. Evid Based Child Health 9 (2014), 486–494.
9. Pieper, D., Antoine, S.L., Mathes, T., Neugebauer, E.A.M., Eikermann, M., Systematic review finds overlapping reviews were not mentioned in every other overview. J Clin Epidemiol 67 (2014), 368–375.
10. Hartling, L., Chisholm, A., Thomson, D., Dryden, D.M., A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One, 7, 2012, e49667.
11. Pieper, D., Antoine, S.L., Morfeld, J.C., Mathes, T., Eikermann, M., Methodological approaches in conducting overviews: current state in HTA agencies. Res Synth Methods 5 (2014), 187–199.
12. Pieper, D., Antoine, S.L., Neugebauer, E.A.M., Eikermann, M., Up-to-dateness of reviews is often neglected in overviews: a systematic review. J Clin Epidemiol 67 (2014), 1302–1308.
13. Haidich, A.B., Birtsou, C., Dardavessis, T., Tirodimos, I., Arvanitidou, M., The quality of safety reporting in trials is still suboptimal: survey of major general medical journals. J Clin Epidemiol 64 (2011), 124–135.
14. Saini, P., Loke, Y.K., Gamble, C., Altman, D.G., Williamson, P.R., Kirkham, J.J., Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ, 349, 2014, g6501.
15. Mahady, S.E., Schlub, T., Bero, L., Moher, D., Tovey, D., George, J., et al. Side effects are incompletely reported among systematic reviews in gastroenterology. J Clin Epidemiol 68 (2015), 144–153.
16. Zorzela, L., Golder, S., Liu, Y., Pilkington, K., Hartling, L., Joffe, A., et al. Quality of reporting in systematic reviews of adverse events: systematic review. BMJ, 348, 2014, f7668.
17. Wallace, B.C., Dahabreh, I.J., Schmid, C.H., Lau, J., Trikalinos, T.A., Modernizing the systematic review process to inform comparative effectiveness: tools and methods. J Comp Eff Res 2 (2013), 273–282.
18. Pollock, M., Fernandes, R.M., Becker, L.A., Featherstone, R., Hartling, L., What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Syst Rev, 5, 2016, 190.
19. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Group, P., Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ, 339, 2009, b2535.
20. Zorzela, L., Loke, Y.K., Ioannidis, J.P., Golder, S., Santaguida, P., Altman, D.G., et al. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ, 352, 2016, i157.
21. Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev, 4, 2015, 1.
22. Li, L., Tian, J., Tian, H., Sun, R., Liu, Y., Yang, K., Quality and transparency of overviews of systematic reviews. J Evid Based Med 5 (2012), 166–173.
23. Onishi, A.F.T., State of the art reporting. Biondi-Zoccai, G., (eds.) Umbrella Rev. Evid. Synth. with Overviews Rev. Meta-Epidemiologic Stud, 2016, Springer, London, UK, 189–202.
24. Aromataris, E., Fernandez, R., Godfrey, C.M., Holly, C., Khalil, H., Tungpunkom, P., Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc 13 (2015), 132–140.
25. Caird, J., Sutcliffe, K., Kwan, I., Dickson, K.T.J., Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach?. Evid Policy 11 (2015), 81–97.
26. Smith, V., Devane, D., Begley, C.M., Clarke, M., Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol, 11, 2011, 15.
27. Ballard, M., Montgomery, P., Risk of bias in overviews of reviews: a scoping review of methodological guidance and four-item checklist. Res Synth Methods 8 (2017), 92–108.
28. Fix, O.K., Liou, I., Karvellas, C.J., Ganger, D.R., Forde, K.A., Subramanian, R.M., et al. Development and pilot of a checklist for management of Acute Liver Failure in the Intensive care Unit. PLoS One, 11, 2016, e0155500.
29. Pollock, M., Sinha, I.P., Hartling, L., Rowe, B.H., Schreiber, S., Fernandes, R.M., Inhaled short-acting bronchodilators for managing emergency childhood asthma: an overview of reviews. Allergy 72 (2017), 183–200.
30. Gwet, K.L., Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 61 (2008), 29–48.
31. Wongpakaran, N., Wongpakaran, T., Wedding, D., Gwet, K.L., A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol, 13, 2013, 61.
32. El Emam, K., Benchmarking Kappa: interrater agreement in software process assessments. Empir Softw Eng 4 (1999), 113–133.
33. Gwet, K.L., Handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among raters. 4th ed., 2014, Advanced Analytics, LLC, Gaithersburg.
34. Martire, RLo, rel: Reliability coefficients. R package version 1.3.0. 2017 Available at https://CRAN.R-project.org/package=rel. Accessed November 6, 2017.
35. Büchter, R.B., Pieper, D., Most overviews of Cochrane reviews neglected potential biases from dual authorship. J Clin Epidemiol 77 (2016), 91–94.
36. Shojania, K.G., Sampson, M., Ansari, M.T., Ji, J., Doucette, S., Moher, D., Hοw quickly do systematic reviews go out of date? a survival analysis. Ann Intern Med 147 (2007), 224–233.
37. McNutt, M., Bradford, M., Drazen, J., Hanson, R.B., Howard, B., Jamieson, K.H., et al. Transparency in authors’ contributions and responsibilities to promote integrity in scientific publication. bioRxiv, 2017 Available at https://doi.org/10.1101/140228. Accessed November 6, 2017.
38. Ioannidis, J.P.A., Evans, S.J.W., Gøtzsche, P.C., O'Neill, R.T., Altman, D.G., Schulz, K., et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 141 (2004), 781–788.
39. Chou, R., Aronson, N., Atkins, D., Ismaila, A.S., Santaguida, P., Smith, D.H., et al. Assessing harms when comparing medical interventions. 2008 Rockville, MD. Available at https://www.ncbi.nlm.nih.gov/books/NBK47098/. Accessed November 6, 2017.
40. Sweeting, M.J., Sutton, A.J., Lambert, P.C., What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 23 (2004), 1351–1375.
41. Warren, F.C., Abrams, K.R., Golder, S., Sutton, A.J., Systematic review of methods used in meta-analyses where a primary outcome is an adverse or unintended event. BMC Med Res Methodol, 12, 2012, 64.
42. Cooper, H., Koenka, A.C., The overview of reviews: unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. Am Psychol 67 (2012), 446–462.
43. Siontis, K.C., Hernandez-Boussard, T., Ioannidis, J.P.A., Overlapping meta-analyses on the same topic: survey of published studies. BMJ, 347, 2013, f4501.
44. Jadad, A.R., Moore, R.A., Carroll, D., Jenkinson, C., Reynolds, D.J., Gavaghan, D.J., et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials 17 (1996), 1–12.
45. Higgins, J.P.T., Altman, D.G., Gøtzsche, P.C., Jüni, P., Moher, D., Oxman, A.D., et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 343, 2011, d5928.
46. Sterne, J.A., Hernan, M.A., Reeves, B.C., Savovic, J., Berkman, N.D., Viswanathan, M., et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 355, 2016, i4919.
47. Wells G, Shea B, O'Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomized studies in meta-analysis. Ottawa, ON The Ottawa Health Research Institute. Available at http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed July 15, 2017. [n.d].
48. Zhou, X.H., Perkins, A.J., Hui, S.L., Comparisons of software packages for generalized linear multilevel models. Am Stat 53 (1999), 282–290.
49. Bergmann, R., Ludbrook, J., Spooren, W.P.J.M., Different outcomes of the Wilcoxon-Mann-Whitney test from different statistics packages. Am Stat 54 (2000), 72–77.
50. McDonagh, M., Peterson, K., Raina, P., Chang, S., Shekelle, P., Avoiding bias in selecting studies. 2008 Rockville, MD. Available at https://www.ncbi.nlm.nih.gov/books/NBK126701/. Accessed November 6, 2017.
51. Zintzaras, E., Gamma-aminobutyric acid A receptor, alpha-2 (GABRA2) variants as individual markers for alcoholism: a meta-analysis. Psychiatr Genet 22 (2012), 189–196.
52. Niv, Y., Estrogen receptor beta expression and colorectal cancer: a systematic review and meta-analysis. Eur J Gastroenterol Hepatol 27 (2015), 1438–1442.
53. Al-Hamoudi, N., Is antimicrobial photodynamic therapy an effective treatment for chronic periodontitis in diabetes mellitus and cigarette smokers: a systematic review and meta-analysis. Photodiagnosis Photodyn Ther 19 (2017), 375–382.
54. Hamley, S., The effect of replacing saturated fat with mostly n-6 polyunsaturated fat on coronary heart disease: a meta-analysis of randomised controlled trials. Nutr J, 16, 2017, 30.
55. Puljak, L., If there is only one author or only one database was searched, a study should not be called a systematic review. J Clin Epidemiol, 2017 http://dx.doi.org/10.1016/j.jclinepi.2017.08.002 [Epub ahead of print].
56. Dosenovic, S., Jelicic Kadic, A., Miljanovic, M., Biocic, M., Boric, K., Cavar, M., et al. Interventions for Neuropathic Pain: an overview of systematic reviews. Anesth Analg 125 (2017), 643–652.
57. Gajsak, L.R., Gelemanovic, A., Kuzman, M.R., Puljak, L., Impact of stress response in development of first-episode psychosis in schizophrenia: an overview of systematic reviews. Psychiatr Danub 29 (2017), 14–23.
58. Pollock, M., Fernandes, R.M., Hartling, L., Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions. BMC Med Res Methodol, 17, 2017, 48.
59. Whiting, P., Savović J., Higgins, J.P.T., Caldwell, D.M., Reeves, B.C., Shea, B., et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 69 (2016), 225–234.
60. Bühn, S., Mathes, T., Prengel, P., Wegewitz, U., Ostermann, T., Robens, S., et al. The risk of bias in systematic reviews (ROBIS) tool showed fair reliability and good construct validity. J Clin Epidemiol, 2017 http://dx.doi.org/10.1016/j.jclinepi.2017.06.019 [Epub ahead of print].
61. Ioannidis, J.P.A., Adverse events in randomized trials: neglected, restricted, distorted, and silenced. Arch Intern Med 169 (2009), 1737–1739.
62. Pitrou, I., Boutron, I., Ahmad, N., Ravaud, P., Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med 169 (2009), 1756–1761.
63. Jonsson, U., Alaie, I., Parling, T., Arnberg, F.K., Reporting of harms in randomized controlled trials of psychological interventions for mental and behavioral disorders: a review of current practice. Contemp Clin Trials 38 (2014), 1–8.
64. Hadi, M.A., McHugh, G.A., Conaghan, P.G., Quality of reporting of harms in randomised controlled trials of pharmacological interventions for rheumatoid arthritis: a systematic review. Evid Based Med 22 (2017), 170–177.
65. Hodkinson, A., Kirkham, J.J., Tudur-Smith, C., Gamble, C., Reporting of harms data in RCTs: a systematic review of empirical assessments against the CONSORT harms extension. BMJ Open, 3, 2013, e003436.
66. Haidich, A.B., Pilalas, D., Contopoulos-Ioannidis, D.G., Ioannidis, J.P.A., Most meta-analyses of drug interventions have narrow scopes and many focus on specific agents. J Clin Epidemiol 66 (2013), 371–378.
67. Lu, G., Ades, A.E., Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 23 (2004), 3105–3124.
68. Lee, A.W., Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. J Clin Epidemiol 67 (2014), 138–143.
69. Ioannidis, J.P.A., Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ 181 (2009), 488–493.
70. Cahill, K., Stevens, S., Perera, R., Lancaster, T., Pharmacological interventions for smoking cessation: an overview and network meta-analysis. Cochrane Database Syst Rev, 5, 2013, CD009329.
71. Xin, Y., Manson, J., Govan, L., Harbour, R., Bennison, J., Watson, E., et al. Pharmacological regimens for eradication of Helicobacter pylori: an overview of systematic reviews and network meta-analysis. BMC Gastroenterol, 16, 2016, 80.
72. Hutton, B., Salanti, G., Caldwell, D.M., Chaimani, A., Schmid, C.H., Cameron, C., et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med 162 (2015), 777–784.