Pitfalls of systematic reviews and meta-analyses in imaging research1

Radiology

Volumn 277, Issue 1, 2015, Pages 13-21

  McInnes, Matthew D F ^a,b   Bossuyt, Patrick M M ^c  

^a UNIVERSITY OF OTTAWA   (Canada)

^b OTTAWA HOSPITAL RESEARCH INSTITUTE   (Canada)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL PRACTICE; CLINICAL STUDY; DIAGNOSTIC ACCURACY; EVIDENCE BASED PRACTICE; MEDICAL LITERATURE; MEDICAL RESEARCH; META ANALYSIS (TOPIC); NONHUMAN; PRACTICE GUIDELINE; PRIORITY JOURNAL; PUBLISHING; RADIOLOGY; REVIEW; SENSITIVITY ANALYSIS; STANDARDIZATION; SYSTEMATIC REVIEW (TOPIC); DIAGNOSTIC IMAGING; HUMAN; LITERATURE;

BIOMEDICAL RESEARCH; DIAGNOSTIC IMAGING; HUMANS; META-ANALYSIS AS TOPIC; REVIEW LITERATURE AS TOPIC;

EID: 84942421302     PISSN: 00338419     EISSN: 15271315     Source Type: Journal    
DOI: 10.1148/radiol.2015142779     Document Type: Review

References (69)

1. Mulrow CD. The medical review article: state of the science. Ann Intern Med 1987;106(3): 485-488.
2. Mulrow CD. Rationale for systematic reviews. BMJ 1994;309(6954):597-599.
3. Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. New York, NY: Wiley, 2008.
4. Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC. Meta-analyses of randomized controlled trials. N Engl J Med 1987;316(8):450-455.
5. Irwig L, Macaskill P, Glasziou P, Fahey M. Meta-analytic methods for diagnostic test accuracy. J Clin Epidemiol 1995;48(1):119-130; discussion 131-132.
6. Littenberg B, Moses LE. Estimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method. Med Decis Making 1993;13(4):313-321.
7. Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM; Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Ann Intern Med 2008; 149(12):889-897.
8. Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0.0. The Cochrane Collaboration, 2013. http://srdta. cochrane.org/handbook-dta-reviews. Edited September 13, 2013. Accessed November 2, 2014.
9. Tunis AS, McInnes MD, Hanna R, Esmail K. Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology 2013;269(2):413-426.
10. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA 1995;274(22):1800-1804.
11. Ebell MH, Siwek J, Weiss BD, et al. Simplifying the language of evidence to improve patient care: Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in medical literature. J Fam Pract 2004;53(2):111-120.
12. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med 2007;4(3):e78.
13. Radiological Society of North America. Radiology Publication Information for Authors. http://pubs.rsna.org/page/radiology/pia. Accessed November 14, 2014.
14. PRISMA. The PRISMA statement. http://www.prisma-statement.org/statement.htm. Published 2014. Accessed November 15, 2014.
15. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 2009;62(10):e1-e34.
16. University of York. International prospective register of systematic reviews (PROSPERO). http://www.crd.york.ac.uk/PROSPERO/. Accessed November 14, 2014.
17. Systematic Reviews Journal Web site. http://www.systematicreviewsjournal.com/. Accessed November 14, 2014.
18. de Vet HCW, Riphagen II, Aertgeerts B, Pewsner D. Chapter 7: Searching for studies. In: Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.4. The Cochrane Collaboration, 2008. http://srdta.cochrane.org/handbook-dta-reviews. Updated September 2008. Accessed November 8, 2014.
19. Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155(8):529-536.
20. The Cochrane Collaboration. The Cochrane Collaboration's tool for assessing risk of bias. http://ohg.cochrane.org/sites/ohg.cochrane. org/files/uploads/Risk%20of%20bias%20 assessment%20tool.pdf. Published 2014. Accessed November 15, 2014.
21. Macaskill P, Gatsonis C, Deeks JJ, Harbord RM, Takwoingi Y. Chapter 10: Analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0. The Cochrane Collaboration, 2010. http://srdta. cochrane.org/sites/srdta.cochrane.org/files/uploads/Chapter%2010%20-%20Version% 201.0.pdf. Released December 23, 2010. Accessed November 18, 2014.
22. Radiological Society of North America. Radiology: Evidence-based practice. http://pubs.rsna.org/topic/evidence-based-practi ce?SeriesKey=radiology&sortBy=Ppub. Accessed November 15, 2014.
23. Pickhardt PJ, Hassan C, Halligan S, Marmo R. Colorectal cancer: CT colonography and colonoscopy for detection-systematic review and meta-analysis. Radiology 2011;259(2):393-405.
24. McInnes MD, Kielar AZ, Macdonald DB. Percutaneous image-guided biopsy of the spleen: systematic review and meta-analysis of the complication rate and diagnostic accuracy. Radiology 2011;260(3):699-708.
25. Kiewiet JJ, Leeuwenburgh MM, Bipat S, Bossuyt PM, Stoker J, Boermeester MA. A systematic review and meta-analysis of diagnostic performance of imaging in acute cholecystitis. Radiology 2012;264(3):708-720.
26. Toor SS, Jaberi A, Macdonald DB, McInnes MD, Schweitzer ME, Rasuli P. Complication rates and effectiveness of uterine artery embolization in the treatment of symptomatic leiomyomas: a systematic review and meta-analysis. AJR Am J Roentgenol 2012;199(5):1153-1163.
27. Oba Y, Zaza T. Abandoning daily routine chest radiography in the intensive care unit: meta-analysis. Radiology 2010;255(2):386-395.
28. Schieda N, McInnes MD, Cao L. Diagnostic accuracy of segmental enhancement inversion for diagnosis of renal oncocytoma at biphasic contrast enhanced CT: systematic review. Eur Radiol 2014;24(6):1421-1429.
29. O'Connor OJ, McSweeney SE, McWilliams S, et al. Role of radiologic imaging in irritable bowel syndrome: evidence-based review. Radiology 2012;262(2):485-494.
30. Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med 2002;21(16):2313-2324.
31. Staunton M. Evidence-based radiology: steps 1 and 2-asking answerable questions and searching for evidence. Radiology 2007; 242(1):23-31.
32. Whiting P, Westwood M, Beynon R, Burke M, Sterne JA, Glanville J. Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies. J Clin Epidemiol 2011;64(6):602-607.
33. Beynon R, Leeflang MM, McDonald S, et al. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev 2013;9:MR000022.
34. Ritchie G, Glanville J, Lefebvre C. Do published search filters to identify diagnostic test accuracy studies perform adequately? Health Info Libr J 2007;24(3):188-192.
35. Leeflang MM, Scholten RJ, Rutjes AW, Reitsma JB, Bossuyt PM. Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies. J Clin Epidemiol 2006;59(3):234-240.
36. Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidencebased practice guideline for the peer review of electronic search strategies. J Clin Epidemiol 2009;62(9):944-952.
37. Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol 2002;31(1):115-123.
38. Morrison A, Polisena J, Husereau D, et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int J Technol Assess Health Care 2012;28(2):138-144.
39. Pham B, Klassen TP, Lawson ML, Moher D. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conven-tional or complementary. J Clin Epidemiol 2005;58(8):769-776. [Published correction appears in J Clin Epidemiol 2006;59(2):216.]
40. Benzies KM, Premji S, Hayden KA, Serrett K. State-of-the-evidence reviews: advantages and challenges of including grey literature. Worldviews Evid Based Nurs 2006;3(2): 55-61.
41. Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev 2007 (2):MR000010.
42. Citrome L. Beyond PubMed: searching the "grey literature" for clinical trial results. Innov Clin Neurosci 2014;11(7-8):42-46.
43. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007;7:10.
44. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3:25.
45. Whiting P, Harbord R, Kleijnen J. No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Med Res Methodol 2005;5:19.
46. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Radiology 2003;226(1):24-28.
47. Altman DG. Better reporting of randomised controlled trials: the CONSORT statement. BMJ 1996;313(7057):570-571.
48. Whiting PF, Rutjes AW, Westwood ME, Mallett S; QUADAS-2 Steering Group. A systematic review classifies sources of bias and variation in diagnostic test accuracy studies. J Clin Epidemiol 2013;66(10):1093-1104.
49. Wu MZ, McInnes MD, Macdonald DB, Kielar AZ, Duigenan S. CT in adults: systematic review and meta-analysis of interpretation discrepancy rates. Radiology 2014;270(3): 717-735.
50. Dubben HH, Beck-Bornholdt HP. Systematic review of publication bias in studies on publication bias. BMJ 2005;331(7514):433-434.
51. Sterne JA, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 2000;53(11):1119-1129.
52. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315(7109): 629-634.
53. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol 2005;58(9):882-893.
54. Borenstein M, Hedges LV, Higgins JPT, Rothstein H. Introduction to meta-analysis. Chichester, England: Wiley, 2009.
55. Lee YJ, Lee JM, Lee JS, et al. Hepatocellular carcinoma: diagnostic performance of multidetector CT and MR imaging-a systematic review and meta-analysis. Radiology 2015;275(1):97-109.
56. Harbord RM, Whiting P, Sterne JA, et al. An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary. J Clin Epidemiol 2008;61(11):1095-1103.
57. Macaskill P. Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. J Clin Epidemiol 2004;57(9): 925-932.
58. Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JA. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics 2007;8(2):239-251.
59. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58(10):982-990.
60. Lumbreras B, Parker LA, Porta M, Pollán M, Ioannidis JP, Hernández-Aguado I. Overinterpretation of clinical applicability in molecular diagnostic research. Clin Chem 2009;55(4):786-794.
61. Yavchitz A, Boutron I, Bafeta A, et al. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med 2012;9(9):e1001308.
62. Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM. Overinterpretation and misreporting of diagnostic accuracy studies: evidence of "spin". Radiology 2013;267(2):581-588.
63. Levine D, Bankier AA, Kressel HY. Spin in radiology research: let the data speak for themselves. Radiology 2013;267(2):324-325.
64. Bossuyt P, Davenport C, Deeks J, Hyde C, Leeflang M, Scholten R. Chapter 11: Interpreting results and drawing conclusions. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.9. The Cochrane Collaboration, 2013. http://srdta. cochrane.org/sites/srdta.cochrane.org/files/uploads/DTA%20Handbook%20Chapter%20 11%20201312.pdf. Released December 13, 2013. Accessed November 28, 2014.
65. Taylor T, Dineen RA, Gardiner DC, Buss CH, Howatson A, Pace NL. Computed tomography (CT) angiography for confirmation of the clinical diagnosis of brain death. Cochrane Database Syst Rev 2014;3:CD009694.
66. Josephson CB, White PM, Krishan A, Al-Shahi Salman R. Computed tomography angiography or magnetic resonance angiography for detection of intracranial vascular malformations in patients with intracerebral haemorrhage. Cochrane Database Syst Rev 2014;9:CD009372.
67. Equator Network. Reporting Guidelines under development. http://www.equator-network. org/library/reporting-guidelines-underdevelopment/#99. Accessed November 28, 2014.
68. International Society for Magnetic Resonance in Medicine. JMRI Instructions for Authors. http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1522-2586/homepage/ForAuthors.html. Accessed November 14, 2014.
69. van der Pol CB, McInnes MDF, Petrcich W, Tunis AS, Hanna R. Is quality and completeness of reporting of systematic reviews and meta-analyses published in high impact radiology journals associated with citation rates? PLOS One 2015; 10(3).