Primer on systematic review and meta-analysis

Methods in Molecular Biology

Volumn 473, Issue , 2009, Pages 217-233

  Tonelli, Marcello ^a   Hackam, Dan ^b   Garg, Amit X ^c  

^a UNIVERSITY OF ALBERTA   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c UNIVERSITY OF WESTERN ONTARIO   (Canada)

Author keywords

meta analysis; publication bias; randomized controlled trials; study heterogeneity; Systematic review

Indexed keywords

CLINICAL TRIAL; EVIDENCE BASED MEDICINE; HUMAN; LITERATURE; META ANALYSIS; METHODOLOGY; REVIEW;

CLINICAL TRIALS AS TOPIC; EVIDENCE-BASED MEDICINE; HUMANS; META-ANALYSIS AS TOPIC; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC;

EID: 59649109788     PISSN: 10643745     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-1-59745-385-1_13     Document Type: Review

References (101)

1. U.S. National Library of Medicine Fact Sheet Medline. Available at www.nlm.nih. gov/pubs/factsheets/medline.html, last accessed March 3, 2007.
2. Ioannidis, J. P. (2005) Contradicted and initially stronger effects in highly cited clinical research. JAMA 294, 218-228. (Pubitemid 41043792)
3. Garg, A. X., Iansavichus, A. V., Kastner, M., Walters, L. A., Wilczynski, N., McKibbon, K. A., Yang, R. C., Rehman, F., Haynes, R. B. (2006) Lost in publication: Half of all renal practice evidence is published in non-renal journals. Kidney Int 70, 1995-2005. (Pubitemid 44863389)
4. Haynes, R. B., Cotoi, C., Holland, J., Walters, L., Wilczynski, N., Jedraszewski, D., McKinlay, J., Parrish, R., McKibbon, K. A. (2006) Second-order peer review of the medical literature for clinical practitioners. JAMA 295, 1801-1808.
5. Barrett, B. J., Parfrey, P. S. (2006) Clinical practice. Preventing nephropathy induced by contrast medium. N Engl J Med 354, 379-386. (Pubitemid 43145062)
6. Halloran, P. F. (2004) Immunosuppressive drugs for kidney transplantation. N Engl J Med 351, 2715-2729. (Pubitemid 39665325)
7. Schrier, R. W., Wang, W. (2004) Acute renal failure and sepsis. N Engl J Med 351, 159-169.
8. Cook, D. J., Mulrow, C. D., Haynes, R. B. (1997) Systematic reviews: Synthesis of best evidence for clinical decisions. Ann Intern Med 126, 376-380. (Pubitemid 27102522)
9. Oxman, A. D., Cook, D. J., Guyatt, G. H., Evidence-Based Medicine Working Group. (1994) Users' guides to the medical literature, VI. How to use an overview. JAMA 272, 1367-1371. (Pubitemid 24323620)
10. Lewis, S., Clarke, M. (2001) Forest plots: Trying to see the wood and the trees. BMJ 322, 1479-1480. (Pubitemid 32575420)
11. Lyman, G. H., Kuderer, N. M. (2005) The strengths and limitations of meta-analyses based on aggregate data. BMC Res Meth-odol 5, 14.
12. Simmonds, M. C., Higgins, J. P., Stewart, L. A., Tierney, J. F., Clarke, M., J., Thompson, S. G. (2005) Meta-analysis of individual patient data from randomized trials: A review of methods used in practice. Clin Trials 2, 209-217. (Pubitemid 40910470)
13. Schmid, C. H., Landa, M., Jafar, T. H., Giatras, I., Karim, T., Reddy, M., Stark, P. C., Levey, A.S. (2003) Constructing a database of individual clinical trials for longitudinal analysis. Control Clin Trials 24, 324-340. (Pubitemid 36570382)
14. Schmid, C. H., Stark, P. C., Berlin, J. A., Landais, P., Lau, J. (2004) Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. J Clin Epidemiol 57, 683-697. (Pubitemid 39195218)
15. Fouque, D., Laville, M., Haugh, M., Bois-sel, J. P. (1996) Systematic reviews and their roles in promoting evidence-based medicine in renal disease. Nephrol Dial Transplant 11, 2398-2401. (Pubitemid 27033767)
16. Campbell, M. K., Daly, C., Wallace, S. A., Cody, D. J., Donaldson, C., Grant, A. M., Khan, I. H., Lawrence, P., Vale, L., MacLeod, A. M. (2000) Evidence-based medicine in nephrology: Identifying and critically appraising the literature. Nephrol Dial Transplant 15, 1950-1955.
17. Cochrane Collaboration. Available at: www. cochrane.org/index.htm., last accessed March 3 2007.
18. Blettner, M., Sauerbrei, W., Schlehofer, B., Scheuchenpflug, T., Friedenreich, C. (1999) Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol. 28, 1-9. (Pubitemid 29086203)
19. Haynes, R. B., Devereaux, P. J., Guyatt, G. H. (2002) Physicians' and patients' choices in evidence based practice. BMJ 324, 1350. (Pubitemid 34628001)
20. Fones, C. S., Kua, E. H., Goh, L. G. (1998) What makes a good doctor? views of the medical profession and the public in setting priorities for medical education. Singapore Med J 39, 537-542. (Pubitemid 128520739)
21. Sterne, J. A., Egger, M., Smith, G. D. (2001) Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ 323, 101-105. (Pubitemid 32681979)
22. Simes, R. J. (1987) Confronting publication bias: A cohort design for meta-analysis. Stat Med 6, 11-29. (Pubitemid 17017356)
23. Easterbrook, P. J., Berlin, J. A., Gopalan, R., Matthews, D. R. (1991) Publication bias in clinical research. Lancet 337, 867-872.
24. Egger, M., Smith, G. D. (1998) Bias in location and selection of studies. BMJ 316, 61-66.
25. Dickersin, K., Min, Y. I., Meinert, C. L. (1992) Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 267, 374-378.
26. Stern, J. M., Simes, R. J. (1997) Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. BMJ 315, 640-645. (Pubitemid 27387174)
27. Pogue, J., Yusuf, S. (1998) Overcoming the limitations of current meta-analysis of randomised controlled trials. Lancet 351, 47-52. (Pubitemid 28020173)
28. Giatras, I., Lau, J., Levey, A. S., Angiotensin-Converting-Enzyme Inhibition and Progressive Renal Disease Study Group. (1997) Effect of angiotensin-converting enzyme inhibitors on the progression of nondiabetic renal disease: A meta-analysis of randomized trials. Ann Intern Med 127, 337-345. (Pubitemid 27362741)
29. Guyatt, G., Gutterman, D., Baumann, M. H., Addrizzo-Harris, D., Hylek, E. M., Phillips, B., Raskob, G., Lewis, S. Z., Schunemann, H. (2006) Grading strength of recommendations and quality of evidence in clinical guidelines: Report from an American College of Chest Physicians task force. Chest 129, 174-181. (Pubitemid 43265259)
30. Anello, C., Fleiss, J. L. (1995) Exploratory or analytic meta-analysis: Should we distinguish between them? J Clin Epidemiol. 48, 109-116.
31. Boudville, N., Prasad, G. V., Knoll, G., Muirhead, N., Thiessen-Philbrook, H., Yang, R. C., Rosas-Arellano, M. P., Housawi, A., Garg, A. X. (2006) Meta-analy-sis: Risk for hypertension in living kidney donors. Ann Intern Med 145, 185-196. (Pubitemid 46780564)
32. LeLorier, J., Gregoire, G., Benhaddad, A., Lapierre, J., Derderian, F. (1997) Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 337, 536-542. (Pubitemid 27352021)
33. Hackam, D. G., Redelmeier, D. A. (2006) Translation of research evidence from animals to humans. JAMA 296, 1731-1732. (Pubitemid 44547792)
34. Thompson, S. G., Higgins, J. P. (2002) How should meta-regression analyses be undertaken and interpreted? Stat Med 21, 1559-1573. (Pubitemid 34746063)
35. Colditz, G. A., Brewer, T. F., Berkey, C. S., Wilson, M. E., Burdick, E., Fineberg, H. V., Mosteller, F. (1994) Efficacy of BCG vaccine in the prevention of tuberculosis. Meta-analysis of the published literature. JAMA 271, 698-702. (Pubitemid 24061706)
36. Palma, S., Delgado-Rodriguez, M. (2005) Assessment of publication bias in meta-analyses of cardiovascular diseases. J Epide-miol.Community Health 59, 864-869. (Pubitemid 41360911)
37. Lau, J., Ioannidis, J. P., Schmid, C. H. (1998) Summing up evidence: One answer is not always enough. Lancet 351, 123-127. (Pubitemid 28026648)
38. Thompson, S. G. (1994) Why sources of heterogeneity in meta-analysis should be investigated. BMJ 309, 1351-1355. (Pubitemid 24345976)
39. Berlin, J. A. (1995) Invited commentary: Benefits of heterogeneity in meta-analysis of data from epidemiologic studies. Am J Epidemio. 142, 383-387.
40. Davey, S. G., Egger, M., Phillips, A. N. (1997) Meta-analysis. Beyond the grand mean? BMJ 315, 1610-1614.
41. Thompson, S. G., Higgins, J. P. (2005) Treating individuals, 4: Can meta-analysis help target interventions at individuals most likely to benefit? Lancet 365, 341-346. (Pubitemid 40139113)
42. Yusuf, S. (1997) Meta-analysis of randomized trials: Looking back and looking ahead. Control Clin Trials 18, 594-601. (Pubitemid 27520847)
43. Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., Stroup, D. F. (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM [Quality of Reporting of Meta-analyses] statement. Lancet 354, 1896-1900.
44. Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., Moher, D., Becker, B. J., Sipe, T. A., Thacker, S. B., Meta-analysis of Observa-tional Studies in Epidemiology (MOOSE) Group. (2000) Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA 283, 2008-2012. (Pubitemid 30214957)
45. Choi, P. T., Halpern, S. H., Malik, N., Jadad, A. R., Tramer, M. R., Walder, B. (2001) Examining the evidence in anesthesia literature: A critical appraisal of systematic reviews. Anesth Analg 92, 700-709. (Pubitemid 32185115)
46. Dixon, E., Hameed, M., Sutherland, F., Cook, D. J., Doig, C. (2005) Evaluating meta-analyses in the general surgical literature: A critical appraisal. Ann Surg 241, 450-459. (Pubitemid 40299721)
47. Kelly, K. D., Travers, A., Dorgan, M., Slater, L., Rowe, B. H. (2001) Evaluating the quality of systematic reviews in the emergency medicine literature. Ann Emerg. Med 38, 518-526. (Pubitemid 33041928)
48. Sacks, H. S., Reitman, D., Pagano, D., Kupelnick, B. (1996) Meta-analysis: An update. Mt. Sinai J Med 63, 216-224.
49. Assendelft, W. J., Koes, B. W., Knipschild, P. G., Bouter, L. M. (1995) The relationship between methodological quality and conclusions in reviews of spinal manipulation. JAMA 274, 1942-1948. (Pubitemid 26003066)
50. Jadad, A. R., McQuay, H. J. (1996) Meta-analyses to evaluate analgesic interventions: A systematic qualitative review of their methodology. J Clin Epidemiol. 49, 235-243. (Pubitemid 26123935)
51. Jadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., Moher, D. (1998) Methodology and reports of systematic reviews and meta-analyses: A comparison of Cochrane reviews with articles published in paper-based journals. JAMA 280, 278-280. (Pubitemid 28493907)
52. Bero, L. A., Rennie, D. (1996) Influences on the quality of published drug studies. Int J Technol. Assess. Health Care 12, 209-237 (Pubitemid 26195521)
53. Barnes, D. E., Bero, L. A. (1998) Why review articles on the health effects of passive smoking reach different conclusions. JAMA 279, 1566-1570. (Pubitemid 28233013)
54. Jadad, A. R., Moher, M., Browman, G. P., Booker, L., Sigouin, C., Fuentes, M., Stevens, R. (2000) Systematic reviews and meta-analyses on treatment of asthma: Critical evaluation. BMJ 320, 537-540. (Pubitemid 30175797)
55. Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., Klassen, T. P. (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352, 609-613. (Pubitemid 28380142)
56. Mrkobrada, M., Thiessen-Philbrook, H., Haynes, R. B., Iansavichus, A. V., Rehman, F., Garg, A. X. (2007) Improving the quality of renal systematic reviews. Submitted for publication.
57. Jorgensen, A. W., Hilden, J., Gotzsche, P. C. (2006) Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: Systematic review. BMJ 333, 782.
58. Katerndahl, D. A., Lawler, W. R. (1999) Variability in meta-analytic results concerning the value of cholesterol reduction in coronary heart disease: A meta-meta-analysis. Am J Epidemiol 149, 429-441. (Pubitemid 29100780)
59. Counsell, C. (1997) Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 127, 380-387. (Pubitemid 27362747)
60. Vickers, A., Goyal, N., Harland, R., Rees, R. (1998) Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 19, 159-166. (Pubitemid 28191970)
61. Gregoire, G., Derderian, F., Le Lorier, J. (1995) Selecting the language of the publications included in a meta-analysis: Is there a Tower of Babel bias? J Clin Epidemiol. 48, 159-163.
62. Egger, M., Zellweger-Zahner, T., Schneider, M., Junker, C., Lengeler, C., Antes, G. (1997) Language bias in randomised controlled trials published in English and German. Lancet 350, 326-329. (Pubitemid 27351115)
63. Moher, D., Pham, B., Klassen, T. P., Schulz, K. F., Berlin, J. A., Jadad, A. R., Liberati, A. (2000) What contributions do languages other than English make on the results of meta-analyses? J Clin Epidemiol. 53, 964-972.
64. Juni, P., Holenstein, F., Sterne, J., Bartlett, C., Egger, M. (2002) Direction and impact of language bias in meta-analyses of controlled trials: Empirical study. Int J Epidemiol. 31, 115-123. (Pubitemid 34284912)
65. Casas, J. P., Chua, W., Loukogeorgakis, S., Vallance, P., Smeeth, L., Hingorani, A. D., MacAllister, R. J. (2005) Effect of inhibitors of the renin-angiotensin system and other antihypertensive drugs on renal outcomes: Systematic review and meta-analysis. Lancet 366, 2026-2033. (Pubitemid 41773378)
66. Strippoli, G. F., Craig, M. C., Schena, F. P., Craig, J. C. (2006) Role of blood pressure targets and specific antihypertensive agents used to prevent diabetic nephropathy and delay its progression. J Am Soc Nephrol 17, S153-S155.
67. Subramanian, S., Venkataraman, R., Kellum, J. A. (2002) Influence of dialysis membranes on outcomes in acute renal failure: A meta-analysis. Kidney Int 62, 1819-1823. (Pubitemid 35286291)
68. Jaber, B. L., Lau, J., Schmid, C. H., Karsou, S. A., Levey, A. S., Pereira, B. J. (2002) Effect of biocompatibility of hemodialysis membranes on mortality in acute renal failure: A meta-analysis. Clin Nephrol 57, 274-282. (Pubitemid 34296388)
69. Teehan, G. S., Liangos, O., Lau, J., Levey, A. S., Pereira, B. J., Jaber, B. L. (2003) Dialysis membrane and modality in acute renal failure: Understanding discordant meta-analyses. Semin. Dial 16, 356-360. (Pubitemid 37347002)
70. Dickersin, K., Scherer, R., Lefebvre, C. (1994) Identifying relevant studies for systematic reviews. BMJ 309, 1286-1291. (Pubitemid 24340510)
71. Suarez-Almazor, M. E., Belseck, E., Homik, J., Dorgan, M., Ramos-Remus, C. (2000) Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials 21, 76-487.
72. Topfer, L. A., Parada, A., Menon, D., Noorani, H., Perras, C., Serra-Prat, M. (1999) Comparison of literature searches on quality and costs for health technology assessment using the MEDLINE and EMBASE databases. Int J Technol Assess Health Care 15, 297-303. (Pubitemid 29380440)
73. Minozzi, S., Pistotti, V., Forni, M. (2000) Searching for rehabilitation articles on MEDLINE and EMBASE. An example with cross-over design. Arch Phys Med Rehabil 81, 720-722. (Pubitemid 30349711)
74. Embase DataStar Datasheets. Available at http://ds.datastarweb.com/ds/ products/datastar/sheets/emed.htm, last accessed March 3, 2007.
75. Cheema, B. S., Singh, M. A. (2005) Exercise training in patients receiving maintenance hemodialysis: A systematic review of clinical trials. Am J Nephrol 25, 352-364. (Pubitemid 41160994)
76. Clarke, K. S., Klarenbach, S., Vlaicu, S., Yang, R. C., Garg, A. X. (2006) The direct and indirect economic costs incurred by living kidney donors-a systematic review. Nephrol Dial Transplant 21, 1952-1960. (Pubitemid 43985725)
77. Steinbrook, R. (2006) Searching for the right search-reaching the medical literature. N Engl J Med 354, 4-7. (Pubitemid 43048945)
78. Wilczynski, N. L., Haynes, R. B. (2002) Robustness of empirical search strategies for clinical content in MEDLINE. Proc. AMIA. Symp., 904-908.
79. Wilczynski, N. L., Walker, C. J., McKib-bon, K. A., Haynes, R. B. (1995) Reasons for the loss of sensitivity and specificity of methodologic MeSH terms and textwords in MEDLINE. Pro Ann Symp Comput Appl Med Care, 436-440.
80. Edwards, P., Clarke, M., DiGuiseppi, C., Pratap, S., Roberts, I., Wentz, R. (2002) Identification of randomized controlled trials in systematic reviews: Accuracy and reliability of screening records. Stat Med 21, 1635-1640. (Pubitemid 34746067)
81. Davidson, R. A. (1986) Source of funding and outcome of clinical trials. J Gen.Intern Med 1, 155-158.
82. Rochon, P. A., Gurwitz, J. H., Simms, R. W., Fortin, P. R., Felson, D. T., Minaker, K. L., Chalmers, T. C. (1994) A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med 154, 157-163.
83. Biondi-Zoccai, G. G., Lotrionte, M., Abbate, A., Testa, L., Remigi, E., Burzotta, F., Valgimigli, M., Romagnoli, E., Crea, F., Agostoni, P. (2006) Compliance with QUOROM and quality of reporting of overlapping meta-analyses on the role of acetylcysteine in the prevention of contrast associated nephropathy: Case study. BMJ 332, 202-209.
84. Egger, M., Davey, S. G., Schneider, M., Minder, C. (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315, 629-634. (Pubitemid 27387172)
85. Sutton, A. J., et al. (2000) BMJ 320, 1574-1577.
86. Berlin, J. A., University of Pennsylvania Meta-analysis Blinding Study Group. (1997) Does blinding of readers affect the results of meta-analyses? Lancet 350, 185-186. (Pubitemid 27303630)
87. Jadad, A. R., Moore, R. A., Carroll, D., Jenkin-son, C., Reynolds, D. J., Gavaghan, D. J., McQuay, H. J. (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17, 1-12. (Pubitemid 26120810)
88. Balk, E. M., Bonis, P. A., Moskowitz, H., Schmid, C. H., Ioannidis, J. P., Wang, C., Lau, J. (2002) Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 287, 2973-2982. (Pubitemid 34620001)
89. Balk, E. M., Lau, J., Bonis, P. A. (2005) Reading and critically appraising systematic reviews and meta-analyses: A short primer with a focus on hepatology. J Hepatol 43, 729-736. (Pubitemid 41253242)
90. Moher, D., Cook, D. J., Jadad, A. R., Tugwell, P., Moher, M., Jones, A., Pham, B., Klassen, T. P. (1999) Assessing the quality of reports of randomised trials: Implications for the conduct of meta-analyses. Health Technol Assess 3, 1-98.
91. Verhagen, A. P., de Vet, H. C., de Bie, R. A., Boers, M., van den Brandt, P. A. (2001) The art of quality assessment of RCTs included in systematic reviews. J Clin Epidemiol 54, 651-654. (Pubitemid 32594850)
92. Juni, P., Altman, D. G., Egger, M. (2001) Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 323, 42-46. (Pubitemid 32682008)
93. Devereaux, P. J., Choi, P. T., El Dika, S., Bhandari, M., Montori, V. M., Schune-mann, H. J., Garg, A. X., Busse, J. W., Heels-Ansdell, D., Ghali, W. A., Manns, B. J., Guyatt, G. H. (2004) An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 57, 1232-1236. (Pubitemid 40023142)
94. Moher, D., Jadad, A.R., Nichol, G., Penman, M., Tugwell, P., Walsh, S. (1995) Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Control Clin Trials 16, 62-73.
95. Schulz, K. F., Chalmers, I., Hayes, R. J., Altman, D. G. (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273, 408-412.
96. Laupacis, A., Wells, G., Richardson, W. S., Tugwell, P., Evidence-Based Medicine Working Group. (1994) Users' guides to the medical literature, V. How to use an article about prognosis. JAMA 272, 234-237. (Pubitemid 24209087)
97. Garg, A. X., Suri, R. S., Barrowman, N., Reh-man, F., Matsell, D., Rosas-Arellano, M. P., Sal-vadori, M, Haynes, R. B., Clark, W. F. (2003) Long-term renal prognosis of diarrhea-associated hemolytic uremic syndrome: A systematic review, meta-analysis, and meta-regression. JAMA 290, 1360-1370. (Pubitemid 37442634)
98. Deeks, J. J. (2002) Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 21, 1575-1600. (Pubitemid 34746064)
99. DerSimonian, R., Laird, N. (1986) Meta-analysis in clinical trials. Control Clin Trials 7, 177-188. (Pubitemid 17189972)
100. Hardy, R. J., Thompson, S. G. (1998) Detecting and describing heterogeneity in meta-analysis. Stat Med 17, 841-856. (Pubitemid 28185164)
101. Higgins, J. P., Thompson, S. G. (2002) Quantifying heterogeneity in a meta-analysis. Stat Med 21, 1539-1558. (Pubitemid 34746062)