Attention should be given to multiplicity issues in systematic reviews

Journal of Clinical Epidemiology

Volumn 61, Issue 9, 2008, Pages 857-865

  Bender, Ralf ^a,b   Bunce, Catey ^c   Clarke, Mike ^d   Gates, Simon ^e   Lange, Stefan ^a   Pace, Nathan L ^f   Thorlund, Kristian ^g  

^a INSTITUTE FOR QUALITY AND EFFICIENCY IN HEALTH CARE IQWIG   (Germany)

^b UNIVERSITY OF COLOGNE   (Germany)

^c MOORFIELDS EYE HOSPITAL   (United Kingdom)

^d UK COCHRANE CENTRE   (United Kingdom)

^e UNIVERSITY OF WARWICK   (United Kingdom)

^f UNIVERSITY OF UTAH SCHOOL OF MEDICINE   (United States)

^g Mental Health Services CPH   (Denmark)

Author keywords

Cumulative meta analysis; Meta analysis; Multiple comparisons; Simultaneous statistical inference; Statistical data interpretation; Systematic reviews

Indexed keywords

ANALGESIC AGENT; BETA ADRENERGIC RECEPTOR BLOCKING AGENT; CORTICOSTEROID; CYCLOPLEGIC AGENT; LOCAL ANESTHETIC AGENT; N METHYL DEXTRO ASPARTIC ACID RECEPTOR BLOCKING AGENT; NONSTEROID ANTIINFLAMMATORY AGENT; OPIATE; ROFECOXIB;

CLINICAL RESEARCH; CLINICAL TRIAL; COMPARATIVE STUDY; DATA ANALYSIS; DEEP VEIN THROMBOSIS; HUMAN; OPEN ANGLE GLAUCOMA; OUTCOMES RESEARCH; POSTOPERATIVE PAIN; PRACTICE GUIDELINE; PRIORITY JOURNAL; REVIEW; STATISTICAL SIGNIFICANCE; TRABECULOPLASTY;

BIAS (EPIDEMIOLOGY); BIOMEDICAL RESEARCH; DATA INTERPRETATION, STATISTICAL; GUIDELINES AS TOPIC; HUMANS; META-ANALYSIS AS TOPIC; REVIEW LITERATURE AS TOPIC;

EID: 48349096183     PISSN: 08954356     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2008.03.004     Document Type: Review

References (42)

1. Bauer P. Multiple testing in clinical trials. Stat Med 10 (1991) 871-890
2. Ottenbacher K.J. Quantitative evaluation of multiplicity in epidemiology and public health research. Am J Epidemiol 147 (1998) 615-619
3. Biester K, Lange S. The multiplicity problem in systematic reviews [Abstract]. XIII Cochrane Colloquium, Melbourne, 22-26 October 2005. Program and Abstracts; 2005: 153. Available at http://www.cochrane.org/colloquia/abstracts/melbourne/P-155.htm. Accessed May 13, 2008.
4. Bender R., and Lange S. Adjusting for multiple testing-when and how?. J Clin Epidemiol 54 (2001) 343-349
5. Cook R.J., and Dunnett C.W. Multiple comparisons. In: Armitage P., and Colton T. (Eds). Encyclopedia of biostatistics (2005), Wiley, Chichester 3383-3393
6. Dmitrienko A., and Hsu J.C. Multiple testing in clinical trials. In: Kotz S., Read C.B., Balakrishnan N., and Vidakovic B. (Eds). Encyclopedia of statistical sciences (2006), Wiley, Hoboken, New Jersey 5111-5117
7. Chen T., and Hoppe F.M. Simultaneous confidence intervals. In: Armitage P., and Colton T. (Eds). Encyclopedia of biostatistics (2005), Wiley, Chichester 4953-4955
8. Bland J.M., and Altman D.G. Multiple significance tests: the Bonferroni method. BMJ 310 (1995) 170
9. Rabiu M., Alhassan M.B., and Agbabiaka I.O. Interventions for Mooren's ulcer (Protocol). Cochrane Database Syst Rev 3 (2006) CD006131
10. Rolim de Moura C., and Paranhos Jr. A. Laser trabeculoplasty for open angle glaucoma (Protocol). Cochrane Database Syst Rev 4 (2002) CD003919
11. van Houwelingen H.C., Arends L.R., and Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med 21 (2002) 589-624
12. Arends L.R., Voko Z., and Stijnen T. Combining multiple outcome measures in a meta-analysis: an application. Stat Med 22 (2003) 1335-1353
13. Riley R.D., Abrams K.R., Sutton A.J., Lambert P.C., and Thompson J.R. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Med Res Methodol 7 (2007) 3
14. Ishak K.J., Platt R.W., Joseph L., and Hanley J.A. Impact of approximating or ignoring within-study covariances in multivariate meta-analyses. Stat Med 27 (2008) 670-686
15. Ong C.K., Lirk P., Seymour R.A., and Jenkins B.J. The efficacy of preemptive analgesia for acute postoperative pain management: a meta-analysis. Anesth Analg 100 (2005) 757-773
16. Hsu J.C. Multiple comparisons: Theory and methods (1996), Chapman & Hall, Boca Raton, FL
17. Hasselblad V. Meta-analysis of multitreatment studies. Med Decis Making 18 (1998) 37-43
18. Riley R.D., Abrams K.R., Lambert P.C., Sutton A.J., and Thompson J.R. An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Stat Med 26 (2007) 78-97
19. Bouza C., Lopez T., Magro A., Navalpotro L., and Amate J.M. Systematic review and meta-analysis of transurethral needle ablation in symptomatic benign prostatic hyperplasia. BMC Urol 6 (2006) 14
20. Parmar M.K., Torri V., and Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med 17 (1998) 2815-2834
21. Dear K.B.G. Iterative generalized least squares for meta-analysis of survival data at multiple times. Biometrics 50 (1994) 989-1002
22. Deeks J.J., and Altman D.G. Effect measures for meta-analysis of trials with binary outcomes. In: Egger M., Davey Smith G., and Altman D.G. (Eds). Systematic reviews in health care: Meta-analysis in context (2001), BMJ Books, London 313-335
23. Clarke M., Hopewell S., Juszczak E., Eisinga A., and Kjeldstrom M. Compression stockings for preventing deep vein thrombosis in airline passengers. Cochrane Database Syst Rev 2 (2006) CD004002
24. Sweeting M.J., Sutton A.J., and Lambert P.C. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 23 (2004) 1351-1375
25. Davey Smith G., and Egger M. Going beyond the grand mean: subgroup analysis in meta-analysis of randomised trials. In: Egger M., Davey Smith G., and Altman D.G. (Eds). Systematic reviews in health care: Meta-analysis in context (2001), BMJ Books, London 143-156
26. Pocock S.J., Assmann S.E., Enos L.E., and Kasten L.E. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 21 (2002) 2917-2930
27. Austin P.C., Mamdani M.M., Juurlink D.N., and Hux J.E. Testing multiple statistical hypotheses resulted in spurious associations: a study of astrological signs and health. J Clin Epidemiol 59 (2006) 964-969
28. Roberts D., and Dalziel S. Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth. Cochrane Database Syst Rev 3 (2006) CD004454
29. Brookes S.T., Whitley E., Peters T.J., Mulheran P.A., Egger M., and Davey Smith G. Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. Health Technol Assess 5 (2001) 1-56
30. In: Higgins J.P.T., and Green S. (Eds). Cochrane handbook for systematic reviews of interventions 4.2.6 [updated September 2006]. The Cochrane Library, Issue 4, 2006 (2006), Wiley, Chichester, UK
31. Lau J., Antman E.M., Jimenez-Silva J., Kupelnick B., Mosteller F., and Chalmers T.C. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 327 (1992) 248-254
32. Lau J., Schmid C.H., and Chalmers T.C. Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care. J Clin Epidemiol 48 (1995) 45-57
33. McPherson K. Statistics: The problem of examining accumulating data more than once. N Engl J Med 290 (1974) 501-502
34. O'Brien P.C. Data and safety monitoring. In: Armitage P., and Colton T. (Eds). Encyclopedia of biostatistics (2005), Wiley, Chichester 1362-1371
35. Pogue J.M., and Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Control Clin Trials 18 (1997) 580-593
36. Devereaux P.J., Beattie W.S., Choi P.T., Badner N.H., Guyatt G.H., Villar J.C., et al. How strong is the evidence for the use of perioperative beta blockers in non-cardiac surgery? Systematic review and meta-analysis of randomised controlled trials. BMJ 331 (2005) 313-321
37. Wetterslev J., Thorlund K., Brok J., and Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol 61 (2008) 64-75
38. Whitehead A. A prospectively planned cumulative meta-analysis applied to a series of concurrent clinical trials. Stat Med 16 (1997) 2901-2913
39. Lan K.K.G., Hu M., and Cappalleri J.C. Applying the law of iterated logarithm to cumulative meta-analysis of a continuous endpoint. Statistica Sinica 13 (2003) 1135-1145
40. Hu M., Cappelleri J.C., and Lan K.K. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clin Trials 4 (2007) 329-340
41. Jüni P., Nartey L., Reichenbach S., Sterchi R., Dieppe P.A., and Egger M. Risk of cardiovascular events and rofecoxib: cumulative meta-analysis. Lancet 364 (2004) 2021-2029
42. Ioannidis J.P.A., Contopoulos-Ioannidis D.G., and Lau J. Recursive cumulative meta-analysis: a diagnostic for the evolution of total randomized evidence from group and individual patient data. J Clin Epidemiol 52 (1999) 281-291