Meta-analysis in evidence-based healthcare: A paradigm shift away from random effects is overdue

International Journal of Evidence-Based Healthcare

Volumn 15, Issue 4, 2017, Pages 152-160

  Doi, Suhail A R ^a   Furuya Kanamori, Luis ^a   Thalib, Lukman ^b   Barendregt, Jan J ^c  

^a QATAR UNIVERSITY   (Qatar)

^b Qatar University   (Qatar)

^c UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

Coverage; Heterogeneity; Inverse variance; Meta analysis; Random effects; Simulation

Indexed keywords

ANALYTIC METHOD; DECISION MAKING; HUMAN; META ANALYSIS; PUBLIC HEALTH SERVICE; RELIABILITY; SIMULATION; STATISTICAL ANALYSIS; SYNTHESIS; SYSTEMATIC REVIEW; EVIDENCE BASED PRACTICE; META ANALYSIS (TOPIC); RANDOMIZATION; STATISTICAL MODEL; STATISTICS AND NUMERICAL DATA;

DATA INTERPRETATION, STATISTICAL; EVIDENCE-BASED PRACTICE; HUMANS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; RANDOM ALLOCATION;

EID: 85039710361     PISSN: 17441595     EISSN: 17441609     Source Type: Journal    
DOI: 10.1097/XEB.0000000000000125     Document Type: Article

References (64)

1. Sheehan MC, Lam J. Use of systematic review and metaanalysis in environmental health epidemiology: a systematic review and comparison with guidelines. Curr Environ Health Rep 2015; 2: 272-83
2. Sauerland S, Seiler CM. Role of systematic reviews and meta-analysis in evidence-based medicine. World J Surg 2005; 29: 582-7
3. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997; 126: 376-80
4. O'Rourke K. An historical perspective on meta-analysis: dealing quantitatively with varying study results. J R Soc Med 2007; 100: 579-82
5. Bolland MJ, Grey A, Reid IR. The randomised controlled trial to meta-analysis ratio: original data versus systematic reviews in the medical literature. N Z Med J 2007; 120: U2804
6. Achana F, Hubbard S, Sutton A, Kendrick D, Cooper N. An exploration of synthesis methods in public health evaluations of interventions concludes that the use of modern statistical methods would be beneficial. J Clin Epidemiol 2014; 67: 376-90
7. Panesar SS, Bhandari M, Darzi A, Athanasiou T. Meta-analysis: a practical decision making tool for surgeons. Int J Surg 2009; 7: 291-6
8. Cochran WG. Problems arising in the analysis of a series of similar experiments. J R Stat Soc 1937; 4: 102-18
9. Cochran WL, Carroll SP. A sampling investigation of the efficiency of weighting inversely as the estimated variance. Biometrics 1953; 9: 447-59
10. Shuster JJ. Empirical vs natural weighting in random effects meta-analysis. Stat Med 2010; 29: 1259-65
11. Doi SA, Barendregt JJ, Khan S, Thalib L, Williams GM. Advances in the meta-analysis of heterogeneous clinical trials I: The inverse variance heterogeneity model. Contemp Clin Trials 2015; 45: 130-8
12. Efron B, Morris C. Stein's paradox in statistics. Sci Am 1977; 236: 119-27
13. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 2010; 1: 97-111
14. Schmidt FL, Oh IS, Hayes TL. Fixed-versus random-effects models in meta-analysis: model properties and an empirical comparison of differences in results. Br J Math Stat Psychol 2009; 62: 97-128
15. Field AP. The problems in using fixed-effects models of meta-analysis on real-world data. Understanding Stat 2003; 2: 105-24
16. Patsopoulos NA, Evangelou E, Ioannidis JP. Sensitivity of between-study heterogeneity in meta-analysis: proposed metrics and empirical evaluation. Int J Epidemiol 2008; 37: 1148-57
17. Senn S. Trying to be precise about vagueness. Stat Med 2007; 26: 1417-30
18. Senn S. The many modes of meta. Drug Inf J 2000; 34: 535-49
19. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc 2009; 172: 137-59
20. Doi SA, Thalib L. A quality-effects model for meta-analysis. Epidemiology 2008; 19: 94-100
21. Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q 2004; 82: 661-87
22. Higgins JP, Lane PW, Anagnostelis B, Anzures-Cabrera J, Baker NF, Cappelleri JC, et al. A tool to assess the quality of a meta-analysis. Res Synth Methods 2013; 4: 351-66
23. Bailit J, Garrett J. Comparison of risk-adjustment methodologies for cesarean delivery rates. Obstet Gynecol 2003; 102: 45-51
24. Schokkaert E, Van de Voorde C. Direct versus indirect standardization in risk adjustment. J Health Econ 2009; 28: 361-74
25. Doi SA, Barendregt JJ, Rao C. An updated method for risk adjustment in outcomes research. Value Health 2014; 17: 629-33
26. Overton RC. A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator variable effects. Psychol Methods 1998; 3: 354-79
27. Poole C, Greenland S. Random-effects meta-analyses are not always conservative. Am J Epidemiol 1999; 150: 469-75
28. Henmi M, Copas JB. Confidence intervals for random effects meta-analysis and robustness to publication bias. Stat Med 2010; 29: 2969-83
29. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 1997; 337: 536-42
30. Guolo A, Varin C. Random-effects meta-analysis: the number of studies matters. Stat Methods Med Res 2017; 26: 1500-18
31. Nuesch E, Juni P. Commentary: which meta-analyses are conclusive? Int J Epidemiol 2009; 38: 298-303
32. Salanti G, Ioannidis JP. Synthesis of observational studies should consider credibility ceilings. J Clin Epidemiol 2009; 62: 115-22
33. Al Khalaf MM, Thalib L, Doi SA. Combining heterogenous studies using the random-effects model is a mistake and leads to inconclusive meta-analyses. J Clin Epidemiol 2011; 64: 119-23
34. Noma H. Confidence intervals for a random-effects metaanalysis based on Bartlett-type corrections. Stat Med 2011; 30: 3304-12
35. Brockwell SE, Gordon IR. A simple method for inference on an overall effect in meta-analysis. Stat Med 2007; 26: 4531-43
36. Brockwell SE, Gordon IR. A comparison of statistical methods for meta-analysis. Stat Med 2001; 20: 825-40
37. Thorlund K, Wetterslev J, Awad T, Thabane L, Gluud C. Comparison of statistical inferences from the DerSimonian-Laird and alternative random-effects model metaanalyses-an empirical assessment of 920 Cochrane primary outcome meta-analyses. Res Synth Methods 2011; 2: 238-53
38. Rover C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Med Res Methodol 2015; 15: 99
39. IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol 2014; 14: 25
40. Jackson D, Bowden J. A re-evaluation of the 'quantile approximation method' for random effects meta-analysis. Stat Med 2009; 28: 338-48
41. Sanchez-Meca J, Marin-Martinez F. Confidence intervals for the overall effect size in random-effects meta-analysis. Psychol Methods 2008; 13: 31-48
42. Knapp G, Biggerstaff BJ, Hartung J. Assessing the amount of heterogeneity in random-effects meta-analysis. Biom J 2006; 48: 271-85
43. Sidik K, Jonkman JN. A simple confidence interval for metaanalysis. Stat Med 2002; 21: 3153-9
44. Henmi M, Copas JB, Eguchi S. Confidence intervals and P-values for meta-analysis with publication bias. Biometrics 2007; 63: 475-82
45. Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are nonnormally distributed: a simulation study. Stat Methods Med Res 2012; 21: 409-26
46. Malloy MJ, Prendergast LA, Staudte RG. Transforming the Model T: random effects meta-analysis with stable weights. Stat Med 2013; 32: 1842-64
47. Vemer P, Al MJ, Oppe M, Rutten-van Molken MP. A choice that matters? Simulation study on the impact of direct meta-analysis methods on health economic outcomes. Pharmacoeconomics 2013; 31: 719-30
48. Wiksten A, Rucker G, Schwarzer G. Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Stat Med 2016; 35: 2503-15
49. DerSimonian R, Kacker R. Random-effects model for metaanalysis of clinical trials: an update. Contemp Clin Trials 2007; 28: 105-14
50. Langan D, Higgins JP, Simmonds M. Comparative performance of heterogeneity variance estimators in metaanalysis: a review of simulation studies. Res Synth Methods 2017; 8: 181-98
51. Preuss M, Ziegler A. A simplification and implementation of random-effects meta-analyses based on the exact distribution of Cochran's Q. Methods Inf Med 2014; 53: 54-61
52. Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med 2014; 160: 267-70
53. Sidik K, Jonkman JN. A comparison of heterogeneity variance estimators in combining results of studies. Stat Med 2007; 26: 1964-81
54. McKenzie JE, Salanti G, Lewis SC, Altman DG. Metaanalysis and The Cochrane Collaboration: 20 years of the Cochrane Statistical Methods Group. Syst Rev 2013; 2: 80
55. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177-88
56. Hedges LV. A random effects model for effect sizes. Psychol Bull 1983; 93: 388-95
57. Stanley TD, Doucouliagos H. Neither fixed nor random: weighted least squares meta-analysis. Stat Med 2015; 34: 2116-27
58. Hodges JS. Random effects old and new. In: Hodges JS, editor. Richly parameterized linear models. Additive, time series, and spatial models using random effects. USA: Chapman and Hall/CRC; 2013; pp. 285-302
59. Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Stat Med 1997; 16: 753-68
60. Hodges JS, Clayton MK. Random effects old and new. 2011 Available at: http://www.biostat.umn.edu/~hodges/ PubH8492/PubH8492page.html. [Accessed 1 September 2017]
61. Detsky AS, Naylor CD, O'Rourke K, McGeer AJ, L'Abbe KA. Incorporating variations in the quality of individual randomized trials into meta-analysis. J Clin Epidemiol 1992; 45: 255-65
62. Morris CN, Lysy M. Shrinkage estimation in multilevel normal models. Stat Sci 2012; 27: 115-34
63. Doi SA, Barendregt JJ, Khan S, Thalib L, Williams GM. Simulation comparison of the quality effects and random effects methods of meta-analysis. Epidemiology 2015; 26: e42-4
64. Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat Med 2006; 25: 4279-92