A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses

Research Synthesis Methods

Volumn 10, Issue 1, 2019, Pages 83-98

  Langan, Dean ^a,g   Higgins, Julian P T ^b   Jackson, Dan ^c   Bowden, Jack ^b   Veroniki, Areti Angeliki ^d,e,f   Kontopantelis, Evangelos ^g   Viechtbauer, Wolfgang ^h   Simmonds, Mark ⁱ  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF BRISTOL   (United Kingdom)

^c ASTRAZENECA   (United Kingdom)

^d ST MICHAEL'S HOSPITAL   (Canada)

^e UNIVERSITY OF IOANNINA   (Greece)

^f IMPERIAL COLLEGE LONDON   (United Kingdom)

^g UNIVERSITY OF MANCHESTER   (United Kingdom)

^h MAASTRICHT UNIVERSITY   (Netherlands)

ⁱ UNIVERSITY OF YORK   (United Kingdom)

Author keywords

DerSimonian Laird; heterogeneity; random effects; REML; simulation

Indexed keywords

ALGORITHM; ANALYSIS OF VARIANCE; COMPARATIVE STUDY; COMPUTER SIMULATION; HUMAN; META ANALYSIS (TOPIC); METHODOLOGY; ODDS RATIO; PROBABILITY; REPRODUCIBILITY; SELECTION BIAS; SOFTWARE; STATISTICAL ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; ANALYSIS OF VARIANCE; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; HUMANS; LIKELIHOOD FUNCTIONS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; ODDS RATIO; PROBABILITY; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; SELECTION BIAS; SOFTWARE; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85062528296     PISSN: None     EISSN: 17592887     Source Type: Journal    
DOI: 10.1002/jrsm.1316     Document Type: Article

References (50)

1. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188.
2. Malzahn U, Böhning D, Holling H. Nonparametric estimation of heterogeneity variance for the standardised difference used in meta-analysis. Biometrika. 2000;87(3):619-632. https://doi.org/10.1093/biomet/87.3.619
3. Novianti PW, Roes KCB, van der Tweel I. Estimation of between-trial variance in sequential meta-analyses: a simulation study. Contemp Clin Trials. 2014;37(1):129-138. https://doi.org/10.1016/j.cct.2013.11.012
4. Sidik K, Jonkman JN. A comparison of heterogeneity variance estimators in combining results of studies. Stat Med. 2007;26(9):1964-1981. https://doi.org/10.1002/sim.2688
5. Veroniki AA, Jackson D, Viechtbauer W, et al. Methods to estimate heterogeneity variance and its uncertainty in meta-analysis. Res Syn Meth. 2015;7(1):55-79. https://doi.org/10.1002/jrsm.1164
6. Paule RC, Mandel J. Consensus values and weighting factors. J Res Natl Bur Stand. 1982;87(5):377-385.
7. Hartung J, Makambi KH. Reducing the number of unjustified significant results in meta-analysis. Commun Stat Simul Comput. 2003;32(4):1179-1190. https://doi.org/10.1081/SAC-120023884
8. Sidik K, Jonkman JN. Simple heterogeneity variance estimation for meta-analysis. J R Stat Soc Ser B Appl Stat. 2005;54(2):367-384. https://doi.org/10.1111/j.1467-9876.2005.00489.x
9. Harville DA. Maximum likelihood approaches to variance component estimation and to related problems. J Am Stat Assoc. 1977;72(358):320-338. https://doi.org/10.2307/2286796
10. Langan D, Higgins JPT, Simmonds M. An empirical comparison of heterogeneity variance estimators in 12,894 meta-analyses. Res Syn Meth. 2015;6(2):195-205. https://doi.org/10.1002/jrsm.1140
11. Langan D, Higgins JPT, Simmonds M. Comparative performance of heterogeneity variance estimators in meta-analysis: a review of simulation studies. Res Syn Meth. 2016;8(2):181-198. https://doi.org/10.1002/jrsm.1198
12. Panityakul T, Bumrungsup C, Knapp G. On estimating residual heterogeneity in random-effects meta-regression: a comparative study. J Stat Theory Appl. 2013;12(3):253-265.
13. Viechtbauer W. Bias and efficiency of meta-analytic variance estimators in the random-effects model. J Educ Behav Stat. 2005;30(3):261-293. https://doi.org/10.3102/10769986030003261
14. Rukhin AL, Biggerstaff BJ, Vangel MG. Restricted maximum likelihood estimation of a common mean and the Mandel-Paule algorithm. J Stat Plan Inference. 2000;83(2):319-330. https://doi.org/10.1016/S0378-3758(99)00098-1
15. Bhaumik DK, Amatya A, Normand SLT, et al. Meta-analysis of rare binary adverse event data. J Am Stat Assoc. 2012;107(498):555-567.
16. Knapp G, Hartung J. Improved tests for a random effects meta-regression with a single covariate. Stat Med. 2003;22(17):2693-2710.
17. Sanchez-Meca J, Marín-Martínez F. Confidence intervals for the overall effect size in random-effects meta-analysis. Psychol Methods. 2008;13(1):31-48.
18. Kontopantelis E, Springate DA, Reeves D. A re-analysis of the Cochrane library data: the dangers of unobserved heterogeneity in meta-analyses. PLoS One. 2013;8(7):e69930.
19. Chung Y, Rabe-Hesketh S, Choi IH. Avoiding zero between-study variance estimates in random-effects meta-analysis. Stat Med. 2013;32(23):4071-4089.
20. Hunter J, Schmidt F. Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Thousand Oaks, Calif.: SAGE Publications; 2004.
21. Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random effects. Stat Med. 1996;15(6):619-629.
22. Morris CN. Parametric empirical Bayes inference: theory and applications. J Am Stat Assoc. 1983;78(381):47-55.
23. DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemp Clin Trials. 2007;28(2):105-114. https://doi.org/10.1016/j.cct.2006.04.004
24. Bowden J, Tierney JF, Copas AJ, Burdett S. Quantifying, displaying and accounting for heterogeneity in the meta-analysis of RCTs using standard and generalised Q statistics. BMC Med Res Methodol. 2011;11(1):41.
25. Jackson D, Turner R, Rhodes K, Viechtbauer W. Methods for calculating confidence and credible intervals for the residual between-study variance in random effects meta-regression models. BMC Med Res Methodol. 2014;14(1):103.
26. Rukhin AL. Estimating heterogeneity variance in meta-analysis. J R Stat Soc Series B Stat Methodology. 2013;75(3):451-469.
27. Viechtbauer W. Conducting meta-analyses in R with the metaphor package. J Stat Softw. 2010;36(3):1-48.
28. Follmann DA, Proschan MA. Valid inference in random-effects meta-analysis. Biometrics. 1999;55(3):732-737.
29. Hartung J, Knapp G. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Stat Med. 2001;20(24):3875-3889.
30. Sidik K, Jonkman JN. A simple confidence interval for meta-analysis. Stat Med. 2002;21(21):3153-3159. https://doi.org/10.1002/sim.1262
31. Wiksten A, Rücker G, Schwarzer G. Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Stat Med. 2016;35(15):2503-2515. https://doi.org/10.1002/sim.6879
32. Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539-1558. https://doi.org/10.1002/sim.1186
33. Borenstein M, Hedges LV, Higgins JPT. Introduction to Meta-Analysis. Hoboken, NJ, USA: Wiley; 1999.
34. Smith TC, Spiegelhalter DJ, Thomas A. Bayesian approaches to random-effects meta-analysis: a comparative study. Stat Med. 1995;14(24):2685-2699.
35. Böhning D, Malzahn U, Dietz E, Schlattmann P, Viwatwongkasem C, Biggeri A. Some general points in estimating heterogeneity variance with the DerSimonian-Laird estimator. Biostatistics. 2002;3(4):445-457. https://doi.org/10.1093/biostatistics/3.4.445
36. Higgins JPT, Spiegelhalter DJ. Being sceptical about meta-analyses: a Bayesian perspective on magnesium trials in myocardial infarction. Int J Epidemiol. 2002;31(1):96-104. https://doi.org/10.1093/ije/31.1.96
37. Kontopantelis E, Reeves D. metaan: random-effects meta-analysis. Stata J. 2010;10(3):395.
38. Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a simulation study. Stat Methods Med Res. 2012;21(4):409-426.
39. Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a comparison between DerSimonian-Laird and restricted maximum likelihood. Stat Methods Med Res. 2012;21(6):657-659.
40. Bradburn MJ, Deeks JJ, Berlin JA, Russell Localio A. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med. 2007;26(1):53-77.
41. IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol. 2014;14(1):25.
42. Henmi M, Copas JB. Confidence intervals for random effects meta-analysis and robustness to publication bias. Stat Med. 2010;29(29):2969-2983. https://doi.org/10.1002/sim.4029
43. Cornell JE. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160(4):267-270. https://doi.org/10.7326/M13-2886
44. Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med. 2004;23(9):1351-1375. https://doi.org/10.1002/sim.1761
45. Stijnen T, Hamza TH, Ozdemir P. Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Stat Med. 2010;29(29):3046-3067.
46. Simmonds M, Higgins JPT. A general framework for the use of logistic regression models in meta-analysis. Stat Methods Med Res. 2016;25(6):2858-2877.
47. Jackson D, Law M, Stijnen T, Viechtbauer W, White IR. A comparison of 7 random-effects models for meta-analyses that estimate the summary odds ratio. Stat Med. 2018 (in press). doi: https://doi.org/10.1002/sim.7588;37(7):1059-1085.
48. Röver C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Med Res Methodol. 2015;15(1):99.
49. Davey J, Turner RM, Clarke MJ, Higgins JPT. Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Med Res Methodol. 2011;11(1):160. https://doi.org/10.1186/1471-2288-11-160
50. Simmonds M, Stewart G, Stewart L. A decade of individual participant data meta-analyses: a review of current practice. Contemp Clin Trials. 2015;45:76-83.