Methods to calculate uncertainty in the estimated overall effect size from a random-effects meta-analysis

Research Synthesis Methods

Volumn 10, Issue 1, 2019, Pages 23-43

  Veroniki, Areti Angeliki ^a,b   Jackson, Dan ^c   Bender, Ralf ^d   Kuss, Oliver ^e   Langan, Dean ^f   Higgins, Julian P T ^g   Knapp, Guido ^h   Salanti, Georgia ⁱ  

^a ST MICHAEL'S HOSPITAL   (Canada)

^b UNIVERSITY OF IOANNINA   (Greece)

^c MRC BIOSTATISTICS UNIT   (United Kingdom)

^d INSTITUTE FOR QUALITY AND EFFICIENCY IN HEALTH CARE IQWIG   (Germany)

^e HEINRICH HEINE UNIVERSITY   (Germany)

^f UNIVERSITY COLLEGE LONDON   (United Kingdom)

^g UNIVERSITY OF BRISTOL   (United Kingdom)

^h TU DORTMUND UNIVERSITY   (Germany)

ⁱ UNIVERSITY OF BERN   (Switzerland)

Author keywords

confidence interval; evidence synthesis; meta analysis; overall treatment effect; random effects

Indexed keywords

ALGORITHM; BAYES THEOREM; COMPUTER SIMULATION; DECISION MAKING; HUMAN; META ANALYSIS (TOPIC); METHODOLOGY; RANDOMIZATION; REPRODUCIBILITY; SAMPLE SIZE; STATISTICAL ANALYSIS; STATISTICAL MODEL; UNCERTAINTY;

ALGORITHMS; BAYES THEOREM; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; DECISION MAKING; HUMANS; LIKELIHOOD FUNCTIONS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; RANDOM ALLOCATION; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; SAMPLE SIZE; UNCERTAINTY;

EID: 85061921326     PISSN: None     EISSN: 17592887     Source Type: Journal    
DOI: 10.1002/jrsm.1319     Document Type: Article

References (174)

1. da Costa BR, Juni P. Systematic reviews and meta-analyses of randomized trials: principles and pitfalls. Eur Heart J. 2014;35(47):3336-3345.
2. WHO. World Health Organization. http://www.who.int/en/. Accessed 08 February, 2017.
3. EMA. European Medicines Agency. http://www.ema.europa.eu/ema/. Accessed 08 February, 2017.
4. CADTH. Canadian Agency for Drugs and Technologies in Health. https://www.cadth.ca/. Accessed 08 February, 2017.
5. IQWiG. Institute for Quality and Efficiency in Health Care. https://www.iqwig.de/. Accessed 06 October 2017.
6. NICE. National Institute for Health and Clinical Excellence. https://www.nice.org.uk/. Accessed 08 February, 2017.
7. Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.
8. Chess LE, Gagnier JJ. Applicable or non-applicable: investigations of clinical heterogeneity in systematic reviews. BMC Med Res Methodol. 2016;16(1):19.
9. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011.
10. Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693-2708.
11. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188.
12. Cornell JE, Mulrow CD, Localio R, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160(4):267-270.
13. Brockwell SE, Gordon IR. A comparison of statistical methods for meta-analysis. Stat Med. 2001;20(6):825-840.
14. Hartung J, Knapp G. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Stat Med. 2001;20(24):3875-3889.
15. Sidik K, Jonkman JN. A simple confidence interval for meta-analysis. Stat Med. 2002;21(21):3153-3159.
16. IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol. 2014;14(1):25.
17. Noma H. Confidence intervals for a random-effects meta-analysis based on Bartlett-type corrections. Stat Med. 2011;30(28):3304-3312.
18. Poole C, Greenland S. Random-effects meta-analyses are not always conservative. Am J Epidemiol. 1999;150(5):469-475.
19. Longford NT. Estimation of the effect size in meta-analysis with few studies. Stat Med. 2010;29(4):421-430.
20. Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JP. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane database of systematic reviews. Int J Epidemiol. 2012;41(3):818-827.
21. Rhodes KM, Turner RM, Higgins JP. Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data. J Clin Epidemiol. 2015;68(1):52-60.
22. Davey J, Turner RM, Clarke MJ, Higgins JP. Characteristics of meta-analyses and their component studies in the Cochrane database of systematic reviews: a cross-sectional, descriptive analysis. BMC Med Res Methodol. 2011;11(1):160.
23. Kontopantelis E, Springate DA, Reeves D. A re-analysis of the Cochrane library data: the dangers of unobserved heterogeneity in meta-analyses. PLoS One. 2013;8(7):e69930.
24. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser a Stat Soc. 2009;172(1):137-159.
25. Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342(feb10 2):d549.
26. Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to meta-analysis. Chichester, UK: Wiley; 2009.
27. McKenzie JE, Beller EM, Forbes AB. Introduction to systematic reviews and meta-analysis. Respirology. 2016;21(4):626-637.
28. Bender R, Friede T, Koch A, et al. Methods for evidence synthesis in the case of very few studies. Res Synth Methods. 2018. epub ahead of print:1–11
29. Laird NM, Mosteller F. Some statistical methods for combining experimental results. Int J Technol Assess Health Care. 1990;6(1):5-30.
30. Henmi M, Copas JB. Confidence intervals for random effects meta-analysis and robustness to publication bias. Stat Med. 2010;29(29):2969-2983.
31. Schmidt FL, Oh IS, Hayes TL. Fixed- versus random-effects models in meta-analysis: model properties and an empirical comparison of differences in results. Br J Math Stat Psychol. 2009;62(Pt 1):97-128.
32. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods. 2010;1(2):97-111.
33. Hunter JE, Smith FL. Fixed effects vs. random effects meta-analysis models: implications for cumulative research knowledge. Int J Select Assess. 2000;8(4):275-292.
34. Marín-Martínez F, Sánchez-Meca J. Weighting by inverse variance or by sample size in random-effects meta-analysis. Educ Psychol Meas. 2009;70(1):56-73.
35. Sanchez-Meca J, Marín-Martínez F. Weighting by inverse variance or by sample size in meta-analysis: a simulation study. Educ Psychol Meas. 2016;58(2):211-220.
36. Singh K, Xie M, Strawderman W. Combining information from independent sources through confidence distribution. Ann Stat. 2005;33(1):159-183.
37. Xie M, Singh K, Strawderman WE. Confidence distributions and a unifying framework for meta-analysis. J Am Stat Assoc. 2011;106(493):320-333.
38. Simmonds MC, Higgins JP. A general framework for the use of logistic regression models in meta-analysis. Stat Methods Med Res. 2016;25(6):2858-2877.
39. Mawdsley D, Higgins JP, Sutton AJ, Abrams KR. Accounting for heterogeneity in meta-analysis using a multiplicative model-an empirical study. Res Synth Methods. 2017;8(1):43-52.
40. Stanley TD, Doucouliagos H. Neither fixed nor random: weighted least squares meta-analysis. Stat Med. 2015;34(13):2116-2127.
41. Bellio R, Guolo A. Integrated likelihood inference in small sample meta-analysis for continuous outcomes. Scand J Stat. 2016;43(1):191-201.
42. Hoaglin DC. We know less than we should about methods of meta-analysis. Res Synth Methods. 2015;6(3):287-289.
43. Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random effects. Stat Med. 1996;15(6):619-629.
44. Sidik K, Jonkman JN. A comparison of heterogeneity variance estimators in combining results of studies. Stat Med. 2007;26(9):1964-1981.
45. Sanchez-Meca J, Marin-Martinez F. Confidence intervals for the overall effect size in random-effects meta-analysis. Psychol Methods. 2008;13(1):31-48.
46. Kulinskaya E, Dollinger MB, Bjørkestøl K. Testing for homogeneity in meta-analysis I. The one-parameter case: standardized mean difference. Biometrics. 2011;67(1):203-212.
47. Kulinskaya E, Dollinger MB, Bjorkestol K. On the moments of Cochran's Q statistic under the null hypothesis, with application to the meta-analysis of risk difference. Res Synth Methods. 2011;2(4):254-270.
48. Bohning D, Malzahn U, Dietz E, Schlattmann P, Viwatwongkasem C, Biggeri A. Some general points in estimating heterogeneity variance with the DerSimonian-Laird estimator. Biostatistics. 2002;3(4):445-457.
49. Malzahn U, Böhning D, Holling H. Nonparametric estimation of heterogeneity variance for the standardised difference used in meta-analysis. Biometrika. 2000;87(3):619-632.
50. Veroniki AA, Jackson D, Viechtbauer W, et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016;7(1):55-79.
51. Langan D, Higgins JP, Simmonds M. Comparative performance of heterogeneity variance estimators in meta-analysis: a review of simulation studies. Res Synth Methods. 2017;8(2):181-198.
52. Huizenga HM, Visser I, Dolan CV. Testing overall and moderator effects in random effects meta-regression. Br J Math Stat Psychol. 2011;64(Pt 1):1-19.
53. Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Stat Med. 1997;16(7):753-768.
54. Preuß M, Ziegler A. A simplification and implementation of random-effects meta-analyses based on the exact distribution of Cochran's Q. Methods Inf Med. 2014;53(1):54-61.
55. Thomas N. OpenBUGS website. Overview. 2010; http://www.openbugs.net/w/Overview Accessed 3 September 2017.
56. Lunn DJ, Thomas ABN, Spiegelhalter D. WinBUGS—a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000;10(4):325-337.
57. Turner RM, Jackson D, Wei Y, Thompson SG, Higgins JP. Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis. Stat Med. 2015;34(6):984-998.
58. Röver C, Friede T. Bayesian random-effects meta-analysis. 2015; version 1.1:https://cran.r-project.org/web/packages/bayesmeta/index.html. Accessed October 27, 2017.
59. Borenstein M, Hedges L, Higgins JPT, Rothstein H. Comprehensive Meta-Analysis. 104 2005; version 2:https://www.meta-analysis.com/. Accessed 24 October 2017.
60. Kontopantelis E, Reeves D. MetaEasy: a meta-analysis add-in for Microsoft Excel. J Stat Softw. 2009;30(7):1-25.
61. Raudenbush S, Byrk A, Congdon R. HLM 6 for Windows. 2004; n. Accessed 28 September 2017.
62. Meta-DiSc statistical methods [Computer Dent Prog]. 2006.
63. MetaWin: Statistical Software for Meta-Analysis. Version 2 [Computer Program]. Sunderland, MA: Sinauer Associates; 2000.
64. Bax L, Yu LM, Ikeda N, Tsuruta H, Moons KG. Development and validation of MIX: comprehensive free software for meta-analysis of causal research data. BMC Med Res Methodol. 2006;6(1):50.
65. Rasbash J, Charlton C, Browne W, Healy M, Cameron B. MLwiN. Centre forMultilevelModelling. MLwin: A software package for fitting multilevel models 2014; nnn. Accessed 28 September 2017.
66. Wallace BC, Dahabreh IJT, T.A., Lau J, Trow P, Schmid CH. Closing the gap between methodologists and end-users: R as a computational back-end. J Stat Softw. 2012;49:1–15.
67. Centre TNC. Review Manager (RevMan). 2014; www.cochrane.org/. Accessed 28 October 2017.
68. What is R? The R Project for Statistical Comput Secur 2008; nnn. Accessed 29 September 2017.
69. Chung Y, Rabe-Hesketh S, Dorie V, Gelman A, Liu J. A nondegenerate penalized likelihood estimator for variance parameters in multilevel model. Psychometrika. 2013;78(4):685-709.
70. boot: Bootstrap R (S-Plus) functions [Computer Dent Prog]. 2016.
71. Davison A, Hinkley D. Bootstrap methods and their applications. Cambridge: Cambridge University Press; 1997.
72. Thomas A, O'Hara B, Ligges U, Sturtz S. Making BUGS open. R News. 2006;6(1):12-17.
73. Del Re A, Hoyt W. MAd: meta-analysis with mean differences R package version 0.8–2 2014; nn. Accessed 23 May 2018.
74. Doebler P. mada: Meta-analysis of diagnostic accuracy. R package version 0.5.8 2015; https://cran.r-project.org/web/packages/mada/mada.pdf. Accessed 23 May 2018.
75. Schwarzer G. meta: general package for meta-analysis. 2015; version 4.1–0:https://cran.r-project.org/web/packages/meta/meta.pdf. Accessed 1 September 2017.
76. van Valkenhoef G, Kuiper J. GEMTC: Network meta-analysis using Bayesian methods. R Package Version. 2016;0:8-2. https://cran.r-project.org/package=gemtc. Accessed 23 May 2018
77. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1-48.
78. Möbius T. Metagen: inference in meta analysis and meta regression. R Package Version. 2014;1:0. https://cran.r-project.org/package=metagen. Accessed 23 May 2018
79. Guolo A. Higher-order likelihood inference in meta-analysis and meta-regression. Stat Med. 2012;31(4):313-327.
80. Guolo A, Varin C. The R package metaLik for likelihood inference in meta-analysis. J Stat Softw. 2012;50(7):1-14.
81. Debray T, de Jong V. metamisc: Diagnostic and prognostic meta-analysis. R Package Version. 2017;0:1. https://cran.r-project.org/package=metamisc. Accessed 23 May 2018
82. Beath K. metaplus: an R package for the analysis of robust meta-analysis and meta-regression. R Journal. 2016. https://journal.r-project.org/archive/2016-1/beath.pdfS. Accessed 23 May 2018
83. Cheung MWL. Meta-analysis using. Structural Equation Modeling. 2016; version 0.9.8:https://cran.r-project.org/web/packages/metaSEM/metaSEM.pdf. Accessed 1 October 2017
84. Huizenga H, Visser I, Dolan C. Hypothesis testing in random effects meta-regression. Br J Math Stat Psychol. 2011;64(1):1-19.
85. Gasparrini A. Multivariate and univariate meta-analysis and meta-regression. 2015; version 0.4.7:https://cran.r-project.org/web/packages/mvmeta/mvmeta.pdf. Accessed 1 October 2017.
86. Chen H. mvtmeta: multivariate meta-analysis. R Package Version 1.0 2012; https://cran.r-project.org/package=mvtmeta. Accessed 23 May 2018.
87. Rücker G, Schwarzer G, Krahn K, König J. Network meta-analysis using frequentist methods. 2016; version 0.9–0:https://cran.r-project.org/web/packages/netmeta/netmeta.pdf. Accessed 29 September 2017.
88. Sturtz S, Ligges U, Gelman A. R2WinBUGS: a package for running WinBUGS from R. J Stat Softw. 2005;12(3):1-16.
89. Lumley T. rmeta: meta-analysis. R Package Version. 2012;2:16. https://cran.r-project.org/package=rmeta. Accessed 23 May 2018
90. SAS Institute Inc. SAS Software. 2003; http://www.sas.com/technologies/analytics/statistics/stat/. Accessed 3 October 2017.
91. marandom.sas. SAS® meta-analysis macros. http://www.senns.demon.co.uk/SAS%20Macros/SASMacros.html. Accessed 4 October 2017.
92. PROC IML. SAS/IML 9.22 User's guide. http://support.sas.com/documentation/cdl/en/imlug/63541/PDF/default/imlug.pdf. Accessed 29 October 2017.
93. Littell RC, Milliken GA, Stroup WW, Wolfinger RD. SAS system for mixed models. SAS Inst Inc, Cary, NC. 1996;31-63.
94. PROC MIXED. SAS/Stat 9.2 User's guide The MIXED procedure (Book Excerpt). https://support.sas.com/documentation/cdl/en/statugmixed/61807/PDF/default/statugmixed.pdf. Accessed 29 October 2017.
95. PROC GLIMMIX. SAS/Stat 9.2 User's guide The MIXED procedure (Book Excerpt). https://support.sas.com/documentation/cdl/en/statugmixed/61807/PDF/default/statugmixed.pdf. Accessed 29 October 2017.
96. Zhang Z, McArdle JJ, Wang L, Hamagami F. A SAS interface for Bayesian analysis with WinBUGS. Struct Equ Model Multidiscip J. 2008;15(4):705-728.
97. RASmacro. 2017; https://github.com/rsparapa/rasmacro.
98. PROC MCMC. SAS Support Site The power to know https://support.sas.com/en/support-home.html.
99. STATA. Stata Statistical Software 2013; https://www.stata.com/. Accessed 1 October 2017.
100. Huang L, Dziak J, Wagner A, Lanza S. LCA Bootstrap Stata Function Users' Guide (Version 1.0). University Park: The Methodology Center, Penn State; 2016.
101. Rabe-Hesketh S, Skrondal A, Pickles A. Maximum likelihood estimation of generalized linear models with covariate measurement error. Stata Journal. 2003;3(4):386-411.
102. Kontopantelis E, Reeves D. Metaan: random-effects meta-analysis. Stata Journal. 2010;10:395-407.
103. Harbord RM, Higgins JPT. Meta-regression in Stata. Stata J. 2008;8(4):493-519.
104. Harris R, Bradburn M, Deeks J, Harbord R, Altman D, Sterne J. Metan: fixed- and random-effects metaanalysis. Stata Journal. 2008;8:3-28.
105. White IR. Multivariate random-effects meta-analysis. Stata Journal. 2009;9:40-56.
106. Gutierrez RG, Carter S, Drukker DM. sg160: on boundary-value likelihood-ratio tests. College Station, TX: Stata 2001;10:269–273.
107. IBM Corp. IBM SPSS for Windows. 2013; http://www.spss.co.in/. Accessed 29 October 2017.
108. Wilson B D. Meta-analysis stuff. meanes.sps, 2010; http://mason.gmu.edu/~dwilsonb/ma.html. Accessed 3 October 2017.
109. Wilson B D. Meta-analysis stuff. metaf.sps 2010; http://mason.gmu.edu/~dwilsonb/ma.html. Accessed 3 October 2017.
110. Wilson B D. Meta-analysis stuff. metareg.sps, 2010; http://mason.gmu.edu/~dwilsonb/ma.html. Accessed 3 October 2017.
111. Thomas A. BUGS: A statistical modelling package. RTA/BCS Modular Languages Newsletter 1994; https://www.mrc-bsu.cam.ac.uk/software/bugs/the-bugs-project-latest-news/the-winbugs-project-references-to-use-when-citing-bugs/. Accessed 4 October 2017.
112. Polanin JR, Hennessy EA, Tanner-Smith EE. A review of meta-analysis packages in R. J Educ Behav Stat. 2017;42(2):206-242.
113. Langan D, Higgins JPT, Jackson D, et al. A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Res Synth Methods. 2018. Accepted
114. Petropoulou M, Mavridis D. A comparison of 20 heterogeneity variance estimators in statistical synthesis of results from studies: a simulation study. Stat Med. 2017;36(27):4266-4280.
115. Hartung J. An alternative method for meta-analysis. Biom J. 1999;41(8):901-916.
116. Hartung J, Knapp G. On tests of the overall treatment effect in meta-analysis with normally distributed responses. Stat Med. 2001;20(12):1771-1782.
117. Follmann DA, Proschan MA. Valid inference in random effects meta-analysis. Biometrics. 1999;55(3):732-737.
118. Hartung J, Makambi KH. Reducing the number of unjustified significant results in meta-analysis. Commun Stat Simul Comput. 2003;32(4):1179-1190.
119. Makambi KH. The effect of the heterogeneity variance estimator on some tests of treatment efficacy. J Biopharm Stat. 2004;14(2):439-449.
120. Sidik K, Jonkman JN. Robust variance estimation for random effects meta-analysis. Comput Stat Data Anal. 2006;50(12):3681-3701.
121. Sidik K, Jonkman JN. On constructing confidence intervals for a standardized mean difference in meta-analysis. Commun Stat Simul Comput. 2003;32(4):1191-1203.
122. Brockwell SE, Gordon IR. A simple method for inference on an overall effect in meta-analysis. Stat Med. 2007;26(25):4531-4543.
123. Sidik K, Jonkman JN. Simple heterogeneity variance estimation for meta-analysis. J Roy Stat Soc. 2005;54(2):367-384.
124. Jackson D. The significance level of the standard test for a treatment effect in meta-analysis. Stat Biopharm Res. 2009;1(1):92-100.
125. Knapp G, Hartung J. Improved tests for a random effects meta-regression with a single covariate. Stat Med. 2003;22(17):2693-2710.
126. Hartung J, Makambi KH. Positive estimation of the between-study variance in meta-analysis: theory and methods. S Afr Stat J. 2002;36(1):55-76.
127. Jackson D, Bowden J. A re-evaluation of the ‘quantile approximation method’ for random effects meta-analysis. Stat Med. 2009;28(2):338-348.
128. Jackson D, Riley RD. A refined method for multivariate meta-analysis and meta-regression. Stat Med. 2014;33(4):541-554.
129. Jackson D, Bowden J. Confidence intervals for the between-study variance in random-effects meta-analysis using generalised heterogeneity statistics: should we use unequal tails? BMC Med Res Methodol. 2016;16(1):118.
130. Jackson D. Confidence intervals for the between-study variance in random effects meta-analysis using generalised Cochran heterogeneity statistics. Res Synth Methods. 2013;4(3):220-229.
131. Viechtbauer W. Confidence intervals for the amount of heterogeneity in meta-analysis. Stat Med. 2007;26(1):37-52.
132. Wiksten A, Rücker G, Schwarzer G. Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Stat Med. 2016;35(15):2503-2515.
133. Jackson D, Law M, Rucker G, Schwarzer G. The Hartung-Knapp modification for random-effects meta-analysis: a useful refinement but are there any residual concerns? Stat Med. 2017;36(25):3923-3934.
134. Viechtbauer W, López-López JA, Sanchez-Meca J, Marin-Martinez F. A comparison of procedures to test for moderators in mixed-effects meta-regression models. Psychol Methods. 2015;20(3):360-374.
135. Röver C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Med Res Methodol. 2015;15(1):99.
136. Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics. 1997;53(3):983-997.
137. Reid N. The 2000 Wald memorial lectures. Aymptotics and the theory of inference. Ann Stat. 2003;31(6):1695-1731.
138. Bartlett MS. Properties of sufficiency and statistical tests. Proc Royal Soc A. 1937;160(901):268-282.
139. Skovgaard IM. An explicit large-deviation approximation to one-parameter tests. Ther Ber. 1996;2:145-165.
140. Sharma G, Mathew T. Higher order inference for the consensus mean in inter-laboratory studies. Biom J. 2011;53(1):128-136.
141. Guolo A, Varin C. Random-effects meta-analysis: the number of studies matters. Stat Methods Med Res. 2015;26(3):1500-1518.
142. Rukhin AL. Confidence regions and intervals for meta-analysis model parameters. Dent Tech. 2015;57(4):547-558.
143. Kosmidis I, Guolo A, Varin C. Improving the accuracy of likelihood-based inference in meta-analysis and meta-regression. Biometrika. 2017;asx001;104(3):751.
144. Doi SA, Barendregt JJ, Khan S, Thalib L, Williams GM. Advances in the meta-analysis of heterogeneous clinical trials I: the inverse variance heterogeneity model. Contemp Clin Trials. 2015;45(Pt A):130-138.
145. Biggerstaff BJ, Jackson D. The exact distribution of Cochran's heterogeneity statistic in one-way random effects meta-analysis. Stat Med. 2008;27(29):6093-6110.
146. Zeng D, Lin DY. On random-effects meta-analysis. Biometrika. 2015;102(2):281-294.
147. Switzer FS, Paese PW, Drasgow F. Bootstrap estimates of standard errors in validity generalization. Journal of Applied Psychology. 1992;77(2):123-129.
148. Efron B, Tibshirani R. An introduction to the bootstrap. New York: Chapman & Hall; 1993.
149. Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med. 2000;19(24):3417-3432.
150. Van Den Noortgate W, Onghena P. Parametric and nonparametric bootstrap methods for meta-analysis. Behav Res Methods. 2005;37(1):11-22.
151. Efron B. 10. Nonparametric confidence intervals. In: The Jackknife, the Bootstrap and Other Resampling Plans. Philadelphia: SIAM1982:75-90.
152. Higgins JP, Thompson SG. Controlling the risk of spurious findings from meta-regression. Stat Med. 2004;23(11):1663-1682.
153. Smith TC, Spiegelhalter DJ, Thomas A. Bayesian approaches to random-effects meta-analysis: a comparative study. Stat Med. 1995;14(24):2685-2699.
154. Thorlund K, Thabane L, Mills EJ. Modelling heterogeneity variances in multiple treatment comparison meta-analysis-are informative priors the better solution? BMC Med Res Methodol. 2013;13(1):2.
155. Friede T, Röver C, Wandel S, Neuenschwander B. Meta-analysis of few small studies in orphan diseases. Res Synth Methods. 2017;8(1):79-91.
156. Lambert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med. 2005;24(15):2401-2428.
157. Rhodes KM, Turner RM, White IR, Jackson D, Spiegelhalter DJ, Higgins JP. Implementing informative priors for heterogeneity in meta-analysis using meta-regression and pseudo data. Stat Med. 2016;35(29):5495-5511.
158. Jackson D, Bowden J, Baker R. How does the DerSimonian and Laird procedure for random effects meta-analysis compare with its more efficient but harder to compute counterparts? J Stat Plan Infer. 2010;140(4):961-970.
159. López-López JA, Botella J, Sánchez-Meca J, Marín-Martínez F. Alternatives for mixed-effects meta-regression models in the reliability generalization approach: a simulation study. J Educ Behav Stat. 2013;38(5):443-469.
160. Sidik K, Jonkman JN. Authors reply. Stat Med. 2004;23(1):159-162.
161. Paule RC, Mandel J. Consensus values and weighting factors. J Res Natl Inst Stand Technol. 1982;87(5):377-385.
162. Rukhin AL, Biggerstaff BJ, Vangel MG. Restricted maximum-likelihood estimation of a common mean and the Mandel-Paule algorithm. J Stat Plan Infer. 2000;83(2):319-330.
163. Thorlund K, Wetterslev J, Awad T, Thabane L, Gluud C. Comparison of statistical inferences from the DerSimonian-Laird and alternative random-effects model meta-analyses—an empirical assessment of 920 Cochrane primary outcome meta-analyses. Res Synth Methods. 2011;2(4):238-253.
164. Cochran WG. The combination of estimates from different experiments. Biometrics. 1954;10(1):101-129.
165. Veroniki AA, Mavridis D, Higgins JP, Salanti G. Characteristics of a loop of evidence that affect detection and estimation of inconsistency: a simulation study. BMC Med Res Methodol. 2014;14(1):106.
166. Gonnermann A, Framke T, Grosshennig A, Koch A. No solution yet for combining two independent studies in the presence of heterogeneity. Stat Med. 2015;34(16):2476-2480.
167. Friedrich T, Knapp G. Generalised interval estimation in the random effects meta regression model. Comput Stat Data Anal. 2013;64:165-179.
168. Bodnar O, Link A, Arendacka B, Possolo A, Elster C. Bayesian estimation in random effects meta-analysis using a non-informative prior. Stat Med. 2017;36(2):378-399.
169. Graham PL, Moran JL. Robust meta-analytic conclusions mandate the provision of prediction intervals in meta-analysis summaries. J Clin Epidemiol. 2012;65(5):503-510.
170. IntHout J, Ioannidis JP, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open. 2016;6(7):e010247.
171. Partlett C, Riley RD. Random effects meta-analysis: coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med. 2017;36(2):301-317.
172. Lee KJ, Thompson SG. Flexible parametric models for random-effects distributions. Stat Med. 2008;27(3):418-434.
173. Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a simulation study. Stat Methods Med Res. 2012;21(4):409-426.
174. Bowden J, Tierney JF, Copas AJ, Burdett S. Quantifying, displaying and accounting for heterogeneity in the meta-analysis of RCTs using standard and generalised Q statistics. BMC Med Res Methodol. 2011;11(1):41.