A Bayesian "fill-In" Method for Correcting for Publication Bias in Meta-Analysis

Psychological Methods

Volumn 22, Issue 4, 2017, Pages 799-817

  Du, Han ^a   Liu, Fang ^b   Wang, Lijuan ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF NOTRE DAME   (United States)

Author keywords

Bayesian estimation; meta analysis; missing data; publication bias

Indexed keywords

EFFECT SIZE; HUMAN; LIFE; META ANALYSIS; PRACTICE GUIDELINE; PUBLISHING; SIMULATION; STATISTICAL SIGNIFICANCE; SYSTEMATIC REVIEW; VALIDITY; BAYES THEOREM; META ANALYSIS (TOPIC); PROCEDURES; PSYCHOLOGY; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICS AND NUMERICAL DATA;

BAYES THEOREM; DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; PSYCHOLOGY; PUBLICATION BIAS;

EID: 85038638270     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/met0000164     Document Type: Article

References (74)

1. Becker, B. J. (1986). Influence again: An examination of reviews and studies of gender differences in social influence. In J. S. Hyde & M. C. Linn (Eds.), The psychology of gender: Advances through meta-analysis (pp. 178-209). London, United Kingdom: Johns Hopkins University Press.
2. Borenstein, M., Hedges, L. V., Higgins, J., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex, United Kingdom: Wiley.
3. Bozarth, J. D., & Roberts, R. R. (1972). Signifying significant significance. American Psychologist, 27, 774-775.
4. Chung, Y., Rabe-Hesketh, S., & Choi, I.-H. (2013). Avoiding zero between-study variance estimates in random-effects meta-analysis. Statistics in Medicine, 32, 4071-4089.
5. Coburn, K. M., & Vevea, J. L. (2015). Publication bias as a function of study characteristics. Psychological Methods, 20, 310-330.
6. Coburn, K. M., & Vevea, J. L. (2016). weightr: Estimating weight-function models for publication bias in R (Version 1.0.0) [Computer software manual. Retrieved from https://CRAN.R-project.org/package-weightr
7. Coughlin, C. G., Cohen, S. C., Mulqueen, J. M., Ferracioli-Oda, E., Stuckelman, Z. D., & Bloch, M. H. (2015). Meta-analysis: Reduced risk of anxiety with psychostimulant treatment in children with attentiondeficit/hyperactivity disorder. Journal of Child and Adolescent Psychopharmacology, 25, 611-617.
8. Dear, K. B., & Begg, C. B. (1992). An approach for assessing publication bias prior to performing a meta-analysis. Statistical Science, 237-245.
9. Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125, 627-667.
10. DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177-188.
11. Dickersin, K. (1990). The existence of publication bias and risk factors for its occurrence. Journal of the American Medical Association, 263, 1385-1389.
12. Dickersin, K. (1997). How important is publication bias? A synthesis of available data. AIDS Education and Prevention, 9, 15-21.
13. Dobson, A., & Dear, K. (1997). Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate-comments and rejoinders. Statistical Science, 12, 245-246.
14. DuMouchel, W., & Harris, J. (1997). Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate-comments and rejoinders. Statistical Science, 12, 244-245.
15. Duval, S., & Tweedie, R. (2000a). A nonparametric "trim and fill" method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89-98.
16. Duval, S., & Tweedie, R. (2000b). Trim and fill: A simple funnel-plotbased method of testing and adjusting for publication bias in metaanalysis. Biometrics, 56, 455-463.
17. Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629-634.
18. Enders, C. K. (2011). Missing not at random models for latent growth curve analyses. Psychological Methods, 16, 1.
19. Felson, D. T. (1992). Bias in meta-analytic research. Journal of Clinical Epidemiology, 45, 885-892.
20. Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods, 17, 120-128.
21. Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories publication bias and psychological science's aversion to the null. Perspectives on Psychological Science, 7, 555-561.
22. Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345, 1502-1505.
23. Gelfand, A. E., & Smith, A. F. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398-409.
24. Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper). Bayesian Analysis, 1, 515-534.
25. Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2014). Bayesian data analysis (Vol. 2). London, United Kingdom: Chapman & Hall.
26. Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 457-472.
27. Gilks, W. R., Richardson, S., & Spiegelhalter, D. J. (1996). Introducing Markov chain Monte Carlo. In W. R. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.), Markov chain Monte Carlo in practice (pp. 339-357). London, United Kingdom: Chapman & Hall.
28. Givens, G. H., Smith, D., & Tweedie, R. (1997). Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate. Statistical Science, 12, 221-240.
29. Glass, G. V., MacGaw, B., & Smith, M. L. (1984). Meta-analysis in social research. Beverly Hills, CA: Sage.
30. Guan, M., & Vandekerckhove, J. (2015). A Bayesian approach to mitigation of publication bias. Psychonomic Bulletin & Review, 1-13.
31. Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97-109.
32. Hedges, L. V. (1984). Estimation of effect size under nonrandom sampling: The effects of censoring studies yielding statistically insignificant mean differences. Journal of Educational and Behavioral Statistics, 9, 61-85.
33. Hedges, L. V. (1992). Modeling publication selection effects in metaanalysis. Statistical Science, 7, 246-255.
34. Hedges, L. V., & Olkin, I. (1985). Statistical method for meta-analysis. London, United Kingdom: Academic Press.
35. Hedges, L. V., & Vevea, J. L. (1996). Estimating effect size under publication bias: Small sample properties and robustness of a random effects selection model. Journal of Educational and Behavioral Statistics, 21, 299-332.
36. Hopewell, S., Loudon, K., Clarke, M., Oxman, A., & Dickersin, K. (2009). Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews, 1, 1-26.
37. Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods, 11, 193-206.
38. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
39. Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
40. Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107, 139-155.
41. Ioannidis, J. P., & Trikalinos, T. A. (2007). An exploratory test for an excess of significant findings. Clinical Trials, 4, 245-253.
42. Iyengar, S., & Greenhouse, J. B. (1988). Selection models and the file drawer problem. Statistical Science, 3, 109-117.
43. Jackson, D. (2006). The power of the standard test for the presence of heterogeneity in meta-analysis. Statistics in Medicine, 25, 2688-2699.
44. Jin, Z.-C., Zhou, X.-H., & He, J. (2015). Statistical methods for dealing with publication bias in meta-analysis. Statistics in Medicine, 34, 343-360.
45. Kicinski, M. (2013). Publication bias in recent meta-analyses. PloS One, 8, e81823.
46. Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005). How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using winbugs. Statistics in Medicine, 24, 2401-2428.
47. Langan, D., Higgins, J., & Simmonds, M. (2017). Comparative performance of heterogeneity variance estimators in meta-analysis: A review of simulation studies. Research Synthesis Methods, 8, 181-198. http://dx.doi.org/10.1002/jrsm.1198
48. Light, R., & Pillemer, D. (1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.
49. Little, R. J., & Rubin, D. B. (2014). Statistical analysis with missing data. Hoboken, NJ: Wiley.
50. Ma, X. (2015). Statistical methods for multivariate meta-analysis of diagnostic tests (Unpublished doctoral dissertation). University of Minnesota, MN.
51. Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9, 147-163.
52. Møller, A. P., & Jennions, M. D. (2001). Testing and adjusting for publication bias. Trends in Ecology & Evolution, 16, 580-586.
53. Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., Cooper, N. J (2009). Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Medical Research Methodology, 9, 2-20.
54. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599-620.
55. Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2007). Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Statistics in Medicine, 26, 4544-4562.
56. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638-641.
57. Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2006). Publication bias in meta-analysis: Prevention, assessment and adjustments. West Sussex, United Kingdom: Wiley.
58. Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581-592.
59. Schwarzer, G. (2015). meta: General package for meta-analysis [Computer software manual]. Retrieved from http://CRAN.R-project.org/package-meta
60. Silliman, N. P. (1997). Nonparametric classes of weight functions to model publication bias. Biometrika, 84, 909-918.
61. Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014a). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143, 534.
62. Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014b). P-curve and effect size correcting for publication bias using only significant results. Perspectives on Psychological Science, 9, 666-681.
63. Stanley, T. D. (2008). Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection. Oxford Bulletin of Economics and Statistics, 70, 103-127.
64. Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods, 5, 60-78.
65. Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance-or vice versa. Journal of the American Statistical Association, 54, 30-34.
66. Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R. (2000). Modelling publication bias in meta-analysis: A review. Statistical Methods in Medical Research, 9, 421-445.
67. Thompson, S. G. (1993). Controversies in meta-analysis: The case of the trials of serum cholesterol reduction. Statistical Methods in Medical Research, 2, 173-192.
68. van Aert, R. C. (2016). puniform: Meta-analysis with p-uniform (version 0.0.0) [Computer software manual]. Tilburg, the Netherlands: Author.
69. van Aert, R. C., Wicherts, J. M., & van Assen, M. A. (2016). Conducting meta-analyses based on p-values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11, 713-729. http://dx.doi.org/10.1177/1745691616650874
70. van Assen, M. A., van Aert, R., & Wicherts, J. M. (2015). Meta-analysis using effect size distributions of only statistically significant studies. Psychological Methods, 20, 293-309.
71. Vevea, J. L., & Hedges, L. V. (1995). A general linear model for estimating effect size in the presence of publication bias. Psychometrika, 60, 419-435.
72. Vevea, J. L., & Woods, C. M. (2005). Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychological Methods, 10, 428-443.
73. Viechtbauer, W. (2007). Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in Medicine, 26, 37-52.
74. Zhu, M., & Lu, A. Y. (2004). The counter-intuitive non-informative prior for the Bernoulli family. Journal of Statistics Education, 12, 1-10