How does under-reporting of negative and inconclusive results affect the false-positive rate in meta-analysis? A simulation study

BMJ Open

Volumn 4, Issue 8, 2014, Pages

  Kicinski, Michal ^a  

^a HASSELT UNIVERSITY   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

METHAMPHETAMINE;

ANALYTICAL ERROR; ARTICLE; CLINICAL EVALUATION; CLINICAL TRIAL (TOPIC); CONTENT VALIDITY; EFFECT SIZE; EMPIRICAL RESEARCH; FALSE POSITIVE RESULT; HUMAN; MEDICAL RESEARCH; META ANALYSIS (TOPIC); OUTCOME ASSESSMENT; PUBLISHING; SIMULATION; STATISTICAL CONCEPTS; STATISTICAL DISTRIBUTION; STATISTICAL PARAMETERS; LABORATORY DIAGNOSIS; STATISTICS AND NUMERICAL DATA;

BIOMEDICAL RESEARCH; FALSE POSITIVE REACTIONS; HUMANS; META-ANALYSIS AS TOPIC; PUBLICATION BIAS;

EID: 84928891523     PISSN: None     EISSN: 20446055     Source Type: Journal    
DOI: 10.1136/bmjopen-2014-004831     Document Type: Article

References (45)

1. Rosenthal R. File drawer problem and tolerance for null results. Psychol Bull 1979;86:638-41.
2. Chalmers L. Underreporting research is scientific misconduct. JAMA 1990;263:1405-8.
3. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990;263:1385-9. (Pubitemid 20069945)
4. Normand SL. Meta-analysis: formulating, evaluating, combining, and reporting. Stat Med 1999;18:321-59.
5. Borenstein M, Hedges L, Higgins JP, et al. Introduction to meta-analysis. West Sussex, England: Wiley & Sons, 2009.
6. Cooper H. Research synthesis and meta-analysis. Los Angeles, CA: Sage, 2010.
7. Rothstein H, Sutton A, Borenstein M. Publication bias in meta-analysis: prevention, assessment and adjustments. Chichester, West Sussex: Wiley, 2005.
8. Ahmed I, Sutton AJ, Riley RD. Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey. BMJ 2012;344:d7762.
9. Kirkham JJ, Dwan KM, Altman DG, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 2010;340:c365.
10. Nuesch E, Trelle S, Reichenbach S, et al. Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. BMJ 2010;341:c3515.
11. Zhang Z, Xu X, Ni H. Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care 2013;17:R2.
12. Dwan K, Altman DG, Arnaiz JA, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 2008;3:e3081.
13. Song F, Parekh-Bhurke S, Hooper L, et al. Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies. BMC Med Res Methodol 2009;9:79.
14. Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ 2005;331:19-22. (Pubitemid 40942535)
15. Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis. PLoS Med 2008;5:e191.
16. Kicinski M. Publication bias in recent meta-analyses. PLoS ONE 2013;8:e81823.
17. Hedges LV. Estimation of effect size under nonrandom sampling: the effects of censoring studies yielding statistically insignificant mean differences. J Educ Stat 1984;9:61-85.
18. Hedges L, Vevea J. Estimating effect size under publication bias: small sample properties and robustness of a random effects selection model. J Educ Behav Stat 1996;21:299-332.
19. Moreno S, Sutton A, Ades AE, et al. Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Med Res Methodol 2009;9:2-18.
20. Peters JL, Sutton AJ, Jones DR, et al. Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Stat Med 2007;26:4544-62. (Pubitemid 350020393)
21. Ioannidis JA, Cappelleri JC, Lau J. Issues in comparisons between meta-analyses and large trials. JAMA 1998;279:1089-93. (Pubitemid 28177817)
22. LeLorier J, Gregoire G, Benhaddad A, et al. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 1997;337:536-42. (Pubitemid 27352021)
23. Nuesch E, Juni P. Commentary: which meta-analyses are conclusive? Int J Epidemiol 2009;38:298-303.
24. Villar J, Carroli G, Belizan JM. Predictive ability of meta-analyses of randomised controlled trials. Lancet 1995;345:772-6.
25. Egger M, Smith G. Misleading meta-analysis. BMJ 1995;310:752-4.
26. Peters J, Sutton A, Jones D, et al. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006;295: 676-80. (Pubitemid 43209069)
27. Davey J, Turner RM, Clarke MJ, et al. Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Med Res Methodol 2011;11:160.
28. Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med 2006;25:3443-57. (Pubitemid 44575162)
29. Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta-analysis. Stat Med 2001;20:641-54. (Pubitemid 32166424)
30. Turner RM, Davey J, Clarke MJ, et al. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. Int J Epidemiol 2012;41:818-27.
31. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21:1539-58. (Pubitemid 34746062)
32. Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252-60.
33. Sterne JA, Sutton AJ, Ioannidis JP, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 2011;343:d4002.
34. Egger M, Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315:629-34. (Pubitemid 27387172)
35. Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess 2010;14:iii, ix-iii, 1-193.
36. Begg C, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994;50:1088-101.
37. Schwarzer G, Antes G, Schumacher M. A test for publication bias in meta-analysis with sparse binary data. Stat Med 2007;26:721-33. (Pubitemid 46206592)
38. Rucker G, Schwarzer G, Carpenter J. Arcsine test for publication bias in meta-analyses with binary outcomes. Stat Med 2008;27:746-63. (Pubitemid 351278817)
39. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000;56:455-63. (Pubitemid 30422636)
40. Thorlund K, Wetterslev J, Awad T, et al. Comparison of statistical inferences from the DerSimonian-Laird and alternative randomeffects model meta-analyses - an empirical assessment of 920 Cochrane primary outcome meta-analyses. Res Synth Methods 2011;2:238-53.
41. Kromney J, Rendina-Gobioff G. On knowing what we do not know: an empirical comparison of methods to detect publication bias in meta-analysis. Educ Psychol Meas 2006;66:357-73. (Pubitemid 43707933)
42. Hedges LV. Modeling publication selection effects in meta-analysis. Stat Sci 1992;7:246-55.
43. Silliman N. Hierarchical selection models with applications in meta-analysis. J Am Stat Assoc 1997;92:926-36.
44. Dear K, Begg C. An approach for assessing publication bias prior to performing a meta-analysis. Stat Sci 1992;7:237-45.
45. Rufibach K. Selection models with monotone weight functions in meta analysis. Biom J 2011;53:689-704.