Publication bias as a threat to the validity of meta-analytic results

Journal of Experimental Criminology

Volumn 4, Issue 1, 2008, Pages 61-81

  Rothstein, Hannah R ^a  

^a CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

Failsafe N; Funnel plot; Meta analysis; Publication bias; Trim and fill; Validity

Indexed keywords

EID: 40249113327     PISSN: 15733750     EISSN: 15728315     Source Type: Journal    
DOI: 10.1007/s11292-007-9046-9     Document Type: Article

References (53)

1. Begg, C. B. (1994). Publication Bias. In H. Cooper & L. Hedges (Eds.), The Handbook of Research Synthesis, NY: Russell Sage.
2. Begg, C. B., & Berlin, J. A. (1988). Publication bias: A problem in interpreting medical data. Journal of the Royal Statistical Society A, 151, 419-463.
3. Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088-1101.
4. Borenstein, M. (2005). Software for publication bias. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley.
5. Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2005). Comprehensive Meta Analysis, Version 2. Biostat, Englewood, NJ.
6. Braga, A. (2005). Hot Spots Policing and Crime Prevention: A Systematic Review of Randomized Controlled Trials. Journal of Experimental Criminology, 1, 3.
7. Chan, A-W., & Altman, D. G. (2005). Outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ, 330, 753.
8. Chan, A-W., Hrobjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. Journal of the American Medical Association, 291, 2457-2465.
9. Cooper, H. M. (1979). Statistically combining independent studies: A meta-analysis of sex differences in conformity research. Journal of Personality and Social Psychology, 37, 131-146.
10. Copas, J. B. (1999). What works? Selectivity models and meta-analysis. Journal of the Royal Statistical Association, 162, 95-109.
11. Copas, J. B., & Shi, J. Q. (2001). A sensitivity analysis for publication bias in systematic reviews. Statistical Methods in Medical Research, 10, 251-265.
12. Deffenbacher, K., Bornstein, B., Penrod, S., & McGorty, K. (2004). Meta-Analytic Review of the Effects of High Stress on Eyewitness Memory. Law and Human Behavior, 28, 687-706.
13. Del Vecchio, T., & O'Leary, K. D. (2004). The effectiveness of anger treatments for specific anger problems: A meta-analytic review. Clinical Psychology Review, 24, 15-34.
14. Dickersin, K. (2005). Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments (pp. 11-34). Chichester, UK: Wiley.
15. Dreznick, M. (2003). Heterosocial competence of rapists and child molesters: a meta-analysis. Journal of Sex Research, 40, 170-178.
16. Duval, S. (2005). The "trim and fill" method. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments (pp. 127-144). Chichester, UK: Wiley.
17. Duval, S. J., & Tweedie, R. L. (2000a). A non-parametric "trim and fill" method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89-98.
18. Duval, S. J., & Tweedie, R. L. (2000b). Trim and Fill: A simple funnel plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 276-284.
19. Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315, 629-634.
20. Egger, M., Davey Smith, G., & Altman, D. G. (2000). Systematic reviews in health care: Meta-analysis in context. London: BMJ Books.
21. Hedges, L. V. (1992). Modeling publication selection effects in meta-analysis. Statistical Science, 7, 246-255.
22. Hedges, L. V., & Vevea, J. L. (1996). Estimating effect size under publication bias: Small sample properties and robustness of a random effects selection model. Journal of Educational and Behavioral Statistics, 21, 299-332.
23. Hedges, L., & Vevea, J. (2005). The selection model approach to publication bias. In H. Rothstein, A.J. Sutton, & M. Borenstein (Eds.). Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley.
24. Illescas, S., Sanchez-Meca, J., & Garrido, V. (2001). Treatment of offenders and recidivism: Assessment of the effectiveness of programmes applied in Europe. Psychology in Spain, 5, 47-62.
25. Ioannidis, J. (2005). Differentiating biases from genuine heterogeneity. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.). Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley.
26. Krleža-Jeric, K., Chan, A-W., Dickersin, K., Sim, I., Grimshaw, J., Gluud, C., for the Ottawa Group. (2005) Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa Statement (part 1). BMJ, 330, 956-958.
27. Landenberger, N., & Lipsey, M. W. (2005). The positive effects of cognitive-behavioral programs for offenders: A meta-analysis of factors associated with associated with effective treatment. Journal of Experimental Criminology, 1, 451-476.
28. Lau, J., Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992). Cumulative meta-analysis of therapeutic trials for myocardial infarction. New England Journal of Medicine, 327, 248-254.
29. Light, R., & Pillemer, D. (1984). Summing Up: the Science of Reviewing Research. Cambridge, MA: Harvard University Press.
30. Light, R. J., Singer, J. D., & Willett, J. B. (1994). The visual presentation and interpretation of meta-analyses. In H. M. Cooper, & L. V. Hedges (Eds.), The Handbook of Research Synthesis. New York: Russell Sage Foundation.
31. Lipsey, M. W. (2003). Those confounded moderators in meta-analysis: Good, bad, and ugly. Annals of the American Academy of Political and Social Science, 587, 69-81.
32. Losel, F., & Schmucker, M. (2005). The effectiveness of treatment for sexual offenders: A comprehensive meta-analysis.
33. McCord, J. (2003). Cures that harm: Unanticipated outcomes of crime prevention programs. Annals of the American Academy of Political and Social Science, 587, 16-30.
34. Mullen, B., Muellerleile, P., Bryant, B. (2001). Cumulative meta-analysis: A consideration of indicators of sufficiency and stability. Personality and Social Psychology Bulletin, 27, 1450-1462.
35. NY vs GlaxoSmithKline. Filing to the Supreme Court of the State of New York. June 2, 2004.
36. Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157-159.
37. Pratt, T., Cullen, C., Blevins, K., Daigle, L., & Unnver, J. (2002). The relationship of attention deficit hyperactivity disorder to crime and delinquency: A meta-analysis. International Journal of Police Science and Management, 4, 344-360.
38. Rhoades, L., & Eisenberger, R. (2002). Perceived organizational support: A review of the literature. Journal of Applied Psychology, 87, 698-714.
39. Richy, F., & Reginster, J. (2006). A Simple Method for Detecting and Adjusting Meta-Analyses for Publication Bias. The Internet Journal of Epidemiology, 3, 2.
40. Rosenberg, M. S., Adams, D. C., & Gurevitch, J. (2000). MetaWin: Statistical Software for Meta-Analysis, Version 2.0. Sunderland, MA: Sinauer Associates.
41. Rosenthal, R. (1979).The 'File Drawer Problem' and Tolerance for Null Results. Psychological Bulletin, 86, 638-641.
42. Sterne, J. A. C., Bradburn, M. J., & Egger, M. (2001a). Meta-Analysis in Stata. In M. Egger, G. Davey Smith, & D. G. Altman (Eds.), Systematic Reviews in Health Care: Meta Analysis in Context (pp. 347-369). London: BMJ.
43. Sterne, J. A. C., Egger, M., & Davey Smith, G. (2001b). Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ, 323, 101-105.
44. Sterne, J. A. C., Becker, B. J., & Egger, M. (2005). The funnel plot.. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley.
45. Sterne, J. A. C., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. Journal of Clinical Epidemiology, 54, 1046-1055.
46. Sterne, J. A. C., & Egger, M. (2005). Regression methods to detect publication and other bias in meta-analysis. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley.
47. Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000). Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53, 1119-1129.
48. Sterne, J. A. C., Harris, R., Harbord, R., & Steichen, T. (2007). What meta-analysis features are available in Stata? Retrieved from http://www.stata.com/support/faqs/stat/meta.html on 21 June 2007.
49. Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22, 2113-2126.
50. Wall Street Journal, (2004, November 1). E-Mails Suggest Merck Knew Vioxx's Dangers at Early Stage.
51. Weisburd, D., Petrosino, A., & Mason, G. (1993). Design sensitivity in criminal justice experiments. Crime and Justice, 17, 337-379.
52. Wilson, D. B., Mitchell, O., & MacKenzie, D. (2006). A systematic review of drug court effects on recidivism. Journal of Experimental Criminology, 2, 459-487.
53. Zhang, J., Ding, E. L., & Song, Y. (2006). Adverse effects of cyclooxygenase 2 inhibitors on renal and arrhythmia events: meta-analysis of randomized trials. JAMA, 296, 1619-1632.