The funnel plot

Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments

Volumn , Issue , 2006, Pages 75-98

  Sterne, Jonathan A C ^a,b   Becker, Betsy Jane ^c,e   Egger, Matthias ^a,b,d  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

^b Reporting Bias Methods Group of the Cochrane Collaboration   (United Kingdom)

^c Methods Training Group for the Campbell Collaboration   (United Kingdom)

^d UNIVERSITY OF BERN   (Switzerland)

^e FLORIDA STATE UNIVERSITY   (United States)

Author keywords

Fixed effect estimate; Funnel plot asymmetry; Funnel plots; Funnel plots in meta analyses; Precision; Random effects estimate; Standard error; Structured or unstructured employment interviews; Summary effect estimate; Teacher expectancy effects; Variance and inverse variance

Indexed keywords

EID: 84889314860     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/0470870168.ch5     Document Type: Chapter

References (51)

1. Becker, B.J. and Morton, S.C. (2002). Publication bias: Methods and practice in evidence-based medicine and social science. Mathematical Sciences Research Institute, Berkeley, California.
2. Begg, C.B. and Berlin, J.A. (1988). Publication bias: a problem in interpreting medical data. Journal of the Royal Statistical Society (Series A), 151, 419-463.
3. Bond, C.F., Wiitala, W.L. and Richard, F.D. (2003). Meta-analysis of raw mean differences. Psychological Methods, 8, 406-418.
4. Booth-Kewley, S. and Friedman, H.S. (1987). Psychological predictors of heart disease: A quantitative review. Psychological Bulletin, 101, 343-362.
5. Cappelleri, J.C., Ioannidis, J.P., Schmid, C.H., de Ferranti, S.D., Aubert, M., Chalmers, T.C. and Lau, J. (1996). Large trials vs meta-analysis of smaller trials: How do their results compare? Journal of the American Medical Association, 276, 1332-1338.
6. Cooper, H.M. (1998). Synthesizing Research: A Guide for Literature Reviews (3rd edition). Thousand Oaks, CA: Sage.
7. Copas, J. (1999). What works? Selectivity models and meta-analysis. Journal of the Royal Statistical Society (Series A), 162, 95-109.
8. Copas, J.B. and Shi, J.Q. (2000). Reanalysis of epidemiological evidence on lung cancer and passive smoking. British Medical Journal, 320, 417-418.
9. Davey Smith, G. and Egger, M. (1994). Who benefits from medical interventions? Treating low risk patients can be a high risk strategy. British Medical Journal, 308, 72-74.
10. Davey Smith, G., Song, F. and Sheldon, T.A. (1993). Cholesterol lowering and mortality: The importance of considering initial level of risk. British Medical Journal, 306, 1367-1373.
11. Deeks, J.J. and Altman, D.G. (2001). Effect measures for meta-analysis of trials with binary outcomes. In M. Egger, G. Davey Smith and D.G. Altman (eds), Systematic Reviews in Health Care: Meta-Analysis in Context, 2nd edition, pp. 313-335. London: BMJ Books.
12. Devine, E. and Cook, T.D. (1983). A meta-analytic analysis of effects of psychoeducational interventions on length of postsurgical hospital stay. Nursing Research, 32, 267-274.
13. Dickersin, K. (1997). How important is publication bias? A synthesis of available data. AIDS Education & Prevention, 9, S15-21.
14. Dickersin, K., Min, Y.I. and Meinert, C.L. (1992). Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. Journal of the American Medical Association, 263, 374-378.
15. Easterbrook, P.J., Berlin, J.A., Gopalan, R. and Matthews, D.R. (1991). Publication bias in clinical research. Lancet, 337, 867-872.
16. Egger, M. and Davey Smith, G. (1995). Misleading meta-analysis. Lessons from an ‘effective, safe, simple’ intervention that wasn’t. British Medical Journal, 310, 752-754.
17. Egger, M., Davey Smith, G., Schneider, M. and Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629-634.
18. Egger, M., Davey Smith, G. and Altman, D. (2001). Systematic Reviews in Health care: Meta-analysis in Context. London: BMJ Books.
19. Egger, M., Jüni, P., Bartlett, C., Holenstein, F. and Sterne, J. (2003). How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technology Assessment, 7, 1-68.
20. Engels, E.A., Schmid, C.H., Terrin, N.T., Olkin, I. and Lau, J. (2000). Heterogeneity and statistical significance in meta-analysis: An empirical study of 125 meta-analyses. Statistics in Medicine, 19, 1707-1728.
21. Fisher, R.A. (1921). On the ‘probable error’ of a coefficient of correlation deduced from a small sample. Metron, 1, 1-32.
22. Gerber, S., Tallon, D., Schneider, M. and Egger, M. (2005). Characteristics of meta-analyses published in leading medical journals 1993 to 2002: Bibliographic study. Manuscript in preparation.
23. Glasziou, P.P. and Irwig, L.M. (1995). An evidence based approach to individualising treatment. British Medical Journal, 311, 1356-1359.
24. Glasziou, P., Irwig, L.M., Bain, C. and Colditz, G. (2001). Systematic Reviews in Health Care: A Practical Guide. Cambridge: Cambridge University Press.
25. Hackshaw, A.K., Law, M.R. and Wald, N.J. (1997). The accumulated evidence on lung cancer and environmental tobacco smoke. British Medical Journal, 315, 980-988.
26. Hedges, L.V. and Olkin, I. (1985). Statistical Methods for Meta-analysis. Boston: Academic Press.
27. Hedges, L.V. and Vevea, J.L. (1994). Estimating effect size under publication bias: small sample properties and robustness of a random effects selection model. Journal of Educational & Behavioral Statistics, 21, 299-332.
28. Ioannidis, J.P. (1998). Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. Journal of the American Medical Association, 279, 281-286.
29. Ioannidis, J.P.A., Ntzani, E.E., Trikalinos, T.A. and Contopoulos-Ioannidis, D.G. (2001). Replication validity of genetic association studies. Nature Genetics, 29, 306-309.
30. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group (1988). Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17 187 cases of suspected acute myocardial infarction: ISIS-2. Lancet, ii, 349-360.
31. Light, R.J. and Pillemer, D.B. (1984). Summing Up: The Science of Reviewing Research. Cambridge, MA: Harvard University Press.
32. Light, R.J., Singer, J.D. and Willett, J.B. (1994). The visual presentation and interpretation of meta-analyses. In H.M. Cooper and L.V. Hedges (eds), The Handbook of Research Synthesis. New York: Russell Sage Foundation.
33. Lipsey, M.W. and Wilson, D.B. (2001). Practical Meta-analysis. Thousand Oaks, CA: Sage.
34. Lohmueller, K.E., Pearce, C.L., Pike, M., Lander, E.S. and Hirschhorn, J.N. (2003). Metaanalysis of genetic association studies supports a contribution of common variants to susceptibility to common disease. Nature Genetics, 33, 177-182.
35. McDaniel, M.A., Whetzel, D., Schmidt, F.L. and Maurer, S. (1994). The validity of the employment interview: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616.
36. Mullen, B. (1989). Advanced BASIC Meta-analysis. Hillsdale, NJ: Lawrence Erlbaum.
37. Poole, C. and Greenland, S. (1999). Random-effects meta-analyses are not always conservative. American Journal of Epidemiology, 150, 469-475.
38. Raudenbush, S.W. (1984). Magnitude of teacher expectancy effects on pupil IQ as a function of the credibility of expectancy induction. Journal of Educational Psychology, 76, 85-97.
39. Rosenthal, R. (1991) Meta-analytic Procedures for Social Research, revised edition. Newbury Park, CA: Sage.
40. Rosenthal, R. and Jacobson, L. (1968). Pygmalion in the Classroom: Teacher Expectation and Pupils’ Intellectual Development. New York: Holt, Rinehart and Winston.
41. Rothstein, H.R., McDaniel, M.A. and Borenstein, M. (2002). Meta-analysis: A review of quantitative cumulation methods. In N. Schmitt and F. Drasgow (eds), Measuring and Analyzing Behavior in Organizations, pp. 534-570. San Francisco: Jossey-Bass.
42. Schulz, K.F., Chalmers, I., Hayes, R.J. and Altman, D.G. (1995). Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Journal of the American Medical Association, 273, 408-412.
43. Stern, J.M. and Simes, R.J. (1997). Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. British Medical Journal, 315, 640-645.
44. Sterne, J.A. and Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology, 54, 1046-1055.
45. Sterne, J.A.C., Gavaghan, D. and Egger, M. (2000). Publication and related bias in metaanalysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53, 1119-1129.
46. Sterne, J.A.C., Egger, M. & Davey Smith, G. (2001). Investigating and dealing with publication and other biases. In M. Egger, D.G. Altman and G. Davey Smith (eds), Systematic Reviews in Health Care: Meta-analysis in Context, 2nd edition. London: BMJ Books.
47. Stuck, A.E., Siu, A.L., Wieland, G.D., Adams, J. and Rubenstein, L.Z. (1993). Comprehensive geriatric assessment: A meta-analysis of controlled trials. Lancet, 342, 1032-1036.
48. Sutton, A.J., Abrams, K., Jones, D.R., Sheldon, T.A. and Song, F. (2002) Methods for Meta-analysis in Medical Research. Chichester: John Wiley & Sons, Ltd.
49. Vandenbroucke, J.P. (1988). Passive smoking and lung cancer: a publication bias? British Medical Journal of Clinical Research Education, 296, 391-392.
50. Wang, M.C. and Bushman, B.J. (1998). Using the normal quantile plot to explore metaanalytic data sets. Psychological Methods, 3, 46-54.
51. Woods, K.L. and Fletcher, S. (1994). Long term outcome after magnesium sulphate in suspected acute myocardial infarction: The Second Leicester Intravenous Magnesium Intervention Trial (LIMIT-2). Lancet, 343, 816-819.