Publication Bias: The Antagonist of Meta-Analytic Reviews and Effective Policymaking

Educational Evaluation and Policy Analysis

Volumn 34, Issue 3, 2012, Pages 259-277

  Banks, George C ^a   Kepes, Sven ^a   Banks, Karen P ^b  

^a VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

^b GEORGE MASON UNIVERSITY   (United States)

Author keywords

education reviews; literacy; meta analysis; publication bias; systematic reviews

Indexed keywords

EID: 84864798514     PISSN: 01623737     EISSN: 19351062     Source Type: Journal    
DOI: 10.3102/0162373712446144     Document Type: Article

References (60)

1. Publication manual of the American Psychological Association. 2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.Washington, DC: American Psychological Association; 2010:.
2. Bangert-Drowns R.L.,Hurley M.M.,Wilkinson B.The effects of school-based writing-to-learn interventions on academic achievement: A meta-analysis.Review of Educational Research. 2004;74:29-58.
3. Banks G.C.,Batchelor J.H.,McDaniel M.A.Smarter people are (a bit) more symmetrical: A meta-analysis of the relationship between intelligence and fluctuating asymmetry.Intelligence. 2010;38:393-401.
4. Banks G.C.,Kepes S.,McDaniel M.A.Publication bias: A call for improved meta-analytic practice in the organizational sciences.International Journal of Selection and Assessment. 2012;20:182-196.
5. Banks G.C.,McDaniel M.A.The kryptonite of evidence-based I-O psychology.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2011;4:40-44.
6. Becker B.J.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:111-126.
7. Begg C.B.,Mazumdar M.Operating characteristics of a rank correlation test for publication bias.Biometrics. 1994;50:1088-1101.
8. Berlin J.A.,Ghersi D.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:35-48.
9. Borenstein M.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:111-126.
10. Borenstein M.,Hedges L.V.,Higgins J.P.,Rothstein H.R.Englewood, NJ: Biostat; 2005:.
11. Borenstein M.,Hedges L.V.,Higgins J.P.,Rothstein H.R.Introduction to meta-analysis. Chichester, UK: Wiley, 2009.
12. Cohen J.Statistical power analysis for the behavioral sciences. 1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Hillsdale, NJ: Lawrence Erlbaum Associates; 1988:.
13. Desjardins R.Researching the links between education and well-being.European Journal of Education. 2008;43:23-35.
14. Dickersin K.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:11-34.
15. Duval S.J.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:127-144.
16. Duval S.J.,Tweedie R.L.A non- parametric "trim and fill" method of accounting for publication bias in meta-analysis.Journal of the American Statistical Association. 2000a;95:89-98.
17. Duval S.J.,Tweedie R.L.Trim and fill: A simple funnel plot-based method of testing and adjusting for publication bias in meta-analysis.Biometrics. 2000b;56:276-284.
18. Egger M.,Smith G.D.,Schneider M.,Minder C.Bias in meta-analysis detected by a simple, graphical test.British Medical Journal. 1997;315:629-634.
19. Evangelou E.,Trikalinos T.A.,Ioannidis J.P.Unavailability of online supplementary scientific information from articles published in major journals.The FASEB Journal. 2005;19:1943-1944.
20. Greenspan A.The importance of education in today's economy.Proceedings. 2001;:6-11.
21. Greenwald A.G.Consequences of prejudice against the null hypothesis.Psychological Bulletin. 1975;82:1-20.
22. Higgins J.P.,Thompson S.G.,Deeks J.J.,Altman D.G.Measuring inconsistency in meta-analyses.British Medical Journal. 2003;327:557-560.
23. Hopewell S.,Clarke M.,Mallett S.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:48-72.
24. Hunter J.E.,Schmidt F.L.Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage; 2004:.
25. Ioannidis J.P.Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.Journal of the American Medical Association. 1998;279:281-286.
26. Ioannidis J.P.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:287-302.
27. Jick T.D.Mixing qualitative and quantitative methods: Triangulation in action.Administrative Science Quarterly. 1979;24:602-611.
28. Johnston R.,Viadero D.Unmet promise: Raising minority achievement. The achievement gap.Education Week. 2000;:1-23.
29. Education Week. 2010;:.
30. Lau J.,Antman E.M.Cumulative meta-analysis of therapeutic trials for myocardial infarction.New England Journal of Medicine. 1992;327:248-254.
31. Lau J.,Schmid C.H.,Chalmers T.C.Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care.Journal of Clinical Epidemiology. 1995;48:45-57.
32. McDaniel M.A. Paper presented at the annual meeting of the Society for Industrial and Organizational Psychology; 2009 New Orleans, LA; 2009.
33. McDaniel M.A.,Rothstein H.R.,Whetzel D.L.Publication bias: A case study of four test vendors.Personnel Psychology. 2006;59:927-953.
34. Mullen B.,Muellerleile P.,Bryant B.Cumulative meta-analysis: A consideration of indicators of sufficiency and stability.Personality and Social Psychology Bulletin. 2001;27:1450-1462.
35. The nation's report card: Reading 2009. Washington, DC: Institute of Education Sciences, U.S. Department of Education; 2009.
36. Newman D.A.Longitudinal modeling with randomly and systematically missing data: A simulation of ad hoc, maximum likelihood, and multiple imputation techniques.Organizational Research Methods. 2003;6:328-362.
37. Orwin R.F.A fail-safe N for effect size in meta-analysis.Journal of Educational Statistics. 1983;8:157-159.
38. Palmer T.M.,Peters J.,Sutton A.J.,Moreno S.G.Contour enhanced funnel plots for meta-analysis.The Stata Journal. 2008;8:242-254.
39. Pearson P.D.,Hiebert E.H.National reports in literacy: Building a scientific base for practice and policy.Educational Researcher. 2010;79:286-294.
40. Peters J.,Sutton A.J.,Jones D.R.,Abrams K.R.,Rushton L.Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry.Journal of Clinical Epidemiology. 2008;61:991-996.
41. Piasta S.B.,Wagner R.K.Developing early literacy skills: A meta-analysis of alphabet learning and instruction.Reading Research Quarterly. 2010;45:8-38.
42. Rolstad K.,Mahoney K.,Glass G.The big picture: A meta-analysis of program effectiveness research on English language learners.Educational Policy. 2005;19:572-594.
43. Rosenthal R.The "file drawer problem" and tolerance for null results.Psychological Bulletin. 1979;86:638-641.
44. Rothstein H.APA handbook of research methods in psychology: Vol. 1. Foundations, planning, measures, and psychometrics. Cooper H.M., ed. Washington, DC: American Psychological Association; 2012:133-144.
45. Rothstein H.R.,Sutton A.J.,Borenstein M.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005a:1-7.
46. Rothstein H.R.,Sutton A.J.,Borenstein M.Publication bias in meta-analysis: Prevention, assessment and adjustments. Chichester, UK: Wiley; 2005b:.
47. Sackett P.R.,Larson J.R.Handbook of industrial and organizational psychology. Dunnette M.D.Hough L.M., ed. Palo Alto, CA: Consulting Psychologists Press; 1990:419-489.
48. Schafer J.L.,Graham J.W.Missing data: Our view of the state of the art.Psychological Methods. 2002;7:147-177.
49. Song F.,Easterwood A.,Gilbody S.,Duley L.,Sutton A.J.Publication and other selection biases in systematic reviews.Health Technology. 2000;4:1-115.
50. Song F.,Parekh S.,Hooper L.,Loke Y.K.,Ryder J.,Sutton A.J., Harvey I.Dissemination and publication of research findings: An updated review of related biases.Health Technology Assessment. 2010;14:1-220.
51. Sterne J.A.,Egger M.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:99-110.
52. Sterne J.A.,Gavaghan D.,Egger M.Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature.Journal of Clinical Epidemiology. 2000;53:1119-1129.
53. Sterne J.A.,Gavaghan D.,Egger M.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:75-98.
54. Sterne J.A.,Sutton A.J.,Ioannidis J.P.,Terrin N.,Jones D.R.,Lau J.,Higgins J.P.Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials.British Medical Journal. 2011;342:d4002-d4010.
55. Superfine B.M.Court-driven reform and equal educational opportunity: Centralization, decentralization, and the shifting judicial role.Review of Educational Research. 2010;80:108-137.
56. Sutton A.J.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:175-192.
57. Sutton A.J.The handbook of research synthesis and meta-analysis. 2009). Publication bias. In The handbook of research synthesis and meta-analysis (2nd ed., pp. 435-452). New York, NY: Russell Sage Foundation.New York, NY: Russell Sage Foundation; 2009:435-452.
58. Sutton A.J.,Pigott T.D.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:223-240.
59. Torgerson C.J.Publication bias: The Achilles' heel of systematic reviews?.British Journal of Educational Studies. 2006;54:89-102.
60. Trikalinos T.A.,Ioannidis J.P.Publication bias in meta-analysis: Prevention, assessment and adjustments. Rothstein H.R.Sutton A.J.Borenstein M., ed. Chichester, UK: Wiley; 2005:241-259.