Why publishing everything is more effective than selective publishing of statistically significant results

PLoS ONE

Volumn 9, Issue 1, 2014, Pages

  Van Assen, Marcel A L M ^a   Van Aert, Robbie C M ^a   Nuijten, Michèle B ^a   Wicherts, Jelte M ^a  

^a TILBURG UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; EFFECT SIZE; MEASUREMENT ACCURACY; MEDICAL RESEARCH; OUTCOME ASSESSMENT; PUBLISHING; STATISTICAL SIGNIFICANCE;

META-ANALYSIS AS TOPIC; PUBLICATIONS; PUBLISHING; STATISTICS AS TOPIC;

EID: 84898460382     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0084896     Document Type: Article

References (36)

1. De Winter J, Happee R (2013) Why Selective Publication of Statistically Significant Results Can Be Effective. PLoS ONE 8: e66463.
2. Fanelli D (2010) "Positive" Results Increase Down the Hierarchy of the Sciences. PLoS One 5: e10068.
3. Fanelli D (2012) Negative results are disappearing from most disciplines and countries. Scientometrics 90: 891-904.
4. Asendorpf JB, Conner M, Fruyt FD, Houwer JD, Denissen JJA, et al. (2013) Recommendations for increasing replicability in psychology. European Journal of Personality 27: 108-119.
5. Kraemer HC, Gardner C, Brooks JO, Yesavage JA (1998) Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints. Psychological Methods 3: 23-31.
6. Gerber AS, Green DP, Nickerson D (2001) Testing for Publication Bias in Political Science. Political Analysis 9: 385-392.
7. Rosenthal R (1979) The file drawer problem and tolerance for null results. Psychological Bulletin 86: 638-641.
8. Sena ES, van der Worp HB, Bath PM, Howells DW, Macleod MR (2010) Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS biology 8: e1000344.
9. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K (2009) Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews.
10. Cuijpers P, van Straten A, Bohlmeijer E, Hollon SD, Andersson G (2010) The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size. Psychological Medicine 40: 211-223.
11. Pashler H, Wagenmakers EJ (2012) Editors' Introduction to the Special Section on Replicability in Psychological Science: A Crisis of Confidence? Perspectives on Psychological Science 7: 528-530.
12. Francis G (2012) Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin & Review 19: 975-991.
13. Ioannidis JPA (2005) Why most published research findings are false. Plos Medicine 2: e124.
14. Van Noorden R (2013) The true cost of science publishing. Nature 495: 426-429.
15. Nelson LD, Simmons JP, Simonsohn U (2012) Let's Publish Fewer Papers. Psychological Inquiry 23: 291-293.
16. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, et al. (2013) Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14: 1-12.
17. Bakker M, van Dijk A, Wicherts JM (2012) The rules of the game called psychological science. Perspectives on Psychological Science 7: 543-554.
18. Niemeyer H, Musch J, Pietrowsky R (2012) Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for schizophrenia. Schizophrenia Research 138: 103-112.
19. Niemeyer H, Musch J, Pietrowsky R (2013) Publication Bias in Meta-Analyses of the Efficacy of Psychotherapeutic Interventions for Depression. Journal of Consulting and Clinical Psychology 81: 58-74.
20. Ferguson CJ, Brannick MT (2012) Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods 17: 120-128.
21. Ioannidis JPA (2011) Excess Significance Bias in the Literature on Brain Volume Abnormalities. Archives of General Psychiatry 68: 773-780.
22. Ioannidis JPA, Trikalinos TA (2007) An exploratory test for an excess of significant findings. Clinical Trials 4: 245-253. (Pubitemid 47400532)
23. Francis G (2012) Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review 19: 151-156.
24. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Introduction to meta-analysis. Chichester, UK: John Wiley & Sons, Ltd.
25. Viechtbauer W (2010) The metafor package: A meta-analysis package for R (Version 1.3-0).
26. Rothstein HR, Bushman BJ (2012) Publication bias in psychological science: comment on Ferguson and Brannick (2012). Psychological Methods 17: 129-136.
27. van Assen MALM, van Aert RCM, Wicherts JM (2013) Meta-analysis using effect size distributions of only significant studies. Tilburg University.
28. Hunter JE, Schmidt FL (2000) Fixed Effects vs. Random Effects Meta-Analysis Models: Implications for Cumulative Research Knowledge. International Journal of Selection and Assessment 8: 275-292. (Pubitemid 33203822)
29. Field AP, Gillett R (2010) How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology 63: 665-694.
30. Schmidt FL, Oh IS, Hayes TL (2009) Fixed-versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology 62: 97-128.
31. Viechtbauer W (2007) Hypothesis tests for population heterogeneity in meta-analysis. British Journal of Mathematical & Statistical Psychology 60: 29-60.
32. Giner-Sorolla R (2012) Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspectives on Psychological Science 7: 562-571.
33. Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22: 1359-1366.
34. Ioannidis JPA (2008) Why most discovered true associations are inflated. Epidemiology 19: 640-648.
35. Nosek BA, Bar-Anan Y (2012) Scientific Utopia: I. Opening Scientific Communication. Psychological Inquiry 23: 217-243.
36. Nosek BA, Spies J, Motyl M (2012) Scientific Utopia: II - Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science 7: 615-631.