Why Selective Publication of Statistically Significant Results Can Be Effective

PLoS ONE

Volumn 8, Issue 6, 2013, Pages

  de Winter, Joost ^a   Happee, Riender ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; MEDICAL RESEARCH; META ANALYSIS (TOPIC); PHENOMENOLOGY; PRACTICE GUIDELINE; PUBLICATION; REPRODUCIBILITY; SCIENTIFIC LITERATURE; SIMULATION; STATISTICAL SIGNIFICANCE;

ALGORITHMS; COMPUTER SIMULATION; FALSE POSITIVE REACTIONS; HUMANS; META-ANALYSIS AS TOPIC; PEER REVIEW, RESEARCH; PUBLICATION BIAS; PUBLISHING; REPRODUCIBILITY OF RESULTS;

EID: 84879259544     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0066463     Document Type: Article

References (41)

1. Asendorpf JB, Conner M, De Fruyt F, De Houwer J, Denissen JJ, et al. (2013) Recommendations for increasing replicability in psychology. Eur J Pers 27: 108-119.
2. Ioannidis JP, (2005) Why most published research findings are false. PLoS Med 2: e124.
3. Ioannidis JP, (2008) Why most discovered true associations are inflated. Epidemiology 19: 640-648.
4. Vul E, Harris C, Winkielman P, Pashler H, (2009) Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspect Psychol Sci 4: 274-290.
5. Wacholder S, Chanock S, Garcia-Closas M, Rothman N, (2004) Assessing the probability that a positive report is false: An approach for molecular epidemiology studies. J Natl Cancer Inst 96: 434-442.
6. Bertamini M, Munafò MR, (2012) Bite-size science and its undesired side effects. Perspect Psychol Sci 7: 67-71.
7. Pashler H, Harris CR, (2012) Is the replicability crisis overblown? Three arguments examined. Perspect Psychol Sci 7: 531-536.
8. Callaham ML, Wears RL, Weber EJ, Barton C, Young G, (1998) Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. JAMA 280: 254-257.
9. Easterbrook PJ, Gopalan R, Berlin J, Matthews DR, (1991) Publication bias in clinical research. The Lancet 337: 867-872.
10. Rosenthal R, (1979) The file drawer problem and tolerance for null results. Psychol Bull 86: 638-641.
11. Pautasso M, (2010) Worsening file-drawer problem in the abstracts of natural, medical and social science databases. Scientometrics 85: 193-202.
12. Sena ES, Van der Worp HB, Bath PM, Howells DW, Macleod MR, (2010) Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol 8: e1000344.
13. John LK, Loewenstein G, Prelec D, (2012) Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol Sci 23: 524-532.
14. Simmons JP, Nelson LD, Simonsohn U, (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22: 1359-1366.
15. Boffetta P, McLaughlin JK, La Vecchia C, Tarone RE, Lipworth L, et al. (2008) False-positive results in cancer epidemiology: a plea for epistemological modesty. J Natl Cancer Inst 100: 988-995.
16. Ferguson CJ, Heene M, (2012) A vast graveyard of undead theories: Publication bias and psychological science's aversion to the null. Perspect Psychol Sci 7: 555-561.
17. Giner-Sorolla R, (2012) Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspect Psychol Sci 7: 562-571.
18. Young NS, Ioannidis JP, Al-Ubaydli O, (2008) Why current publication practices may distort science. PLoS Med 5: e201.
19. Dirnagl U, (2010) Fighting publication bias: Introducing the Negative Results section. J Cereb Blood Flow Metab 30: 1263-1264.
20. Sterling TD, Rosenbaum W, Weinkam J, (1995) Publication decisions revisited: the effect of the outcome of statistical tests on the decision to publish and vice versa. Am Stat 49: 108-112.
21. Suñé P, Suñé JM, Montoro JB, (2013) Positive outcomes influence the rate and time to publication, but not the impact factor of publications of clinical trial results. PLoS One 8: e54583.
22. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, et al. (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 3: e3081.
23. Davison M, Nevin JA, (2005) On science and the discriminative law of effect. J Exp Anal Behav 83: 85-92.
24. Ioannidis J (2012) Reporting and reproducible research: Salvaging the self-correction principle of science. Freiburg, Germany: Annual Lecture given at the EQUATOR Network Scientific Symposium. Available: http://www.equator-network.org/index.aspx?o=5599. Accessed 1 March 2013.
25. Binfield P (2009) PLoS One: Background, future development, and article-level metrics. ELPUB2009 Conference on Electronic Publishing. Milan, Italy. 69-86.
26. Bronowski J (1979) The origins of knowledge and imagination: Yale University Press.
27. Ioannidis JP, Trikalinos TA, (2005) Early extreme contradictory estimates may appear in published research: The Proteus phenomenon in molecular genetics research and randomized trials. J Clin Epidemiol 58: 543-549.
28. Pfeiffer T, Bertram L, Ioannidis JP, (2011) Quantifying selective reporting and the Proteus phenomenon for multiple datasets with similar bias. PLoS One 6: e18362.
29. Cook C, (2010) Mode of administration bias. J Man Manip Ther 18: 61.
30. Ioannidis J (2013) Clarifications on the application and interpretation of the test for excess significance and its extensions. J Math Psychol: in press.
31. Alsheikh-Ali AA, Qureshi W, Al-Mallah MH, Ioannidis JP, (2011) Public availability of published research data in high-impact journals. PLoS One 6: e24357.
32. Ioannidis JP, (2006) Evolution and translation of research findings: From bench to where. PLoS Clin Trials 1: e36.
33. Ioannidis JP, Ntzani EE, Trikalinos TA, Contopoulos-Ioannidis DG, (2001) Replication validity of genetic association studies. Nat Genet 29: 306-309.
34. Schooler J (2011) Unpublished results hide the decline effect. Nature 470: 437.
35. Makel MC, Plucker JA, Hegarty B, (2012) Replications in Psychology Research How Often Do They Really Occur? Perspect Psychol Sci 7: 537-542.
36. Ioannidis JP, (2012) Why science is not necessarily self-correcting. Perspect Psychol Sci 7: 645-654.
37. Nelson LD, Simmons JP, Simonsohn U, (2012) Let's publish fewer papers. Psychol Inq 23: 291-293.
38. Fanelli D, (2012) Negative results are disappearing from most disciplines and countries. Scientometrics 90: 891-904.
39. Jagacinski RJ, (1977) A qualitative look at feedback control theory as a style of describing behavior. Hum Factors 19: 331-347.
40. Drummond C (2009) Replicability is not reproducibility: Nor is it good science. International Conference on Machine Learning. Montreal, Canada. Available: http://www.csi.uottawa.ca/~cdrummon/pubs/ICMLws09.pdf. Accessed 1 March 2013.
41. Fiedler K, Kutzner F, Krueger JI, (2012) The long way from α-error control to validity proper: Problems with a short-sighted false-positive debate. Perspect Psychol Sci 7: 661-669.