Preventing the ends from justifying the means: Withholding results to address publication bias in peer-review

BMC Psychology

Volumn 4, Issue 1, 2016, Pages

  Button, Katherine S ^a   Bal, Liz ^b   Clark, Anna ^b   Shipley, Tim ^b  

^a University of Bath   (United Kingdom)

^b BIOMED CENTRAL   (United Kingdom)

Author keywords

Peer review; Publication bias; Results free review; Transparency

Indexed keywords

METHODOLOGY; PEER REVIEW; PSYCHOLOGY; PUBLICATION; PUBLISHING; REPRODUCIBILITY;

PEER REVIEW, RESEARCH; PERIODICALS AS TOPIC; PSYCHOLOGY; PUBLICATION BIAS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 85000645628     PISSN: None     EISSN: 20507283     Source Type: Journal    
DOI: 10.1186/s40359-016-0167-7     Document Type: Editorial

References (45)

1. Beck AT. Cognitive therapy and the emotional disorders. New York: International Universities Press; 1976.
2. Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012;483:531-3.
3. Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010;153:158-66.
4. Button KS. Statistical Rigor and the Perils of Chance. eNeuro. 2016;3(4). doi: 10.1523/ENEURO.0030-16.2016.
5. Button KS, Ioannidis JP, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14:365-76.
6. Button KS, Lawrence NS, Chambers CD, et al. Instilling scientific rigour at the grassroots. Psychol. 2016;29:158-67.
7. Button KS and Munafò MR. Powering Reproducible Research. In: Lilienfeld SO and Waldman ID (eds) Psychological science under scrutiny: Recent challenges and proposed solutions. New York: Wiley & Sons. (in press).
8. Chambers CD. Registered reports: a new publishing initiative at Cortex. Cortex. 2013;49:609-10.
9. Chambers CD, Dienes Z, McIntosh RD, et al. Registered reports: realigning incentives in scientific publishing. Cortex. 2015;66:A1-2.
10. Coursol A, Wagner EE. Effect of positive findings on submission and acceptance: A note of meta-analysis bias. Prof Psychol Res Parctice. 1986;17:136-7.
11. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 1990;263:1385-9.
12. Fanelli D. Do pressures to publish increase scientists' bias? An empirical support from US States Data. PLoS ONE. 2010;5:e10271.
13. Findley MG, Jensen NM, Malesky EJ, Pepinsky TB. Can results-free review reduce publication bias? The results and implications of a pilot study. Comparative Political Studies. 2016;49(13):1667-1703.
14. Fisher R. Statistical methods and scientific induction. J Royal Stat Soc Series B-Stat Methodol. 1955;17:69-78.
15. Francis G. Publication bias and the failure of replication in experimental psychology. Psychon Bull Rev. 2012;19:975-91.
16. Franco A, Malhotra N, Simonovits G. Social science. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345:1502-5.
17. Goldacre B. Bad pharma: how drug companies mislead doctors and harm patients. London: Fourth Estate; 2012.
18. Greenwald AG. Consequences of prejudice against the null hyptohesis. Psychol Bull. 1975;82:1-20.
19. Ioannidis, JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA. 1998;279:281-6.
20. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2:e124.
21. Ioannidis JPA. Why Science Is Not Necessarily Self-Correcting. Perspect Psychol Sci. 2012;7:645-54.
22. John LK, Loewenstein G, Prelec D. Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychol Sci. 2012;23:524-32.
23. Kenall A, Edmunds S, Goodman L, et al. Better reporting for better research: a checklist for reproducibility. BMC Neurosci. 2015;16:44.
24. Laws KR. Psychology, replication & beyond. BMC Psychol. 2016;4:30.
25. Leimu R, Koricheva J. What determines the citation frequency of ecological papers? Trends Ecol Evol. 2005;20:28-32.
26. Mahoney MJ. Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognit Ther Res. 1977;1:161-75.
27. Munafo MR, Stothart G, Flint J. Bias in genetic association studies and impact factor. Mol Psychiatry. 2009;14:119-20.
28. Murtaugh PA. Journal qulaity, effect size, and publication bias in meta-analysis. Ecology. 2002;83:1162-6.
29. Neuroskeptic. The Nine Circles of Scientific Hell. Perspect Psychol Sci. 2012;7:643-4.
30. Nuzzo R. Fooling ourselves. Nature. 2015;S26:182-5.
31. Perel P, Roberts I, Sena E, et al. Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ. 2007;334:197.
32. Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov. 2011;10:712.
33. Rosenthal R. The File Drawer Problem and Tolerance for Null Results. Psychol Bull. 1979;86:638-641.
34. Rothstein H, Sutton AJ, Borenstein M. Publication bias in meta-analysis: prevention, assessment and adjustments. John Wiley: Chichester. 2005.
35. Schillebeeckx M, Maricque B, Lewis C. The missing piece to changing the university culture. Nat Biotechnol. 2013;31:938-41.
36. Scott S, Kranz JE, Cole J, et al. Design, power, and interpretation of studies in the standard murine model of ALS. Amyotroph Lateral Scler. 2008;9:4-15.
37. Shadish Jr WR, Doherty M, Montgomery LM. How many studies are in the file drawer? an estimate from the family/marital psychotherapy literature. Clin Psychol Rev. 1989;9:589-603.
38. Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22:1359-66.
39. Smart RG. The importance of negative results in psychological research. Canadian Psychol. 1964;5:225-32.
40. Smith ML. Sex bias in counseling and psychotherapy. Psychol Bull. 1980;87:392-407.
41. Sterling TD. Publication decisions and their possible effects on inferences drawn from tests of significance--or vice versa. J Am Stat Assoc. 1959;54:30-4.
42. Steward O, Popovich PG, Dietrich WD, et al. Replication and reproducibility in spinal cord injury research. Exp Neurol. 2012;233:597-605.
43. van Dijk D, Manor O, Carey LB. Publication metrics and success on the academic job market. Curr Biol. 2014;24:R516-7.
44. Vankov I, Bowers J, Munafo MR. On the persistence of low power in psychological science. Q J Exp Psychol (Hove). 2014;67:1037-40.
45. Yong E. Replication studies: Bad copy. Nature. 2012;485:298-300.