Excess success for psychology articles in the journal Science

PLoS ONE

Volumn 9, Issue 12, 2014, Pages

  Francis, Gregory ^a,b   Tanzman, Jay ^c   Matthews, William J ^d  

^a PURDUE UNIVERSITY   (United States)

^b EPFL   (Switzerland)

^c Tanzman Statistical Consulting   (United States)

^d UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONTROLLED STUDY; CORRELATION ANALYSIS; MONTE CARLO METHOD; NULL HYPOTHESIS; PROBABILITY; PSYCHOLOGY; PUBLICATION; REPRODUCIBILITY; STATISTICAL ANALYSIS; STUDENT T TEST; SYSTEMATIC ANALYSIS; TEST FOR EXCESS SIGNIFICANCE ANALYSIS; THEORY; HUMAN; PEER REVIEW; RESEARCH; TRENDS;

HUMANS; PEER REVIEW, RESEARCH; PERIODICALS AS TOPIC; PSYCHOLOGY, EXPERIMENTAL; RESEARCH;

EID: 84916229695     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0114255     Document Type: Article

References (65)

1. Bem DJ (2011) Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology 100: 407-425.
2. Doyen S, Klein O, Pichon C-L, Cleeremans A (2012) Behavioral priming: It's all in the mind, but whose mind? PLoS ONE 7(1): e29081.
3. Shanks DR, Newell BR, Lee EH, Balakrishnan D, Ekelund L, et al. (2013) Priming intelligent behavior: An elusive phenomenon. PLoS ONE 8(4): e56515.
4. Stapel DA, Lindenberg S (2011) Retraction of Stapel and Lindenberg, Science, 332 (6026) 251-253. Science 334: 1202.
5. Johnson CS, Smeesters D, Wheeler SC (2012) Retraction of Johnson, Smeesters, and Wheeler (2012). Journal of Personality and Social Psychology 103: 605.
6. Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22: 1359-1366.
7. John LK, Loewenstein G, Prelec D (2012) Measuring the prevalence of questionable research practices with incentives for truth-telling. Psychological Science 23: 524-532.
8. Yong E (2012) Bad copy. Nature 485: 298-300.
9. Yong E (2012) Nobel laureate challenges psychologists to clean up their act. Nature doi:10.1038/nature.2012.11535.
10. Levelt Noort, Drenth Committees (2012) Flawed science: The fraudulent research practices of social psychologist Diederik Stapel.Downloaded from https://www.commissielevelt.nl.
11. Ioannidis JPA, Trikalinos TA (2007) An exploratory test for an excess of significant findings. Clinical Trials 4: 245-253.
12. Francis G (2012) Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review 19: 151-156.
13. Francis G (2012) The same old New Look: Publication bias in a study of wishful seeing. i-Perception 3(3): 176-178.
14. Francis G (2012) Evidence that publication bias contaminated studies relating social class and unethical behavior. Proceedings of the National Academy of Sciences 109: E1587.
15. Francis G (2012) Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin & Review 19(6): 975-991.
16. Francis G (2012) The psychology of replication and replication in psychology. Perspectives on Psychological Science 7(6): 580-589.
17. Francis G (2013) Publication bias in "Red, Rank, and Romance in Women Viewing Men" by Elliot, et al. (2010). Journal of Experimental Psychology: General 142: 292-296.
18. Francis G (2013) Replication, statistical consistency, and publication bias. Journal of Mathematical Psychology 57: 153-169.
19. Francis G (2013) We should focus on the biases that matter: A reply to commentaries. Journal of Mathematical Psychology 57: 190-195.
20. Renkewitz F, Fuchs HM, Fiedler S (2011) Is there evidence of publication biases in JDM research? Judgment and Decision Making 6: 870-881.
21. Schimmack U (2012) The ironic effect of significant results on the credibility of multiple study articles. Psychological Methods 17(4): 551-566.
22. Francis G (2014) The frequency of excess success for articles in Psychological Science. Psychonomic Bulletin & Review 21(5): 1180-1187.
23. Begg CB, Mazumdar M (1994) Operating characteristics of a rank correlation test for publication bias. Biometrics 50: 1088-1101.
24. Champely S (2009) pwr: Basic functions for power analysis. R package version 1.1.1. http://CRAN.Rproject.org/package5pwr.
25. Dijksterhuis A, Bos MW, Nordgren LF, van Baaren RB (2006) On making the right choice: The deliberation-without-attention effect. Science 311: 1005-1007.
26. Vohs KD, Mead NL, Goode MR (2006) The psychological consequences of money. Science 311: 1154-1156.
27. Zhong C-B, Liljenquist K (2006) Washing away your sins: Threatened morality and physical cleansing. Science 313: 1451-1452.
28. Wood JN, Glynn DD, Phillips BC, Hauser MD (2007) The perception of rational, goal-directed action in nonhuman primates. Science 317: 1402-1405.
29. Whitson JA, Galinsky AD (2008) Lacking control increases illusory pattern perception. Science 332: 115-117.
30. Mehta R, Zhu R (2009) Blue or red? Exploring the effect of color on cognitive task performances. Science 323: 1226-1229.
31. Paukner A, Suomi SJ, Visalberghi E, Ferrari PF (2009) Capuchin monkeys display affiliation toward humans who imitate them. Science 325: 880-883.
32. Weisbuch M, Pauker K, Ambady N (2009) The subtle transmission of race bias via televised nonverbal behavior. Science 323: 1711-1714.
33. Ackerman JM, Nocera CC, Bargh JA (2010) Incidental haptic sensations influence social judgments and decisions. Science 328: 1712-1715.
34. Bahrami B, Olsen K, Latham PE, Roepstorff A, Rees G, et al. (2010) Optimally interacting minds. Science 329: 1081-1085.
35. Kovács A M, Téglás E, Endress AD (2010) The social sense: Susceptibility to others' beliefs in human infants and adults. Science 330: 1830-1834.
36. Morewedge CK, Huh YE, Vosgerau J (2010) Thought for food: Imagined consumption reduces actual consumption. Science 330: 1530-1533.
37. Halperin E, Russell AG, Trzesniewski KH, Gross JJ, Dweck CS (2010) Promoting the Middle East peace process by changing beliefs about group malleability. Science 333: 1767-1769.
38. Ramirez G, Beilock SL (2011) Writing about testing worries boosts exam performance in the classroom. Science 331: 211-213.
39. Stapel DA, Lindenberg S (2011) Coping with chaos: How disordered contexts promote stereotyping and discrimination. Science 332: 251-253.
40. Gervais WM, Norenzayan A (2012) Analytic thinking promotes religious disbelief. Science 336: 493-496.
41. Seeley TD, Visscher PK, Schlegel T, Hogan PM, Franks NR, et al. (2012) Stop signals provide cross inhibition in collective decision-making by honeybee swarms. Science 335: 108-111.
42. Shah AK, Mullainathan S, Shafir E (2012) Some consequences of having too little. Science 338: 682-685.
43. Masicampo EJ, Lalande DR (2012) A peculiar prevalence of p values just below.05. Quarterly Journal of Experimental Psychology 65(11): 2271-2279.
44. Anscombe FJ (1954) Fixed-sample-size analysis of sequential observations. Biometrics 10(1): 89-100.
45. Strube MJ (2006) SNOOP: A program for demonstrating the consequences of premature and repeated null hypothesis testing. Behavior Research Methods 38: 24-27.
46. Wagenmakers E-J (2007) A practical solution to the pervasive problems of p-values. Psychonomic Bulletin & Review 14: 779-804.
47. Kelley K (2007) Confidence intervals for standardized effect sizes: Theory, application, and implementation. Journal of Statistical Software 20. http://www.jstatsoft.org/v20/a08/
48. Pitt MA, Myung IJ (2002) When a good fit can be bad. Trends in Cognitive Sciences 6(10): 421-425.
49. Roberts S, Pashler H (2000) How persuasive is a good fit? A comment on theory testing. Psychological Review 107: 358-367.
50. Kerr NL (1998) HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review 2: 196-217.
51. Bones AK (2012) We knew the future all along. Perspectives on Psychological Science 7(3): 307-309. doi: 10.1177/1745691612441216
52. Morey RD, Rouder JN, Verhagen J, Wagenmakers E-J (2014) Why hypothesis tests are essential for psychological science: A comment on Cumming. Psychological Science 25, 1289-1290.
53. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, et al. (2013) Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14: 365-376.
54. Kicinski M (2013) Publication bias in recent meta-analyses. PLOS One doi:10.1371/journal.pone.0081823.
55. Fanelli D (2012) Negative results are disappearing from most disciplines and countries. Scientometrics 90: 891-904.
56. Francis G (2014) Too much success for recent groundbreaking epigenetic experiments. Genetics 198: 449-451.
57. Ferguson C, Heene M (2012) A vast graveyard of undead theories: Publication bias and psychological science's aversion to the null. Perspectives on Psychological Science 7(6): 555-561. doi:10.1177/1745691612459059
58. Pashler H, Wagenmakers EJ (2012) Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science 7: 528-530.
59. Matthews WJ (2011) What might judgment and decision making research be like if we took a Bayesian approach to hypothesis testing? Judgment and Decision Making 6(8): 843-856.
60. Cumming G (2014) The new statistics: Why and how. Psychological Science 25(1): 7-29.
61. Kruschke JK (2010) Bayesian data analysis. Wiley Interdisciplinary Reviews: Cognitive Science 1(5): 658-676. doi:10.1002/wcs.72
62. Thompson B (2002) What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher 31: 25-32.
63. Myung IJ, Pitt MA (2009) Optimal experimental design for model discrimination. Psychological Review 116(3): 499-518.
64. Wagenmakers E-J, Wetzels R, Borsboom D, van der Maas HLJ, Kievit RA (2012) An agenda for purely confirmatory research. Perspectives on Psychological Science 7(6): 632-638.
65. Nosek BA, Spies JR, Motyl M (2012) Scientific utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7(6): 615-631.