Erratum: The natural selection of bad science (R. Soc. Open Sci. (2016) 3 (160384) DOI: 10.1098/rsos.160384);The natural selection of bad science

Royal Society Open Science

Volumn 3, Issue 9, 2016, Pages

  Smaldino, Paul E ^a   McElreath, Richard ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MAX PLANCK INSTITUTE FOR EVOLUTIONARY ANTHROPOLOGY   (Germany)

Author keywords

Campbell s law; Cultural evolution; Incentives; Metascience; Replication; Statistical power

Indexed keywords

EID: 84988603450     PISSN: None     EISSN: 20545703     Source Type: Journal    
DOI: 10.1098/rsos.231026     Document Type: Erratum

References (95)

1. Campbell DT. 1976 Assessing the impact of planned social change. Hanover, NH: The Public Affairs Center.
2. Wasserstein RL, Lazar NA. 2016 The ASA’s statement on p-values: context, process, and purpose. Am. Stat. 70, 129–133. (doi:10.1080/00031305.2016.1154108)
3. Meehl PE. 1967 Theory-testing in psychology and physics: a methodological paradox. Phil. Sci. 34, 103–115. (doi:10.1086/288135)
4. Cohen J. 1994 The earth is round (p < .05). Am. Psychol. 49, 997–1003. (doi:10.1037/0003-066X.49.12.997)
5. Enserink M. 2012 Final report on Stapel blames field as a whole. Science 338, 1270–1271. (doi:10.1126/science.338.6112.1270)
6. Vul E, Harris C, Winkielman P, Pashler H. 2009 Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspect. Psychol. Sci. 4, 274–290. (doi:10.1111/j.1745-6924.2009.01125.x)
7. Peterson D. 2016 The baby factory: difficult research objects, disciplinary standards, and the production of statistical significance. Socius 2, 1–10. (doi:10.1177/2378023115625071)
8. Kerr NL. 1998 HARKing: hypothesizing after the results are known. Pers. Soc. Psychol. Rev. 2, 196–217. (doi:10.1207/s15327957pspr0203_4)
9. John LK, Loewenstein G, Prelec D. 2012 Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23, 524–532. (doi:10.1177/0956797611430953)
10. Ioannidis JPA. 2014 Research accomplishments that are too good to be true. Intensive Care Med. 40, 99–101. (doi:10.1007/s00134-013-3100-z)
11. Simmons JP, Nelson LD, Simonsohn U. 2011 False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359–1366. (doi:10.1177/0956797611417632)
12. Horton R. 2015 Offline: what is medicine’s 5 sigma? Lancet 385, 1380. (doi:10.1016/S0140-6736(15) 60696-1)
13. Ioannidis JPA. 2005 Why most published research findings are false. PLoS Med. 2, e124. (doi:10.1371/journal.pmed.0020124)
14. Pashler H, Harris CR. 2012 Is the replicability crisis overblown? Three arguments examined. Perspect. Psychol. Sci. 7, 531–536. (doi:10.1177/1745691612463401)
15. McElreath R, Smaldino PE. 2015 Replication, communication, and the population dynamics of scientific discovery. PLoS ONE 10, e0136088. (doi:10.1371/journal.pone.0136088)
16. Ioannidis JPA. 2005 Contradicted and initially stronger effects in highly cited clinical research. J. Am. Med. Assoc. 294, 218–228. (doi:10.1001/jama.294.2.218)
17. Ioannidis JPA. 2012 Why science is not necessarily self-correcting. Perspect. Psychol. Sci. 7, 645–654. (doi:10.1177/1745691612464056)
18. Steen RG, Casadevall A, Fang FC. 2013 Why has the number of scientific retractions increased? PLoS ONE 8, e68397. (doi:10.1371/journal.pone.0068397)
19. Campbell DT. 1965 Variation and selective retention in socio-cultural evolution. In Social change in developing areas: a reinterpretation of evolutionary theory (eds H Barringer, G Blanksten, R Mack), pp. 19–49. Cambridge, MA: Schenkman.
20. Skinner BF. 1981 Selection by consequences. Science 213, 501–504. (doi:10.1126/science.7244649)
21. Boyd R, Richerson PJ 1985 Culture and the evolutionary process. Chicago, IL: University of Chicago Press.
22. Mesoudi A 2011 Cultural evolution: how Darwinian theory can explain human culture and synthesize the social sciences. Chicago, IL: University of Chicago Press.
23. Whiten A, Hinde RA, Laland KN, Stringer CB. 2011 Culture evolves. Phil. Trans. R. Soc. B 366, 938–948. (doi:10.1098/rstb.2010.0372)
24. Smaldino PE. 2014 The cultural evolution of emergent group-level traits. Behav. Brain Sci. 37, 243–295. (doi:10.1017/S0140525X13001544)
25. Acerbi A, Mesoudi A. 2015 If we are all cultural Darwinians what’s the fuss about? Clarifying recent disagreements in the field of cultural evolution. Biol. Philos. 30, 481–503. (doi:10.1007/s10539-015-9490-2)
26. Popper K 1979 Objective knowledge: an evolutionary approach. Oxford, UK: Oxford University Press.
27. Hull DL. 1988 Science as a process: an evolutionary account of the social and conceptual development of science. Chicago, IL: University of Chicago Press.
28. Larsen PO, von Ins M. 2010 The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics 84, 575–603. (doi:10.1007/s11192-010-0202-z)
29. Bornmann L, Mutz R. 2015 Growth rate of modern science: a bibliometric analysis based on the number of publications and cited references. J. Assoc. Inf. Sci. Technol. 66, 2215–2222. (doi:10.1002/asi.23329)
30. Nabout JC, Parreira MR, Teresa FB, Carneiro FM, da Cunha HF, de Souza Ondei L, Caramori SS, Soares TN. 2015 Publish (in a group) or perish (alone): the trend from single-to multi-authorship in biological papers. Scientometrics 102, 357–364. (doi:10.1007/s11192-014-1385-5)
31. Wardil L, Hauert C. 2015 Cooperation and coauthorship in scientific publication. Phys. Rev. E 91, 012825. (doi:10.1103/PhysRevE.91.012825)
32. Brischoux F, Angelier F. 2015 Academia’s never-ending selection for productivity. Scientometrics 103, 333–336. (doi:10.1007/s11192-015-1534-5)
33. Cyranoski D, Gilbert N, Ledford H, Nayar A, Yahia M. 2011 The PhD factory. Nature 472, 276–279. (doi:10.1038/472276a)
34. Schillebeeckx M, Maricque B, Lewis C. 2013 The missing piece to changing the university culture. Nat. Biotechnol. 31, 938–941. (doi:10.1038/nbt.2706)
35. Powell K. 2015 The future of the postdoc. Nature 520, 144–147. (doi:10.1038/520144a)
36. Kerr E. 1995 Too many scientists, too few academic jobs. Nat. Med. 1, 14. (doi:10.1038/nm0195-14)
37. Clauset A, Arbesman S, Larremore DB. 2015 Systematic inequality and hierarchy in faculty hiring networks. Sci. Adv. 1, e1400005. (doi:10.1126/sciadv.1400005)
38. Ioannidis JPA, Boyack KW, Klavans R. 2014 Estimates of the continuously publishing core in the scientific workforce. PLoS ONE 9, e101698. (doi:10.1371/journal.pone.0101698)
39. Vinkers V, Tijdink J, Otte W. 2015 Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. Br. Med. J. 351, h6467. (doi:10.1136/bmj.h6467)
40. Hirsch JE. 2005 An index to quantify an individual’s scientific research output. Proc. Natl Acad. Sci. USA 102, 16 569–16 572. (doi:10.1073/pnas.0507655102)
41. Goodhart C. 1975 Problems of monetary management: the UK experience. In Papers in monetary economics, volume I. Sydney, Australia: Reserve Bank of Australia.
42. Lucas RE. 1976 Econometric policy evaluation: a critique. In Carnegie-Rochester conference series on public policy, vol. 1, pp. 19–46. Amsterdam, The Netherlands: Elsevier.
43. Bartneck C, Kokkelmans S. 2011 Detecting h-index manipulation through self-citation analysis. Scientometrics 87, 85–98. (doi:10.1007/s11192-010-0306-5)
44. López-Cózar ED, Robinson-García N, Torres-Salinas D. 2014 The Google Scholar experiment: how to index false papers and manipulate bibliometric indicators. J. Assoc. Inf. Sci. Technol. 65, 446–454. (doi:10.1002/asi.23056)
45. Aldhous P. 2011 Journal rejects studies contradicting precognition. New Sci. See https://www.newscientist.com/article/dn20447-journal-rejects-studies-contradicting-precognition.
46. Franco A, Malhotra N, Simonovits G. 2014 Publication bias in the social sciences: unlocking the file drawer. Science 345, 1502–1505. (doi:10.1126/science.1255484)
47. Franco A, Simonovits G, Malhotra N. 2015 Underreporting in political science survey experiments: comparing questionnaires to published results. Polit. Anal. 23, 306–312. (doi:10.1093/pan/mpv006)
48. Schmidt S. 2009 Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Rev. Gen. Psychol. 13, 90–100. (doi:10.1037/a0015108)
49. Makel MC, Plucker JA, Hegarty B. 2012 Replications in psychology research: how often do they really occur? Perspect. Psychol. Sci. 7, 537–542. (doi:10.1177/1745691612460688)
50. Bissell M. 2013 Reproducibility: the risks of the replication drive. Nature 503, 333–334. (doi:10.1038/503333a)
51. Schnall S. 2014 Clean data: statistical artefacts wash out replication efforts. Soc. Psychol. 45, 315–320. (doi:10.1027/1864-9335/a000204)
52. Kahneman D. 2014 A new etiquette for replication. Soc. Psychol. 45, 310–311.
53. Campanario JM. 2000 Fraud: retracted articles are still being cited. Nature 408, 288. (doi:10.1038/35042753)
54. Aschwanden C. 2015 Science isn’t broken. FiveThirtyEight. See http://fivethirtyeight.com/features/science-isnt-broken/.
55. Bennett CM, Miller M, Wolford G. 2009 Neural correlates of interspecies perspective taking in the post-mortem Atlantic salmon: an argument for multiple comparisons correction. Neuroimage 47, S125. (doi:10.1016/S1053-8119(09)71202-9)
56. Wagenmakers E-J, Wetzels R, Borsboom D, van der Maas HL. 2011 Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). J. Pers. Soc. Psychol. 100, 426–432. (doi:10.1037/a0022790)
57. Gigerenzer G. 1998 Surrogates for theories. Theory Psychol. 8, 195–204. (doi:10.1177/095935439808 2006)
58. Gelman A, Loken E. 2014 The statistical crisis in science. Am. Sci. 102, 460–465. (doi:10.1511/2014.111.460)
59. Cohen J. 1992 Statistical power analysis. Curr. Dir. Psychol. Sci. 1, 98–101. (doi:10.1111/1467-8721.ep10768783)
60. Richard FD, Bond CF, Stokes-Zoota JJ. 2003 One hundred years of social psychology quantitatively described. Rev. Gen. Psychol. 7, 331–363. (doi:10.1037/1089-2680.7.4.331)
61. Aguinis H, Beaty JC, Boik RJ, Pierce CA. 2005 Effect size and power in assessing moderating effects of categorical variables using multiple regression: a 30-year review. J. Appl. Psychol. 90, 94–107. (doi:10.1037/0021-9010.90.1.94)
62. Kühberger A, Fritz A, Scherndl T. 2014 Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. PLoS ONE 9, e105825. (doi:10.1371/journal.pone.0105825)
63. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, Munafò MR. 2013 Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365–376. (doi:10.1038/nrn3475)
64. Lakens D, Evers ERK. 2014 Sailing from the seas of chaos into the corridor of stability: practical recommendations to increase the informational value of studies. Perspect. Psychol. Sci. 9, 278–292. (doi:10.1177/1745691614528520)
65. Cohen J. 1962 The statistical power of abnormal-social psychological research: a review. J. Abnorm. Soc. Psychol. 65, 145–153. (doi:10.1037/h0045186)
66. Sedlmeier P, Gigerenzer G. 1989 Do studies of statistical power have an effect on the power of studies? Psychol. Bull. 105, 309–316. (doi:10.1037/0033-2909.105.2.309)
67. Rossi JS. 1990 Statistical power of psychological research: what have we gained in 20 years? J. Consult. Clin. Psychol. 58, 646–656. (doi:10.1037/0022-006X.58.5.646)
68. Vankov I, Bowers J, Munafò MR. 2014 On the persistence of low power in psychological science. Q. J. Exp. Psychol. 67, 1037–1040. (doi:10.1080/17470218.2014.885986)
69. Nosek BA, Spies JR, Motyl M. 2012 Scientific utopia II. Restructuring incentives and practices to promote truth over publishability. Perspect. Psychol. Sci. 7, 615–631. (doi:10.1177/1745691612459058)
70. Smaldino PE, Calanchini J, Pickett CL. 2015 Theory development with agent-based models. Organ. Psychol. Rev. 5, 300–317. (doi:10.1177/2041386614546944)
71. Wimsatt WC. 1987 False models as means to truer theories. In Neutral models in biology (eds M Nitecki, A Hoffman), pp. 23–55. Oxford, UK: Oxford University Press.
72. Epstein JM. 2008 Why model? J. Artif. Soc. Soc. Simul. 11, 12.
73. MacMillan NA, Creelman CD 1991 Detection theory: a user’s guide. Cambridge, UK: Cambridge University Press.
74. Vines TH et al. 2014 The availability of research data declines rapidly with article age. Curr. Biol. 24, 94–97. (doi:10.1016/j.cub.2013.11.014)
75. MacArthur D. 2012 Face up to false positives. Nature 487, 427–428. (doi:10.1038/487427a)
76. Eyre-Walker A, Stoletzki N. 2013 The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations. PLoS Biol. 11, e1001675. (doi:10.1371/journal.pbio.1001675)
77. Lu SF, Jin GZ, Uzzi B, Jones B. 2013 The retraction penalty: evidence from the web of science. Sci. Rep. 3, 3146. (doi:10.1038/srep03146)
78. Rosenblatt M. 2016 An incentive-based approach for improving data reproducibility. Sci. Transl. Med. 8, 336ed5. (doi:10.1126/scitranslmed.aaf5003)
79. Frank RH. 2011 The Darwin economy: liberty, competition, and the common good. Princeton, NJ: Princeton University Press.
80. Wilson DS, Hayes SC, Biglan A, Embry DD. 2014 Evolving the future: toward a science of intentional change. Behav. Brain Sci. 37, 395–460. (doi:10.1017/S0140525X13001593)
81. Nosek BA, Lakens D. 2014 Registered reports: a method to increase the credibility of published results. Soc. Psychol. 45, 137–141. (doi:10.1027/1864-9335/a000192)
82. Ioannidis JPA. 2014 How to make more published research true. PLoS Med. 11, e1001747. (doi:10.1371/journal.pmed.1001747)
83. Fischer J, Ritchie EG, Hanspach J. 2012 An academia beyond quantity: a reply to Loyola et al. and Halme et al. Trends Ecol. Evol. 27, 587–588. (doi:10.1016/j.tree.2012.08.009)
84. Brembs B, Button K, Munafò M. 2013 Deep impact: unintended consequences of journal rank. Front. Hum. Neurosci. 7, 291. (doi:10.3389/fnhum.2013.00291)
85. Begley CG, Buchan AM, Dirnagl U. 2015 Institutions must do their part for reproducibility. Nature 525, 25–27. (doi:10.1038/525025a)
86. MacCoun R, Perlmutter S. 2015 Hide results to seek the truth. Nature 526, 187–189. (doi:10.1038/526187a)
87. Gigerenzer G, Marewski JN. 2015 Surrogate science: the idol of a universal method for scientific inference. J. Manag. 41, 421–440. (doi:10.1177/0149206314547522)
88. Sills J. 2016 Measures of success. Science 352, 28–30. (doi:10.1126/science.352.6281.28)
89. Sarewitz D. 2016 The pressure to publish pushes down quality. Nature 533, 147. (doi:10.1038/533147a)
90. Rand DG, Nowak MA. 2013 Human cooperation. Trends Cogn. Sci. 17, 413–425. (doi:10.1016/j.tics.2013.06.003)
91. North DC. 1990 Institutions, institutional change and economic performance. Cambridge, UK: Cambridge University Press.
92. Aoki M. 2007 Endogenizing institutions and institutional changes. J. Inst. Econ. 3, 1–31. (doi:10.1017/S1744137406000531)
93. Martin BR. 2016 Editors’ JIF-boosting strategems: which are appropriate and which not? Res. Policy 45, 1–7. (doi:10.1016/j.respol.2015.09.001)
94. Ioannidis JPA. 2006 Concentration of the most-cited papers in the scientific literature: analysis of journal ecosystems. PLoS ONE 1, e5. (doi:10.1371/journal.pone.0000005)
95. Fang FC, Casadevall A. 2011 Retracted science and the retraction index. Infect. Immun. 79, 3855–3859. (doi:10.1128/IAI.05661-11)