Replicability and reproducibility in comparative psychology

Frontiers in Psychology

Volumn 8, Issue MAY, 2017, Pages

  Stevens, Jeffrey R ^a  

^a UNIVERSITY OF NEBRASKA LINCOLN   (United States)

Author keywords

Animal research; Comparative psychology; Pre registration; Replication; Reproducible research

Indexed keywords

EID: 85021353846     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2017.00862     Document Type: Article

References (59)

1. Addessi, E., Paglieri, F., Beran, M. J., Evans, T. A., Macchitella, L., De Petrillo, F., et al. (2013). Delay choice versus delay maintenance: different measures of delayed gratification in capuchin monkeys (Cebus apella). J. Comp. Psychol. 127, 392-398. doi: 10.1037/a0031869
2. Agrillo, C., and Petrazzini, M. E. M. (2012). The importance of replication in comparative psychology: the lesson of elephant quantity judgments. Front. Psychol. 3:181. doi: 10.3389/fpsyg.2012.00181
3. Anderson, D. R., Burnham, K. P., Gould, W. R., and Cherry, S. (2001). Concerns about finding effects that are actually spurious. Wildl. Soc. Bull. 29, 311-316. Available online at: http://www.jstor.org/stable/3784014
4. Arnett, J. J. (2008). The neglected 95%: why American psychology needs to become less American. Am. Psychol. 63, 602-614. doi: 10.1037/0003-066X.63.7.602
5. Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., et al. (2013). Recommendations for increasing replicability in psychology. Eur. J. Personal. 27, 108-119. doi: 10.1002/per.1919
6. Association for Psychological Science (2013). Leading Psychological Science Journal Launches Initiative on Research Replication. Available online at: https://www.psychologicalscience.org/news/releases/initiative-on-research-replication.html (Accessed Dec 29, 2016)
7. Bartlett, T. (2014). Replication Crisis in Psychology Research Turns Ugly and Odd. The Chronicle of Higher Education. Available online at: http://www.chronicle.com/article/Replication-Crisis-in/147301/ (Accessed Dec 29, 2016)
8. Beach, F. A. (1950). The Snark was a Boojum. Am. Psychol. 5, 115-124. doi: 10.1037/h0056510
9. Bem, D. J. (2000). "Writing an empirical article," in Guide to Publishing in Psychology Journals, ed R. J. Sternberg (Cambridge: Cambridge University Press), 3-16
10. Bohannon, J. (2014). Replication effort provokes praise-and "bullying" charges. Science 344, 788-789. doi: 10.1126/science.344.6186.788
11. Buckheit, J., and Donoho, D. L. (1995). "Wavelab and reproducible research," in Wavelets and Statistics, eds A. Antoniadis and G. Oppenheim (New York, NY: Springer-Verlag), 55-81
12. Burnham, K. P., and Anderson, D. R. (2010). Model Selection and Multimodel Inference. New York, NY: Springer
13. Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365-376. doi: 10.1038/nrn3475
14. Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., et al. (2016). Evaluating replicability of laboratory experiments in economics. Science 351, 1433-1436. doi: 10.1126/science.aaf0918
15. Center for Open Science (2016). Registered Reports. Available online at: https://cos.io/rr/ (Accessed Dec 29, 2016)
16. Chamberlin, T. C. (1890). The method of multiple working hypotheses. Science 15, 92-96
17. Cohen, J. (1990). Things I have learned (so far). Am. Psychol. 45, 1304-1312. doi: 10.1037/0003-066X.45.12.1304
18. Collins, F. S., and Tabak, L. A. (2014). Policy: NIH plans to enhance reproducibility. Nature 505:612. doi: 10.1038/505612a
19. Coren, S. (2012). How many dogs are there in the world? Psychology Today. Available online at: http://www.psychologytoday.com/blog/canine-corner/201209/how-many-dogs-are-there-in-the-world (Accessed Dec 29, 2016)
20. Fiske, S. T. (2016). A Call to Change Science's Culture of Shaming. APS Observer. Available online at: https://www.psychologicalscience.org/observer/a-call-to-change-sciences-culture-of-shaming
21. Frank, M. C., and Saxe, R. (2012). Teaching replication. Perspect. Psychol. Sci. 7, 600-604. doi: 10.1177/1745691612460686
22. Gartner, M. C., Powell, D. M., and Weiss, A. (2014). Personality structure in the domestic cat (Felis silvestris catus), Scottish wildcat (Felis silvestris grampia), clouded leopard (Neofelis nebulosa), snow leopard (Panthera uncia), and African lion (Panthera leo): a comparative study. J. Comp. Psychol. 128, 414-426. doi: 10.1037/a0037104
23. Gigerenzer, G. (1998). We need statistical thinking, not statistical rituals. Behav. Brain Sci. 21, 199-200. doi: 10.1017/S0140525X98281167
24. Grassman, L., Lynam, A., Mohamad, S., Duckworth, J., Bora, J., Wilcox, D., et al. (2016). Neofelis nebulosa. The IUCN Red List of Threatened Species 2016, e.T14519A97215090. doi: 10.2305/IUCN.UK.2016-1.RLTS.T14519A97215090.en
25. Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., and Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biol. 13:e1002106. doi: 10.1371/journal.pbio.1002106
26. Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Med. 2:e124. doi: 10.1371/journal.pmed.0020124
27. Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., and Ioannidis, J. P. (2016). Reproducible research practices and transparency across the biomedical literature. PLoS Biol. 14:e1002333. doi: 10.1371/journal.pbio.1002333
28. Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, V., Bernstein, M. J., et al. (2014). Investigating variation in replicability. Soc. Psychol. 45, 142-152. doi: 10.1027/1864-9335/a000178
29. Koole, S. L., and Lakens, D. (2012). Rewarding replications: a sure and simple way to improve psychological science. Perspect. Psychol. Sci. 7, 608-614. doi: 10.1177/1745691612462586
30. MacLean, E. L., Hare, B., Nunn, C. L., Addessi, E., Amici, F., Anderson, R. C., et al. (2014). The evolution of self-control. Proc. Natl. Acad. Sci. U.S.A. 111, E2140-E2148. doi: 10.1073/pnas.1323533111
31. Makel, M. C., Plucker, J. A., and Hegarty, B. (2012). Replications in psychology research: how often do they really occur? Perspect. Psychol. Sci. 7, 537-542. doi: 10.1177/1745691612460688
32. Marewski, J. N., and Olsson, H. (2009). Beyond the null ritual: formal modeling of psychological processes. J. Psychol. 217, 49-60. doi: 10.1027/0044-3409.217.1.49
33. Marsh, D. M., and Hanlon, T. J. (2007). Seeing what we want to see: confirmation bias in animal behavior research. Ethology 113, 1089-1098. doi: 10.1111/j.1439-0310.2007.01406.x
34. Masicampo, E. J., and Lalande, D. R. (2012). A peculiar prevalence of p values just below .05. Q. J. Exp. Psychol. 65, 2271-2279. doi: 10.1080/17470218.2012.711335
35. McNutt, M. (2014). Journals unite for reproducibility. Science 346, 679-679. doi: 10.1126/science.aaa1724
36. Munafò, M. R. (2009). Reliability and replicability of genetic association studies. Addiction 104, 1439-1440. doi: 10.1111/j.1360-0443.2009.02662.x
37. Neuliep, J. W., and Crandall, R. (1990). Editorial bias against replication research. J. Soc. Behav. Personal. 5, 85-90
38. Neuliep, J. W., and Crandall, R. (1993). Reviewer bias against replication research. J. Soc. Behav. Personal. 8, 21-29
39. Nickerson, R. S. (1998). Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 175-220. doi: 10.1037/1089-2680.2.2.175
40. Nickerson, R. S. (2000). Null hypothesis significance testing: a review of an old and continuing controversy. Psychol. Methods 5, 241-301. doi: 10.1037/1082-989X.5.2.241
41. Nosek, B. A., Spies, J. R., and Motyl, M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspect. Psychol. Sci. 7, 615-631. doi: 10.1177/1745691612459058
42. Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science 349:aac4716. doi: 10.1126/science.aac4716
43. Patil, P., Peng, R. D., and Leek, J. (2016). A statistical definition for reproducibility and replicability. bioRxiv. doi: 10.1101/066803
44. Platt, J. R. (1964). Strong inference. Science 146, 347-353. doi: 10.1126/science.146.3642.347
45. Prinz, F., Schlange, T., and Asadullah, K. (2011). Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10, 712-712. doi: 10.1038/nrd3439-c1
46. R Core Team (2017). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing
47. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychol. Bull. 86, 638-641. doi: 10.1037/0033-2909.86.3.638
48. Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance test. Psychol. Bull. 57, 416-428. doi: 10.1037/h0042040
49. Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Rev. Gen. Psychol. 13, 90-100. doi: 10.1037/a0015108
50. Shettleworth, S. J. (2009). The evolution of comparative cognition: is the snark still a boojum? Behav. Proc. 80, 210-217. doi: 10.1016/j.beproc.2008.09.001
51. Simmons, J. P., Nelson, L. D., and Simonsohn, U. (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359-1366. doi: 10.1177/0956797611417632
52. Simonsohn, U., Nelson, L. D., and Simmons, J. P. (2014). P-curve: a key to the file-drawer. J. Exp. Psychol. Gen. 143, 534-547. doi: 10.1037/a0033242
53. Sohn, D. (1993). Psychology of the scientist: LXVI. The idiot savants have taken over the psychology labs! or why in science using the rejection of the null hypothesis as the basis for affirming the research hypothesis is unwarranted. Psychol. Rep. 73, 1167-1175. doi: 10.2466/pr0.1993.73.3f.1167
54. Stevens, J. R. (2010). The challenges of understanding animal minds. Front. Psychol. 1:203. doi: 10.3389/fpsyg.2010.00203
55. Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychon. Bull. Rev. 14, 779-804. doi: 10.3758/BF03194105
56. Wagenmakers, E.-J., Wetzels, R., Borsboom, D., and van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). J. Personal. Soc. Psychol. 100, 426-432. doi: 10.1037/a0022790
57. Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., and Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspect. Psychol. Sci. 7, 632-638. doi: 10.1177/1745691612463078
58. Wason, P. C. (1960). On the failure to eliminate hypotheses in a conceptual task. Q. J. Exp. Psychol. 12, 129-140. doi: 10.1080/17470216008416717
59. Wildlife Institute of India (2014). National Studbook Clouded Leopard (Neofelis nebulosa). New Delhi: Wildlife Institute of India, Dehradun and Central Zoo Authority