Making replication mainstream

Behavioral and Brain Sciences

Volumn 41, Issue , 2018, Pages

  Zwaan, Rolf A ^a   Etz, Alexander ^b   Lucas, Richard E ^c   Donnellan, M Brent ^c,d  

^a ERASMUS UNIVERSITY ROTTERDAM   (Netherlands)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c MICHIGAN STATE UNIVERSITY   (United States)

^d TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Psychological research; Replication; Reproducibility; Research programs

Indexed keywords

HUMAN; PSYCHOLOGY; REPLICATION STUDY; REPRODUCIBILITY; SOCIAL MEDIA; THEORETICAL STUDY; ARTICLE;

EID: 85033378669     PISSN: 0140525X     EISSN: 14691825     Source Type: Journal    
DOI: 10.1017/S0140525X17001972     Document Type: Article

References (339)

1. Agnoli, F., Wicherts, J. M., Veldkamp, C. L. S., Albiero, P. & Cubelli, R. (2017) Questionable research practices among Italian research psychologists. PLoS One 12(3):e0172792. Available at: http://doi.org/10.1371/journal.pone.0172792. [MBN]
2. Aklin, M. & Urpelainen, J. (2014) Perceptions of scientific dissent undermine public support for environmental policy. Environmental Science and Policy 38:173-77. Available at: http://doi.org/10.1016/j.envsci.2013.10.006. [MB]
3. Alexander, D. M., Jurica, P., Trengove, C., Nikolaev, A. R., Gepshtein, S., Zvyagintsev, M., Mathiak, K., Schulze-Bonhage, A., Reuscher, J., Ball, T. & van Leeuwen, C. (2013) Traveling waves and trial averaging: The nature of singletrial and averaged brain responses in large-scale cortical signals. Neuro Image 73:95-112. Available at: https://doi.org/10.1016/j.neuroimage.2013.01.016. [DMA]
4. Alexander, D. M., Trengove, C. & van Leeuwen, C. (2015) Donders is dead: Cortical traveling waves and the limits of mental chronometry in cognitive neuroscience. Cognitive Processing 16(4):365-75. Available at: https://doi.org/10.1007/s10339-015-0662-4. [DMA]
5. Alogna, V. K., Attaya, M. K., Aucoin, P., Bahnik, Š., Birch, S., Birt, A. R., Bornstein, B. H., Bouwmeester, S., Brandimonte, M. A., Brown, C., Buswell, K., Carlson, C., Carlson, M., Chu, S., Cislak, A., Colarusso, M., Colloff, M. F., Dellapaolera, K. S., Delvenne, J.-F., Di Domenico, A., Drummond, A., Echterhoff, G., Edlund, J. E., Eggleston, C. M., Fairfield, B., Franco, G., Gabbert, F., Gamblin, B. W., Garry, M., Gentry, R., Gilbert, E. A., Greenberg, D. L., Halberstadt, J., Hall, L., Hancock, P. J. B., Hirsch, D., Holt, G., Jackson, J. C., Jong, J., Kehn, A., Koch, C., Kopietz, R., Körner, U., Kunar, M. A., Lai, C. K., Langton, S. R. H., Leite, F. P., Mammarella, N., Marsh, J. E., McConnaughy, K. A., McCoy, S., McIntyre, A. H., Meissner, C. A., Michael, R. B., Mitchell, A. A., Mugayar-Baldocchi, M., Musselman, R., Ng, C., Nichols, A. L., Nunez, N. L., Palmer, M. A., Pappagianopoulos, J. E., Petro, M. S., Poirier, C. R., Portch, E., Rainsford, M., Rancourt, A., Romig, C., Rubínová, E., Sanson, M., Satchell, L., Sauer, J. D., Schweitzer, K., Shaheed, J., Skelton, F., Sullivan, G. A., Susa, K. J., Swanner, J. K., Thompson, W. B., Todaro, R., Ulatowska, J., Valentine, T., Verkoijen, P. P. J. L., Vranka, M., Wade, K. A., Was, C. A., Weatherford, D., Wiseman, K., Zaksaite, T., Zuj, D. V. & Zwaan, R. A. (2014) Registered replication report: Schooler & Engstler-Schooler (1990). Perspectives on Psychological Science 9:556-78. [aRAZ]
6. Anderson, C. J., Bahník, Š., Barnett-Cowan, M., Bosco, F. A., Chandler, J., Chartier, C. R., Cheung, F., Christopherson, C. D., Cordes, A., Cremata, E. J., Della Penna, N., Estel, V., Fedor, A., Fitneva, S. A., Frank, M. C., Grange, J. A., Hartshorne, J. K., Hasselman, F., Henninger, F., van der Hulst, M., Jonas, K. J., Lai, C. K., Levitan, C. A., Miller, J. K., Moore, K. S., Meixner, J. M., Munafò, M. R., Neijenhuijs, K. I., Nilsonne, G., Nosek, B. A., Plessow, F., Prenoveau, J. M., Ricker, A. A., Schmidt, K., Spies, J. R., Steiger, S., Strohminger, N., Sullivan, G. B., van Aert, R. C. M., van Assen, M. A. L. M., Vanpaemel, W., Vianello, M., Voracek, M. & Zuni, K. (2016) Response to comment on "estimating the reproducibility of psychological science." Science 351(6277):1037. [aRAZ, TEH]
7. Anderson, S. F., Kelly, K. & Maxwell, S. E. (2017) Sample-size planning for more accurate statistical power: A method of adjusting sample effect sizes for publication bias and uncertainty. Psychological Science 28(11):1547-62. [PDLH]
8. Arechar, A. A., Kraft-Todd, G. T. & Rand, D. G. (2017) Turking overtime: How participant characteristics and behavior vary over time and day on Amazon Mechanical Turk. Journal of the Economic Science Association 3(1):1-11. [GPa]
9. Baker, M. (2015, August 27) Over half of psychology studies fail reproducibility test. Nature News. Available at: https://www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.18248. [FS]
10. Baker, M. (2016) Is there a reproducibility crisis? Nature 533:452-54. [aRAZ]
11. Bakker, M., van Dijk, A. & Wicherts, J. M. (2012) The rules of the game called psychological science. Perspectives on Psychological Science 7(6):543-54. Available at: http://doi.org/10.1177/1745691612459060. [EHW, MBN]
12. Bakker, M. & Wicherts, J. M. (2011) The (mis)reporting of statistical results in psychology journals. Behavior Research Methods 43(3):666-78. Available at: http://doi.org/10.3758/s13428-011-0089-5. [MBN]
13. Bakker, M. & Wicherts, J. M. (2014) Outlier removal and the relation with reporting errors and quality of research. PLoS One 9(7):e103360. Available at: http://doi. org/10.1371/journal.pone.0103360. [MBN]
14. Baribault, B., Donkin, C., Little, D. R., Trueblood, J. S., Oravecz, Z., van Ravenzwaaij, D., White, C. N., De Boeck, P. & Vandekerckhove, J. (2018) Metastudies for robust tests of theory. Proceedings of the National Academy of Sciences of the United States of America 115(11):2607-12. Available at: http:/doi.org/10. 1073/pnas.1708285114. [rRAZ, TEH]
15. Barrera, M., Jr. & Rosen, G. M. (1977) Detrimental effects of a self-reward contracting program on subjects' involvement in self-administered desensitization. Journal of Consulting and Clinical Psychology 45:1180-81. [SOL]
16. Barsalou, L. W. (2016) Situated conceptualization offers a theoretical account of social priming. Current Opinion in Psychology 12:6-11. [aRAZ]
17. Baumeister, R. F. (2016) Charting the future of social psychology on stormy seas: Winners, losers, and recommendations. Journal of Experimental Social Psychology 66:153-58. Available at: http://doi.org/10.1016/j.jesp.2016.02.003. [aRAZ]
18. Baumeister, R. F., Bratslavsky, E., Muraven, M. & Tice, D. M. (1998) Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology 74(5):1252-65. [AK]
19. Bem, D. J. (2003) Writing the empirical journal article. In: The compleat academic: A career guide, 2nd edition, ed. J. M. Darley, M. P. Zanna & H. L. Roediger III, pp. 185-219. American Psychological Association. [aRAZ]
20. Bem, D. J. (2011) Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology 100:407-25. [aRAZ]
21. Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., Cesarini, D., Chambers, C. D., Clyde, M., Cook, T. D., De Boeck, P., Dienes, Z., Dreber, A., Easwaran, K., Efferson, C. Fehr, E., Fidler, F., Field, A. P., Forster, M., George, E. I., Gonzalez, R., Goodman, S., Green, E., Green, D. P., Greenwald, A. G., Hadfield, J. D., Hedges, L. V., Held, L., Ho, T.-H., Hoijtink, H., Hruschka, D. J., Imai, K., Imbens, G., Ioannidis, J. P. A., Jeon, M., Jones, J. H., Kirchler, M., Laibson, D., List, J., Little, R., Lupia, A., Machery, E., Maxwell, S. E., McCarthy, M., Moore, D. A., Morgan, S. L., Munafò, M., Nakagawa, S., Nyhan, B., Parker, T. H., Pericchi, L., Perugini, M., Rouder, J., Rousseau, J., Savalei, V., Shönbrodt, F. D., Sellke, T., Sinclair, B., Tingley, D., Van Zandt, T., Vazire, S., Watts, D. J., Winship, C., Wolpert, R. L., Xie, Y., Young, C., Zinman, J. & Johnson, V. E. (2017) Redefine statistical significance. Nature Human Behaviour 2:6-10. Available at: http://doi.org/10.1038/s41562-017-0189-z. [JPAI]
22. Berk, R. A., Campbell, A., Klap, R. & Western, B. (1992) The deterrent effect of arrest in incidents of domestic violence: A Bayesian analysis of four field experiments. American Sociological Review 57:698-708. [RJM]
23. Berman, J. S. & Reich, C. M. (2010) Investigator allegiance and the evaluation of psychotherapy outcome research. European Journal of Psychotherapy and Counselling 12:11-21. [WT]
24. Blakeley, B., McShane, B. B., Gal, D., Gelman, A., Robert, C. & Tackett, J. L. (2018) Abandon statistical significance. Preprint. Available at: https://arxiv.org/abs/1709.07588. [FS]
25. Blakey, S. M. & Abramowitz, J. S. (2016) The effects of safety behaviors during exposure therapy for anxiety: Critical analysis from an inhibitory learning perspective. Clinical Psychology Review 49:1-15. [SOL]
26. Boekel, W., Wagenmakers, E.-J., Belay, L., Verhagen, J., Brown, S. & Forstmann, B. U. (2015) A purely confirmatory replication study of structural brain-behavior correlations. Cortex 66:115-33. Available at: https://doi.org/10.1016/j.cortex.2014.11. 019. [KG]
27. Bohannon, J. (2014) Replication effort provokes praise - and 'bullying' charges. Science 344:788-89. [GPe, WT]
28. Bonett, D. G. (2012) Replication-extension studies. Current Directions in Psychological Science 21:409-12. [rRAZ]
29. Borenstein, M., Hedges, L. V., Higgins, J. P. T. & Rothstein, H. R. (2009) Introduction to meta-analysis. Wiley. [TEH]
30. Bowen, A. & Casadevall, A. (2015) Increasing disparities between resource inputs and outcomes, as measured by certain health deliverables, in biomedical research. Proceedings of the National Academy of Sciences of the United States of America 112:11335-40. [JPAI]
31. Box, G. E. P. (1979) Robustness in the strategy of scientific model building. In: Robustness in statistics, ed. R. L. Launer & G. N. Wilkinson, pp. 201-36. Academic. [rRAZ]
32. Brandt, M. J., Ijzerman, H., Dijksterhuis, A., Farach, F., Geller, J., Giner-Sorolla, R., Grange, J. A., Perugini, M., Spies, J. & van't Veer, A. (2014) The replication recipe: What makes for a convincing replication? Journal of Experimental Social Psychology 50:217-24. [aRAZ, HI]
33. Brown, N. J. L. & Heathers, J. A. J. (2017) The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science 8(4):363-69. Available at: http://doi.org/10.1177/1948550616673876. [MBN]
34. Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. & Munafò, M. R. (2013) Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14(5):365-76. [aRAZ]
35. Camerer, C. F., Dreber, A., Forsell, E., Ho, T. H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmedj, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M. & Wu, H. (2016) Evaluating replicability of laboratory experiments in economics. Science 351(6280):1433-36. [aRAZ]
36. Campbell, D. T. (1969) Reforms as experiments. American Psychologist 24(4):409. [AGa]
37. Campbell, D. T. (1991) Methods for the experimenting society. Evaluation Practice 12(3):223-60. [AGa]
38. Carlin, J. B. (2016) Is reform possible without a paradigm shift? The American Statistician, Supplemental material to the ASA statement on p-values and statistical significance 10. [JLT]
39. Casey, L., Chandler, J., Levine, A. S., Proctor, A. & Strolovitch, D. Z. (2017, April-June) Intertemporal differences among MTurk worker demographics. SAGE Open, 1-15. doi: 10.1177/2158244017712774. [GPa]
40. Cesario, J. (2014) Priming, replication, and the hardest science. Perspectives on Psychological Science 9:40-48. [aRAZ]
41. Chambers, C. (2017) The 7 deadly sins of psychology: A manifesto for reforming the culture of scientific practice. Princeton University Press. [aRAZ]
42. Chambers, C.D. (2013) Registered reports:A newpublishing initiative at Cortex. Cortex 49(3):609-10. Available at: http://doi.org/10.1016/j.cortex.2012.12.016. [ARK]
43. Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P. & Willmes, K. (2015) Registered reports: Realigning incentives in scientific publishing. Cortex 66:A1-A2. [RG-S]
44. Chambless, D. L. & Ollendick, T. H. (2001) Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology 52:685-716. [SOL]
45. Chandler, J., Mueller, P. & Paolacci, G. (2014) Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods 46(1):112-30. [GPa]
46. Chandler, J., Paolacci, G., Peer, E., Mueller, P. & Ratliff, K. A. (2015) Using nonnaive participants can reduce effect sizes. Psychological Science 26(7):1131-39. [GPa]
47. Chandler, J. & Shapiro, D. (2016) Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology 12:53-81. [GPa]
48. Chartier, C. R. (2017) The psychological science accelerator: A distributed laboratory network. Blog post. Available at: https://christopherchartier.com/2017/09/21/the-psychological-science-accelerator-a-distributed-laboratory-network. [WT]
49. Chavalarias, D., Wallach, J., Li, A. & Ioannidis, J. P. A. (2016) Evolution of reporting of p-values in the biomedical literature, 1990-2015. Journal of the American Medical Association 315(11):1141-48. [JPAI]
50. Chawla, D. S. (2017, April 7) Online platform aims to facilitate replication studies. The Scientist. Available at: https://www.the-scientist.com/?articles.view/articleNo/49161/title/Online-Platform-Aims-to-Facilitate-Replication-Studies/. [MAG]
51. Cheung, I., Campbell, L., LeBel, E. P., Ackerman, R. A., Aykutoǧlu, B., Bahník, Š., Bowen, J. D., Bredbow, C. A., Bromberg, C., Caprariello, P. A., Carcedo, R. J., Carson, K. J., Cobb, R. J., Collins, N. L., Corretti, C. A., DiDonato, T. E., Ellithorpe, C., Fenrnández-Rouco, N., Fuglestad, P. T., Goldberg, R. M., Golom, F. D., Gündoǧdu-Aktürk, E., Hoplock, L. B., Houdek, P., Kane, H. S., Kim, J. S., Kraus, S., Leone, C. T., Li, N. P., Logan, J. M., Millman, R. D., Morry, M. M., Pink, J. C., Ritchey, T., Root Luna, L. M., Sinclair, H. C., Stinson, D. A., Sucharyna, T. A., Tidwell, N. D., Uysal, A., Vranka, M., Winczewski, L. A. & Yong, J. C. (2016) Registered replication report: Study 1 from Finkel, Rusbult, Kumashiro & Hannon (2002). Perspectives on Psychological Science 11(5):750-64. [aRAZ]
52. Cohen, J. (1990) Things I have learned (so far). American Psychologist 45(12):1304. [aRAZ]
53. Cohen, J. (1994) The earth is round (p < .05). American Psychologist 49:997-1003. [TC]
54. Collaborative Replications and Education Project (2018) Current study list and selection methods. Available at: https://osf.io/flaue/wiki/home/. [RG-S]
55. Cook, D. J., Guyatt, G. H., Ryan, G., Clifton, J., Buckingham, L., Willan, A., Mcllor, W. & Oxman, A. D. (1993) Should unpublished data be included in metaanalyses? Current convictions and controversies. JAMA 269(21):2749-53. [aRAZ]
56. Cook, T. D., Campbell, D. T. & Peracchio, L. (1990) Quasi-experimentation. In: Handbook of industrial and organizational psychology, vol. 1, 2nd edition, ed. M. D. Dunnette & L. M. Hough, pp. 491-576. Consulting Psychologists. [SOL]
57. Cortina, J. M., Aguinis, H. & DeShon, R. P. (2017) Twilight of dawn or of evening? A century of research methods in the Journal of Applied Psychology. Journal of Applied Psychology 102(3):274-90. [RJM]
58. Coyne, J. C. (2016) Replication initiatives will not salvage the trustworthiness of psychology. BMC Psychology 4:28. Available at: http://doi.org/10.1186/s40359-016-0134-3. [aRAZ, SOL]
59. Crandall, C. S. & Sherman, J. W. (2016) On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology 66:93-99. Available at: http://doi.org/10.1016/j.jesp.2015.10.002. [aRAZ, AMT, US]
60. Cronbach, L. J. & Meehl, P. E. (1955) Construct validity in psychological tests. Psychological Bulletin 52:281-302. [SOL]
61. Deutsche Forschungsgemeinschaft (2017) Die Replizierbarkeit von Forschungsergebnissen (The replicability of research findings). Available at: www.dfg.de/download/pdf/dfg-im-profil/reden-stellungnahmen/2017/170425-stellungnahme-replizierbarkeit-forschungsergebnisse-en.pdf. [FS]
62. DeVoe, S. E. & House, J. (2016). Replications with MTurkers who are naïve versus experienced with academic studies: A comment on Connors, Khamitov, Moroz, Campbell, and Henderson (2015). Journal of Experimental Social Psychology 67:65-67. [GPa]
63. Difallah, D., Filatova, E. & Ipeirotis, P. (2018) Demographics and dynamics of mechanical Turk workers. In: Proceedings of WSDM 2018: The Eleventh ACM International Conference on Web Search and Data Mining, Marina Del Rey, CA, USA February 5-9, 2018, pp. 135-143. Available at: https://dl.acm.org/citation.cfm?doid=3159652.3159661. [GPa]
64. Donnellan, M. B., Lucas, R. E. & Cesario, J. (2015) On the association between loneliness and bathing habits: Nine replications of Bargh and Shalev (2012): Study 1. Emotion 15(1):109-19. [rRAZ]
65. Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., Nosek B. A. & Johannesson, M. (2015) Using prediction markets to estimate the reproducibility of scientific research. Proceedings of the National Academy of Sciences of the United States of America 112:15343-47. [WT]
66. Duarte, J. L., Crawford, J. T., Stern, C., Haidt, J., Jussim L. & Tetlock, P. E. (2015) Political diversity will improve social psychological science. Behavioral and Brain Sciences 38:e130. Available at: http://doi.org/10.1017/S0140525X14000430. [BE]
67. Dubé, C., Rotello, C. M. & Heit, E. (2010) Assessing the belief bias effect with ROCs: It's a response bias effect. Psychological Review 117:831-63. [EH]
68. Dudo, A., Dunwoody, S. & Scheufele, D. A. (2011) The emergence of nano news: Tracking thematic trends and changes in US newspaper coverage of nanotechnology. Journalism and Mass Communication Quarterly 88:55-75. Available at: http://doi.org/10.1177/107769901108800104. [MB]
69. Duhem, P. (1954) The aim and structure of physical theory. Princeton University Press. [AOH]
70. Dunlap, K. (1926) The experimental methods of psychology. In: Psychologies of 1925, ed. C. Murchison, pp. 331-53. Clark University Press. [aRAZ, WT]
71. Dunning, T., Grossman, G., Humphreys, M., Hyde, S. & McIntosh, C., eds. (in press) Information and accountability: A new method for cumulative learning. Cambridge University Press. [AGa]
72. Ebersole, C. R., Alaei, R., Atherton, O. E., Bernstein, M. J., Brown, M., Chartier, C. R., Chung, L. Y., Hermann, A. D., Joy-Gaba, J. A., Line, M. J., Rule, N. O., Sacco, D. F., Vaughn, L. A. & Nosek, B. A. (2017) Observe, hypothesize, test, repeat: Luttrell, Petty & Xu (2017) demonstrate good science. Journal of Experimental Social Psychology 69:184-86. [DTW]
73. Ebersole, C. R., Atherton, O. E., Belanger, A. L., Skulborstad, H. M., Allen, J. M., Banks, J. B., Baranski, E., Bernstein, M. J., Bofiglio, D. B. V., Boucher, L., Brown, E. R., Budima, N. I., Cairo, A. H., Capaldi, C. A., Chartier, C. R., Chung, J. M., Cicero, D. C., Coleman, J. A., Conway, J. G., Davis, W. E., Devos, T., Fletcher, M. M., German, K., Grahe, J. E., Hermann, A. D., Hicks, J. A., Honeycutt, N., Humphrey, B., Janus, M., Johnson, D. J., Joy-Gaba, J. A., Juzeler, H., Keres, A., Kinney, D., Kirschenbaum, J., Klein, R. A., Lucas, R. E., Lustgraff, C. J. N., Martin, D., Menon, M., Metzger, M., Moloney, J. M., Morse, P. J., Prislin, R., Razza, T., Re, D. E., Rule, N. O., Sacco, D. F., Sauerberger, K., Shrider, E., Shultz, M., Siesman, C., Sobocko, K., Sternglanz, R. W., Summerville, A., Tskhay, K. O., van Allen, Z., Vaughn, L. A., Walker, R. J., Weinberg, A., Wilson, J. P., Wirth, J. H., Wortman, J. & Nosek, B. A. (2016a) Many Labs 3: Evaluating participant pool quality across the academic semester via replication. Journal of Experimental Social Psychology 67:68-82. Available at: http://doi.org/10.1016/j.jesp.2015.10.012. [DTW, RG-S, SS]
74. Ebersole, C. R., Axt, J. R. & Nosek, B. A. (2016b) Scientists' reputations are based on getting it right, not being right. PLoS Biology 14(5):e1002460. Available at: https://doi.org/10.1371/journal.pbio.1002460. [aRAZ, GPe]
75. Ebrahim, S., Sohani, Z. N., Montoya, L., Agarwal, A., Thorlund, K., Mills, E. J. & Ioannidis, J. P. (2014) Reanalysis of randomized clinical trial data. Journal of the American Medical Association 312(10):1024-32. Available at: http://doi.org/10. 1001/jama.2014.9646. [MBN]
76. Eerland, A., Sherrill, A. M., Magliano, J. P., Zwaan, R. A., Arnal, J. D., Aucoin, P., Berger, S. A., Birt, A. R., Capezza, N., Carlucci, M., Crocker, C., Ferretti, T. R., Kibbe, M. R., Knepp, M. M., Kurby, C. A., Melcher, J. M., Michael, S. W., Poirier, C. & Prenoveau, J. M. (2016) Registered replication report: Hart & Albarracín (2011). Perspectives on Psychological Science 11:158-71. [arRAZ]
77. Epskamp, S. & Nuijten, M. B. (2016) statcheck: Extract statistics from articles and recompute p-values. Available at: https://cran.r-project.org/web/packages/statcheck/ (R package version 1.2.2). [MBN]
78. Errington, T. M., Iorns, E., Gunn, W., Tan, F. E., Lomax, J. & Nosek, B. A. (2014) An open investigation of the reproducibility of cancer biology research. Elife 3: e04333. [aRAZ]
79. Estes, W. K. (1956) The problem of inference from curves based on group data. Psychological Bulletin 53(2):134-40. [DMA]
80. Etz, A. (2015, August 30) The Bayesian Reproducibility Project. Weblog post. Retrieved 23 August 2017 from: https://web.archive.org/web/20160407113631/http://alexanderetz.com:80/2015/08/30/the-bayesian-reproducibility-project/. [aRAZ]
81. Etz, A., Haaf, J. M., Rouder, J. N. & Vandekerckhove, J. (in press) Bayesian inference and testing any hypothesis you can specify. Preprint. Available at: https://psyarxiv.com/wmf3r/. [rRAZ]
82. Etz, A. & Vandekerckhove, J. (2016) A Bayesian perspective on the reproducibility project: Psychology. PLoS ONE 11(2):e0149794. Available at: http://doi.org/10. 1371/journal.pone.0149794. [aRAZ]
83. Etz, A. & Wagenmakers, E. J. (2017) JBS Haldane's contribution to the Bayes factor hypothesis test. Statistical Science 32(2):313-29. [aRAZ]
84. Evans, J. St. B. T., Barston, J. L. & Pollard, P. (1983) On the conflict between logic and belief in syllogistic reasoning. Memory and Cognition 11:295-306. [EH]
85. Everett, J. A. & Earp B. D. (2015) A tragedy of the (academic) commons: Interpreting the replication crisis in psychology as a social dilemma for early-career researchers. Frontiers in Psychology 6:1-4. [WT]
86. Fabrigar, L. R. & Wegener, D. T. (2016) Conceptualizing and evaluating the replication of research results. Journal of Experimental Social Psychology 66:68-80. [aRAZ, DTW]
87. Fanelli, D. (2011) Negative results are disappearing from most disciplines and countries. Scientometrics 90(3):891-904. Available at: http://doi.org/10.1007/s11192-011-0494-7. [ARK]
88. Fanelli, D., Costas, R. & Ioannidis, J. P. A. (2017) A meta-assessment of bias in science. Proceedings of the National Academy of Sciences of the United States of America 114:3714-19. [JPAI]
89. Fayant, M. P., Sigall, H., Lemonnier, A., Retsin, E. & Alexopoulos, T. (2017) On the limitations of manipulation checks: An obstacle toward cumulative science. International Review of Social Psychology 30(1):125-30. Available at: https://doi.org/10.5334/irsp.102. [EHW]
90. Ferguson, C. J. & Brannick, M. T. (2012) Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods 17(1):120-28. Available at: http://doi.org/10.1037/a0024445. [aRAZ, ARK]
91. Ferguson, C. J. & Heene, M. (2012) A vast graveyard of undead theories publication bias and psychological science's aversion to the null. Perspectives on Psychological Science 7(6):555-61. [aRAZ]
92. Fetterman, A. K. & Sassenberg, K. (2015) The reputational consequences of failed replications and wrongness admission among scientists. PLoS One 10(12):e0143723. Available at: https://doi.org/10.1371/journal.pone.0143723. [aRAZ, GPe]
93. Fiedler, K. & Schwarz N. (2015) Questionable research practices revisited. Social Psychological and Personality Science 7:45-52. Available at: http://doi.org/10. 1177/1948550615612150. [aRAZ]
94. Finkel, E. J., Eastwick, P. W. & Reis, H. T. (2015) Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science. Journal of Personality and Social Psychology 108:275-97. Available at: http://doi.org/10.1037/pspi0000007. [aRAZ]
95. Fiske, S. T. (1992) Thinking is for doing: Portraits of social cognition from daguerreotype to laserphoto. Journal of Personality and Social Psychology 63 (6):877. [AGa]
96. Fletcher, P. C. & Grafton, S. T. (2013) Repeat after me: Replication in clinical neuroimaging is critical. NeuroImage: Clinical 2:247-48. Available at: https://doi.org/10.1016/j.nicl.2013.01.007. [KG]
97. Foss, D. J. & Blank, M. A. (1980) Identifying the speech codes. Cognitive Psychology 12(1):1-31. Available at: https://doi.org/10.1016/0010-0285(80)90002-X. [MAG]
98. Foss, D. J. & Gernsbacher, M. A. (1983) Cracking the dual code: Toward a unitary model of phoneme identification. Journal of Verbal Learning and Verbal Behavior 22:609-32. Available at: https://doi.org/10.1016/S0022-5371(83) 90365-1. [MAG]
99. Franco, A., Malhotra, N. & Simonovits, G. (2014) Publication bias in the social sciences: Unlocking the file drawer. Science 345(6203):1502-505. [aRAZ]
100. Frankfurt, H. (2005) On bullshit. Princeton University Press. Available at: http://journals.cambridge.org/production/action/cjoGetFulltext?fulltextid=5452992. [GPe]
101. Freudenburg, W. R., Gramling, R. & Davidson, D. J. (2008) Scientific certainty argumentation methods (SCAMs): Science and the politics of doubt. Sociological Inquiry 78:2-38. Available at: http://doi.org/10.1111/j.1475-682X.2008. 00219.x. [MB]
102. Garnham, A., Traxler, M., Oakhill, J. & Gernsbacher, M. A. (1996) The locus of implicit causality effects in comprehension. Journal of Memory and Language 35:517-43. Available at: https://doi.org/10.1006/jmla.1996.0028. [MAG]
103. Gelman, A. (2013) I'm negative on the expression "false positives." Blog post. Available at: http://andrewgelman.com/2013/11/07/nix-expression-false-positives/. [AGe]
104. Gelman, A. (2015) The connection between varying treatment effects and the crisis of unreplicable research: A Bayesian perspective. Journal of Management 41 (2):632-43. [JLT]
105. Gelman, A. (2016a) The problems with p-values are not just with p-values. The American Statistician, Supplemental material to the ASA statement on p-values and statistical significance 10. [JLT]
106. Gelman, A. (2016b) The time-reversal heuristic - A new way to think about a published finding that is followed up by a large, preregistered replication (in context of Amy Cuddy's claims about power pose). Blog post. Available at: http://andrewgelman.com/2016/01/26/more-power-posing/. [AGe, AMT]
107. Gelman, A. & Carlin, J. B. (2014) Beyond power calculations: Assessing Type S (sign) and Type M (magnitude) errors. Perspectives on Psychological Science 9:641-51. [AGe]
108. Gelman, A. & Loken, E. (2014) The statistical crisis in science data-dependent analysis - a "garden of forking paths" - explains why many statistically significant comparisons don't hold up. American Scientist 102(6):460. [aRAZ]
109. Gernsbacher, M. A. (1989) Mechanisms that improve referential access. Cognition 32:99-156. Available at: https://doi.org/10.1016/0010-0277(89) 90001-2. [MAG]
110. Gernsbacher, M. A. & Hargreaves, D. J. (1988) Accessing sentence participants: The advantage of first mention. Journal of Memory and Language 27:699-717. [MAG]
111. Gernsbacher, M. A., Keysar, B., Robertson, R. R. W. & Werner, N. K. (2001a) The role of suppression and enhancement in understanding metaphors. Journal of Memory and Language 45:433-50. Available at: https://doi.org/10.1006/jmla. 2000.2782. [MAG]
112. Gernsbacher, M. A., Robertson, R. R. W. & Werner, N. K. (2001b) The costs and benefits of meaning. In: On the consequences of meaning selection: Perspectives on resolving lexical ambiguity, ed. D. S. Gorfein, pp. 119-37. APA. Available at: https://doi.org/10.1037/10459-007. [MAG]
113. Ghelfi, E., Christopherson, C. D., Fischer, M. A., Legate, N., Lenne, R., Urry, H., Wagemans, F. M. A., Wiggins, B., Barrett, T., Glass, M., Guberman, J., Hunt, J., Issa, N., Paulk, A., Peck, T., Perkinson, J., Sheelar, K., Theado, R. & Turpin, R. (in preparation) The influence of gustatory disgust on moral judgement: A pre-registered multi-lab replication. [HI]
114. Gilbert, D. T., King, G., Pettigrew, S, & Wilson, T. D. (2016) Comment on "estimating the reproducibility of psychological science." Science 351(6277):1037. Available at: http://doi.org/10.1126/science.aad7243. [aRAZ, MB, RG-S, TEH]
115. Giner-Sorolla, R. (2016) Approaching a fair deal for significance and other concerns. Journal of Experimental Social Psychology 65:1-6. [RG-S]
116. Giofrè, D., Cumming, G., Fresc, L., Boedker, I. & Tressoldi, P. (2017) The influence of journal submission guidelines on authors' reporting of statistics and use of open research practices. PLoS One 12(4):e0175583. Available at: http://doi.org/10.1371/journal.pone.0175583. [MBN]
117. Glick, A. R. (2015) The role of serotonin in impulsive aggression, suicide, and homicide in adolescents and adults: A literature review. International Journal of Adolescent Medicine and Health 27:143-50. [SOL]
118. Goldin-Meadow, S. (2016) Preregistration, replication, and nonexperimental studies. Association for Psychological Science Observer 29(8):2. [NAC]
119. Goodman, J. K. & Paolacci, G. (2017) Crowdsourcing consumer research. Journal of Consumer Research 44(1):196-210. [GPa]
120. Gorgolewski, K., Nichols, T., Kennedy, D. N., Poline, J.-B. & Poldrack, R. A. (2017a) Promoting replications through positive incentives. Figshare. Available at: https://doi.org/10.6084/m9.figshare.5278327.v1. [KG]
121. Gorgolewski, K., Nichols, T., Kennedy, D. N., Poline, J.-B. & Poldrack, R. A. (2017b) Replication award creation kit. Figshare. Available at: https://doi.org/10.6084/m9.figshare.5567083.v1. [KG]
122. Grahe, J., Brandt, M., Ijzerman, H. & Cohoon, J. (2014, February 28) Replication education. Association for Psychological Science: Observer. Available at: https://www.psychologicalscience.org/observer/replication-education. [MAG]
123. Grahe, J. E., Brandt, M. J., Ijzerman, H., Cohoon, J., Peng, C., Detweiler-Bedell, B. & Weisberg, Y. (2015) Collaborative Replications and Education Project (CREP). Available at: http://osf.io/wfc6u. [HI]
124. Gray, K. (2017) How to map theory: Reliable methods are fruitless without rigorous theory. Perspectives on Psychological Science 12(5):731-41. [TC]
125. Greenberg, J., Solomon, S., Pyszczynski, T. & Steinberg, L. (1988) A reaction to Greenwald, Pratkanis, Leippe, and Baumgardner (1986): Under what conditions does research obstruct theory progress? Psychological Review 95:566-71. [HI]
126. Greenwald, A. G. (1975) Significance, nonsignificance, and interpretation of an ESP experiment. Journal of Experimental Social Psychology 11:180-91. [aRAZ]
127. Greenwald, A. G., Pratkanis, A. R., Leippe, M. R. & Baumgardner, M. H. (1986) Under what conditions does theory obstruct research progress? Psychological Review 93:216. [HI]
128. Grice, J., Barrett, P., Cota, L., Felix, C., Taylor, Z., Garner, S., Medellin, E. & Vest, A. (2017) Four bad habits of modern psychologists. Behavioral Sciences 7:53. [DRL]
129. Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., Brand, R., Brandt, M. J., Brewer, G., Bruyneel, S., Calvillo, D. P., Campbell, W. K., Cannon, P. R., Carlucci, M., Carruth, N. P., Cheung, T., Crowell, A., De Ridder, D. T. D., Dewitte, S., Elson, M., Evans, J. R., Fay, B. A., Fennis, B. M, Finley, A., Francis, Z., Heise, E., Hoemann, H., Inzlicht, M., Koole, S. L., Koppel, L., Kroese, F., Lange, F., Lau, K., Lynch, B. P., Martijn, C., Merckelbach, H., Mills, N. V., Michirev, A., Miyake, A., Mosser, A. E., Muise, M., Muller, D., Muzi, M., Nalis, D., Nurwanti, R., Otgaar, H, Philipp, M. C., Primoceri, P., Rentzsch, K., Ringos, L., Schlinkert, C., Schmeichel, B. J., Schoch, S. F., Schrama, M., Schütz, A., Stamos, A., Tinghög, G., Ullrich, J., vanDellen, M., Wimbarti, S., Wolff, W., Yusainy, C., Zerhouni, O. & Zwienenberg, M. (2016) A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science 11(4):546-73. [arRAZ, AK, EHW]
130. Hawthorne, J. (2014) Bayesian confirmation theory. In: The Bloomsbury Companion to the Philosophy of Science, ed. S. French & J. Saatsi, p. 197. Bloomsbury Academic. [AOH]
131. Heit, E., Hahn, U. & Feeney, A. (2005) Defending diversity. In: Categorization inside and outside the laboratory: Essays in honor of Douglas L. Medin, ed. W.-K. Ahn, R. L. Goldstone, B. C. Love, A. B. Markman & P. Wolff, pp. 87-99. American Psychological Association. [EH]
132. Heit, E. & Rotello, C. M. (2014) Traditional difference-score analyses of reasoning are flawed. Cognition 131:75-91. [EH]
133. Henrich, J., Heine, S. J. & Norenzayan, A. (2010) Most people are not WEIRD. Nature 466(7302):29. [DJS]
134. Hewitt, J. K. (2012) Editorial policy on candidate gene association and candidate gene-by-environment interaction studies of complex traits. Behavior Genetics 42(1):1-2. [aRAZ]
135. Hofstee, W. K. B. (1984) Methodological decision rules as research policies: A betting reconstruction of empirical research. Acta Psychologica 56:93-109. [aRAZ]
136. Hovland, C. I. & Weiss. W. (1951) The influence of source credibility on communication effectiveness. Public Opinion Quarterly 15:635-50. [REP]
137. Hüffmeier, J., Mazei, J. & Schultze, T. (2016) Reconceptualizing replication as a sequence of different studies: A replication typology. Journal of Experimental Social Psychology 66:81-92. [aRAZ]
138. Hultsch, D. F. & Hickey, T. (1978) External validity in the study of human development: Theoretical and methodological issues. Human Development 21(2):76-91. Available at: https://doi.org/10.1159/000271576. [DMA]
139. IntHout, J., Ioannidis, J. P. & Borm, G. (2016) Obtaining evidence by a single wellpowered trial or by several modestly powered trials. Statistical Methods in Medical Research 25:538-52. [JPAI]
140. Ioannidis, J. P. (2005) Why most published research findings are false. PLoS Medicine 2(8):e124. [aRAZ, JPAI]
141. Ioannidis, J. P. (2008) Why most discovered true associations are inflated. Epidemiology 19:640-48. [JPAI]
142. Ioannidis, J. P. (2013a) Implausible results in human nutrition research. British Medical Journal 347:f6698. [JPAI]
143. Ioannidis, J. P. (2013b) Discovery can be a nuisance, replication is science, implementation matters. Frontiers in Genetics 4:33. [JPAI]
144. Ioannidis, J. P., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., Mangion, J., Mehta, T., Nitzburg, M., Page, G. P., Petretto, E. & van Noort, V. (2009) Repeatability of published microarray gene expression analyses. Nature Genetics 41(2):149-55. [MBN]
145. Ioannidis, J. P. & Trikalinos, T. A. (2007) An exploratory test for an excess of significant findings. Clinical Trials 4:245-53. [TEH]
146. James, W. (1907) Pragmatism's conception of truth. The Journal of Philosophy, Psychology and Scientific Methods 4(6):141-55. [AGa]
147. Jeffreys, H. (1961) Theory of probability. Oxford University Press. [aRAZ]
148. John, L. K., Loewenstein, G. & Prelec, D. (2012) Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science 23(5):524-32. [aRAZ, MBN]
149. Kahneman, D. (2003) Experiences of collaborative research. American Psychologist 58:723. [aRAZ]
150. Kahneman, D. (2014) A new etiquette for replication. Social Psychology 45(4):310-11. [BAS, DJS, WT]
151. Kanai, R., Bahrami, B., Roylance, R. & Rees, G. (2012) Online social network size is reflected in human brain structure. Proceedings of the Royal Society B Biological Sciences 279(1732):1327-34. Available at: https://doi.org/10.1098/rspb. 2011.1959. [KG]
152. Kane, M. T. (2001) Current concerns in validity theory. Journal of Educational Measurement 38:319-42. [SOL]
153. Kerr, N. L. (1998) HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review 2:196-217. Available at: http://doi.org/10.1207/s15327957pspr0203-4. [aRAZ]
154. Kevic, K., Murphy, B., Williams, L. & Beckmann, J. (2017) Characterizing experimentation in continuous deployment: A case study on bing. In: Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track, pp. 123-32. IEEE Press. [AGa]
155. Kidwell, M. C. Lazarevic, L. B. Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S. & Nosek, B. A. (2016) Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLoS Biology 14(5):e1002456. Available at: http://doi.org/10.1371/journal. pbio.1002456. [MBN]
156. Klein, J. R. & Roodman, A. (2005) Blind analysis in nuclear and particle physics. Annual Review of Nuclear and Particle Physics 55:141-63. [RJM]
157. Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Jr., Bahník, S., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., Hasselman, F., Hicks, J. A., Hovermale, J. F., Hunt, S. J., Hunstinger, J. R., Ijerzman, H., John, M.-S., Joy-Gaba, J. A., Kappes, H. B., Krueger, L. E., Kurtz, J., Levitan, C. A., Mallett, R. K., Morris, W. L., Nelson, A. J., Nier, J. A., Packard, G., Pilati, R., Rutchick, A. M., Schmidt, K., Skorinko, J. L., Smith, R., Steiner, T. G., Storbeck, J., Van Swol, L. M., Thompson, D., van't Veer, A. E., Vaughn, L. A., Vranka, M., Wichman, A. L., Woodzicka, J. A. & Nosek, B. A. (2014a) Investigating variation in replicability: A "Many Labs" replication project. Social Psychology 45(3):142-52. Available at: http://doi.org/10.1027/1864-9335/a000178. [aRAZ, PDLH, SS, JLT]
158. Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Jr., Bahník, S., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., Hasselman, F., Hicks, J. A., Hovermale, J. F., Hunt, S. J., Huntsinger, J. R., Ijerzman, H., John, M.-S., Joy-Gaba, J. A., Kappes, H. B., Krueger, L. E., Kurtz, J., Levitan, C. A., Mallett, R. K., Morris, W. L., Nelson, A. J., Nier, J. A., Packard, G., Pilati, R., Rutchick, A. M., Schmidt, K., Skorinko, J. L., Smith, R., Steiner, T. G., Storbeck, J., Van Swol, L. M., Thompson, D., van't Veer, A. E., Vaughn, L. A., Vranka, M., Wichman, A. L., Woodzicka, J. A. & Nosek, B. A. (2014b). Data from investigating variation in replicability: A "Many Labs" Replication Project. Journal of Open Psychology Data 2(1):e4. [RG-S]
159. Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Jr., Bahník, S., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., Hasselman, F., Hicks, J. A., Hovermale, J. F., Hunt, S. J., Huntsinger, J. R., Ijerzman, H., John, M.-S., Joy-Gaba, J. A., Kappes, H. B., Krueger, L. E., Kurtz, J., Levitan, C. A., Mallett, R. K., Morris, W. L., Nelson, A. J., Nier, J. A., Packard, G., Pilati, R., Rutchick, A. M., Schmidt, K., Skorinko, J. L., Smith, R., Steiner, T. G., Storbeck, J., Van Swol, L. M., Thompson, D., van't Veer, A. E., Vaughn, L. A., Vranka, M., Wichman, A. L., Woodzicka, J. A. & Nosek, B. A. (2014c). Theory building through replication: Response to commentaries on the "Many Labs" replication project. Social Psychology 45(4):299-311. [TEH]
160. Koehler, D. J. (2016) Can journalistic "false balance" distort public perception of consensus in expert opinion? Journal of Experimental Psychology: Applied 22 (1):24-38. Available at: http://doi.org/10.1037/xap0000073. [MB]
161. Krupnikov, Y. & Levine, A. S. (2014). Cross-sample comparisons and external validity. Journal of Experimental Political Science 1(1), 59-80. [GPa]
162. Kühberger, A., Fritz, A. & Scherndl, T. (2014) Publication bias in psychology: A diagnosis based on the correlation between effect size and sample size. PLoS One 9(9):e105825. [aRAZ]
163. Kunert, R. (2016) Internal conceptual replications do not increase independent replication success. Psychonomic Bulletin and Review 23(5):1631-38. [aRAZ]
164. Lakatos, I. (1970) Falsification and the methodology of scientific research programmes. In: Criticism and the growth of knowledge, ed. I. Lakatos & A. Musgrave, pp. 91-196. Cambridge University Press. [arRAZ]
165. Lakatos, I. (1978) The methodology of scientific research programs, vol. I. Cambridge University Press. [EHW]
166. Lakens, D. (2013) Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology 4:863. Available at: https://doi.org/10.3389/fpsyg.2013.00863. [DMA]
167. Lakens, D. (2016) The replication value: What should be replicated? Blog post. Available at: http://daniellakens.blogspot.co.uk/2016/01/the-replication-valuewhat-should-be.html. [RG-S]
168. Lakens, D. (2017) Five reasons blog posts are of higher scientific quality than journal articles. Blog post. Available at: http://daniellakens.blogspot.de/2017/04/fivereasons-blog-posts-are-of-higher.html. [BE]
169. Lawrence, P. A. (2003) The politics of publication. Nature 422:259-61. Available at: http://doi.org/10.1038/422259a. [ARK]
170. Leary, M. R., Diebels, K. J., Davisson, E. K., Jongman-Sereno, K. P., Isherwood, J. C., Raimi, K. T., Deffler, S. A. & Hoyle, R. H. (2017) Cognitive and interpersonal features of intellectual humility. Personality and Social Psychology Bulletin, 43 (6):793-813. Available at: https://doi.org/10.1177/0146167217697695. [GPe]
171. LeBel, E. P., Berger, D., Campbell, L. & Loving, T. J. (2017) Falsifiability is not optional. Journal of Personality and Social Psychology 113:254-61. [aRAZ]
172. LeBel, E. P. & Peters, K. R. (2011) Fearing the future of empirical psychology: Bem's (2011) evidence of psi as a case study of deficiencies in modal research practice. Review of General Psychology 15(4):371-79. [RG-S]
173. Lee, M. D. & Wagenmakers, E. J. (2005) Bayesian statistical inference in psychology: Comment on Trafimow (2003). Psychological Review 112:662-68. [DRL]
174. Leek, J., McShane, B. B., Gelman, A., Colquhoun, D., Nuitjen, M. B., and Goodman, S. N. (2017) Five ways to fix statistics. Nature 551(7682):557-59. [JLT]
175. Leighton, D. C., Legate, N., LePine, S., Anderson, S. F. & Grahe, J. (2018) Selfesteem, self-disclosure, self-expression, and connection on Facebook: A collaborative replication meta-analysis. Psi Chi Journal of Psychological Research 23:98-109. [HI]
176. Levelt Committee, Noort Committee & Drent Committee (2012, November 28) Flawed science: The fraudulent research practices of social psychologist Diederik Stapel. Retrieved 21 August 2017 from www.tilburguniversity.edu/upload/3ff904d7-547b-40ae85febea38e05a34a-Final%20report%20Flawed% 20Science.pdf. [aRAZ]
177. Lewin, K. (1943/1997). Psychological ecology. In G. W. Lewin & D. Cartwright (Ed.), Resolving social conflicts & field theory in social science (pp. 289-300). American Psychological Association. [AGa]
178. Lewis, M. L. & Frank, M. C. (2016) Understanding the effect of social context on learning: A replication of Xu and Tenenbaum (2007b) Journal of Experimental Psychology: General 145:e72-80. [TEH]
179. Lilienfeld, S. O. & Pinto, M. D. (2015) Risky tests of etiological models in psychopathology research: The need for meta-methodology. Psychological Inquiry 26:253-58. [SOL]
180. Lindsay, D. S., Simons, D. J. & Lilienfeld, S. O. (2016) Research preregistration 101. Association for Psychological Science Observer 29:14-17. [SOL]
181. Lipsey, M. W. & Wilson, D. B. (1993) The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist 48:1181-209. [RJM]
182. Little, D. R., Altieri, N., Fific, M. & Yang, C-T. (2017) Systems factorial technology: A theory driven methodology for the identification of perceptual and cognitive mechanisms. Academic. [DRL]
183. Lord, F. M. & Novick, M. R. (1968) Statistical theories of mental test scores. Addison-Wesley. [EHW]
184. Lucas, R. E. & Donnellan, M. B. (2013) Improving the replicability and reproducibility of research published in the Journal of Research in Personality. Journal of Research in Personality 4(47):453-54. [rRAZ]
185. Lupia, A. & Elman, C. (2014) Openness in political science: Data access and research transparency. PS - Political Science and Politics 47:19-42. [aRAZ, SS]
186. Luttrell, A., Petty, R. E. & Xu, M. (2017) Replicating and fixing failed replications: The case of need for cognition and argument quality. Journal of Experimental Social Psychology 69:178-83. [DTW]
187. Ly, A., Etz, A., Marsman, M. & Wagenmakers, E. J. (2017) Replication Bayes factors from evidence updating. PsyArXiv preprints. Available at: https://psyarxiv.com/u8m2s/. [aRAZ]
188. Ly, A., Verhagen, J. & Wagenmakers, E. J. (2016) Harold Jeffreys's default Bayes factor hypothesis tests: Explanation, extension, and application in psychology. Journal of Mathematical Psychology 72:19-32. [aRAZ]
189. Lykken, D. T. (1968) Statistical significance in psychological research. Psychological Bulletin 70:151-59. [aRAZ, SOL]
190. MacCoun, R. J. & Perlmutter, S. (2015) Hide results to seek the truth. Nature 526:187-89. [RJM]
191. MacCoun, R. J. & Perlmutter, S. (2017) Blind analysis as a correction for confirmatory bias in physics and in psychology In: Psychological science under scrutiny: Recent challenges and proposed solutions, ed. S. O. Lilienfeld & I. Waldman, pp. 297-322. Wiley. [RJM]
192. MacKay, D. J. (1992) Information-based objective functions for active data selection. Neural Computation 4(4):590-604. [TEH]
193. Maher, B. & Anfres, M. S. (2016) Young scientists under pressure: What the data show. Nature 538:444-45. Available at: http://doi.org/10.1038/538444a [ARK]
194. Makel, M. C., Plucker, J. A. & Hegarty, B. (2012) Replications in psychology research: How often do they occur? Perspectives on Psychological Science 7 (6):537-42. [aRAZ, PDLH]
195. Manicas, P. T. & Secord, P. F. (1983) Implication for psychology of the new philosophy of science. American Psychologist 38(4):399-413. Available at: https://doi.org/10.1037/0003-066X.38.4.399. [DMA]
196. Many junior scientists need to take a hard look at their job prospects. Editorial. (2017) Nature 550(7677):429. Available at: http://doi.org/10.1038/550429a. [ARK]
197. Martin, G. N. & Clarke, M. (2017) Are psychology journals anti-replication? A snapshot of editorial practices. Frontiers in Psychology 8(523):1-6. [PDLH]
198. Martinez, J. E., Funk, F. & Todorov, A. (2018). Quantifying idiosyncratic and shared contributions to stimulus evaluations. Available at: http://psyarxiv.com/6vr8z. [AGa]
199. Matzke, D., Nieuwenhuis, S., van Rijn, H., Slagter, H. A., van der Molen, M. W. & Wagenmakers, E.-J. (2015) The effect of horizontal eye movements on free recall: A preregistered adversarial collaboration. Journal of Experimental Psychology: General 144(1):e1-15. Available at: http://doi.org/10.1037/xge0000038. [aRAZ]
200. Maxwell, S. E., Lau, M. Y. & Howard, G. S. (2015) Is psychology suffering from a replication crisis? What does "failure to replicate" really mean? American Psychologist 70:487-98. Available at: http://dx.doi.org/10.1037/a0039400. [aRAZ]
201. McShane, B. B. & Bockenholt, U. (2017) Single paper meta-analysis: Benefits for study summary, theory-testing, and replicability. Journal of Consumer Research 43(6):1048-63. [JLT]
202. McShane, B. B. & Bockenholt, U. (2018) Multilevel multivariate meta-analysis with application to choice overload. Psychometrika 83(1):255-271. [JLT]
203. McShane, B. B., Bockenholt, U. & Hansen, K. T. (2016) Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science 11(5):730-49. [JLT]
204. McShane, B. B. & Gal, D. (2016) Blinding us to the obvious? The effect of statistical training on the evaluation of evidence. Management Science 62(6):1707-18. [JLT]
205. McShane, B. B. & Gal, D. (2017) Statistical significance and the dichotomization of evidence. Journal of theAmerican Statistical Association 112(519):885-95. [JLT]
206. McShane, B. B., Gal, D., Gelman, A., Robert, C. & Tackett, J. L. (2017) Abandon statistical significance. Technical report, Northwestern University. Available at: https://arxiv.org/abs/1709.07588. [AGe, JLT]
207. Meehl, P. E. (1967) Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science 34(2):103-15. [DMA, DRL]
208. Meehl, P. E. (1990a) Why summaries of research on psychological theories are often uninterpretable. Psychological Reports 66(1):195-244. Available at: https://doi. org/10.2466/pr0.1990.66.1.195. [DMA, TC, DRL]
209. Meehl, P. E. (1990b) Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry 1:108-41. Available at: http://doi.org/10.1207/s15327965pli0102-1. [aRAZ, TC, SS]
210. Mellers, B., Hertwig, R. & Kahneman, D. (2001) Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science 12:269-75. [aRAZ]
211. Merton, R. K. (1942) The normative structure of science. In: The sociology of science: Theoretical and empirical investigations, ed. R. K. Merton, pp. 267-80. University of Chicago Press. [SS]
212. Mihura, J. L., Meyer, G. J., Dumitrascu, N. & Bombel, G. (2013). The validity of individual Rorschach variables: Systematic reviews and meta-analyses of the comprehensive system. Psychological Bulletin 139, 548-605. [SOL]
213. Mill, J. S. (1882/2014) A system of logic, 8th edition. Harper and Brothers. (Original work published 1882.) Available at: https://ebooks.adelaide.edu.au/m/mill/john-stuart/system-of-logic/index.html [SS]
214. Morey, R. D. & Lakens, D. (2016) Why most of psychology is statistically unfalsifiable. Available at: https://github.com/richarddmorey/psychology-resolution/blob/master/paper/response.pdf [aRAZ]
215. Motyl, M., Demos, A. P., Carsel, T. S., Hanson, B. E., Melton, Z. J., Mueller, A. B., Prims, J. P., Sun, J., Washburn, A. N., Wong, K., Yantis, C. A. & Skitka, L. J. (2017) The state of social and personality science: Rotten to the core, not so bad, getting better, or getting worse? Journal of Personality and Social Psychology 113(1):34. [TC]
216. Mullen, B., Muellerleile, P. & Bryant, B. (2001) Cumulative meta-analysis: A consideration of indicators of sufficiency and stability. Personality and Social Psychology Bulletin 27(11):1450-62. [TEH]
217. Munafò, M. R., Nosek, B. A., Bishop, D. V., Button, K. S., Chambers, C., Nosek, B., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J. & Ioannidis, J. P. A. (2017) A manifesto for reproducible science. Nature Human Behaviour 1:0021. [JPAI]
218. National Cancer Institute-National Human Genome Research Institute (NCINHGRI) Working Group on Replication in Association Studies (2007) Replicating genotype-phenotype associations. Nature 447:655-60. [aRAZ]
219. Nelson, L. D., Simmons, J. P. & Simonsohn, U. (2018) Psychology's renaissance. Annual Review of Psychology 69:511-34. Available at: http://doi.org/10.1146/annurev-psych-122216-011836. [BE, BAS]
220. Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T. A., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Levy Paluck, E., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R. & Yarkoni, T. (2015) Promoting an open research culture. Science 348:1422-25. [BAS]
221. Nosek, B. A. & Bar-Anan, Y. (2012) Scientific utopia: I. Opening scientific communication. Psychological Inquiry 23:217-43. Available at: http://doi.org/10. 1080/1047840X.2012.692215. [BE]
222. Nosek, B. A. & Errington, T. M. (2017) Making sense of replications. eLife 6:e23383. [arRAZ]
223. Nosek, B. A., Spies, J. R. & Motyl, M. (2012) Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7(6):615-31. Available at: http://doi.org/10.1177/1745691612459058. [ARK, GPe]
224. Nuijten, M. B., Borghuis, J., Veldkamp, C. L. S., Dominguez-Alvarez, L., Van Assen, M. A. L. M. & Wicherts, J. M. (2017) Journal data sharing policies and statistical reporting inconsistencies in psychology. Collabra: Psychology 3(1):1-22. Available at: http://doi.org/10.1525/collabra.102. [MBN]
225. Nuijten, M. B., Hartgerink, C. H. J., Van Assen, M. A. L. M., Epskamp, S. & Wicherts, J. M. (2016) The prevalence of statistical reporting errors in psychology (1985-2013). Behavior Research Methods 48(4):1205-26. Available at: http://doi.org/10.3758/s13428-015-0664-2. [EH, MBN]
226. Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349(6251):aac4716. Available at: http://doi.org/10.1126/science.aac4716. [arRAZ, MB, BE, RG-S, EH, PDLH, AK, JLT, US, WT, EHW]
227. Paluck, E. L. & Shafir, E. (2017) The psychology of construal in the design of field experiments. Handbook of Economic Field Experiments 1:245-68. [AGa]
228. Pashler, H. & Harris, C. (2012) Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science 7:531-36. [aRAZ]
229. Pashler, H. & Wagenmakers, E.-J. (2012) Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science 7:528-30. [aRAZ]
230. Peer, E., Vosgerau, J. & Acquisti, A. (2014) Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods 46 (4):1023-31. [GPa]
231. Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J. & Fugelsang, J. A. (2015) On the reception and detection of pseudo-profound bullshit. Judgment and Decision Making 10(6):549-63. [GPe]
232. Petrocelli, J., Clarkson, J., Whitmire, M. & Moon, P. (2012) When ab≠c - c′: Published errors in the reports of single-mediator models. Behavior Research Methods 45(2):595-601. Available at: http://doi.org/10.3758/s13428-012-0262-5 [MBN]
233. Petty, R. E. (2015) The replication crisis: Social psychology versus other sciences. Paper presented at the annual meeting of the Society of Experimental Social Psychology, Denver, CO. [REP]
234. Petty, R. E. & Cacioppo, J. T. (2016) Methodological choices have predictable consequences in replicating studies on motivation to think: Commentary on Ebersole et al. (2016). Journal of Experimental Social Psychology 67:86-87. [DTW]
235. Phillips, J., Ong, D. C., Surtees, A. D. R., Xin, Y., Williams, S., Saxe, R. & Frank, M. C. (2015) A second look at automatic theory of mind. Psychological Science 26 (9):1353-67. [TEH]
236. Pitt, J. C. (1990) The myth of science education. Studies in Philosophy and Education 10:7-17. Available at: http://doi.org/10.1007/BF00367684 [MB]
237. Platt, J. R. (1964) Strong inference. Science 146(3642):347-53. [TEH]
238. Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J. B., Vul, E. & Yarkoni, T. (2017) Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience 18(2):115-26. Available at: http://doi. org/10.1038/nrn.2016.167 [ARK]
239. Popper, K. (1959) The logic of scientific discovery. Routledge. [SS]
240. Popper, K. R. (1959) Logic of scientific discovery. Basic. [WT]
241. Popper, K. R. (1959/2002) The logic of scientific discovery, translation of Logik der Forschung. Routledge. [aRAZ]
242. Rand, D. G., Peysakhovich, A., Kraft-Todd, G. T., Newman, G. E., Wurzbacher, O., Nowak, M. A. & Greene, J. D. (2014) Social heuristics shape intuitive cooperation. Nature Communications 5:3677. [GP]
243. Rewarding negative results keeps science on track. Editorial. (n.d.). Nature 551:414. Available at: http://www.nature.com/articles/d41586-017-07325-2. [KG]
244. Rieth, C. A., Piantadosi, S. T., Smith, K. A. & Vul, E. (2013) Put your money where your mouth is: Incentivizing the truth by making nonreplicability costly. European Journal of Personality 27:120-44. [AOH]
245. Robertson, C. T. & Kesselheim, A. S. (2016) Blinding as a solution to bias: Strengthening biomedical science, forensic science, and law. Academic. [RJM]
246. Rohrer, D., Pashler, H. & Harris, C. R. (2015) Do subtle reminders of money change people's political views? Journal of Experimental Psychology: General 144(4): e73. [aRAZ]
247. Rosen, G. M. (1993). Self-help or hype? Comments on psychology's failure to advance self-care. Professional Psychology: Research and Practice 24:340-45. [SOL]
248. Rosenthal, R. (1979) The "file drawer problem" and tolerance for null results. Psychological Bulletin 86(3):638-41. [aRAZ]
249. Rotello, C. M., Heit, E. & Dubé, C. (2015) When more data steer us wrong: Replications with the wrong dependent measure perpetuate erroneous conclusions. Psychonomic Bulletin and Review 22:944-54. [aRAZ, EH]
250. Rotello, C. M., Masson, M. E. J. & Verde, M. F. (2008) Type I error rates and power analyses for single-point sensitivity measures. Perception and Psychophysics 70:389-401. [EH]
251. Rothstein, H. R. & Bushman, B. J. (2012) Publication bias in psychological science: Comment on Ferguson and Brannick (2012). Psychological Methods 17:129-36. [aRAZ]
252. Royal Netherlands Academy of Arts and Sciences (2018) Replication studies. Improving reproducibility in the empirical sciences. KNAW. [MBN]
253. Salmon, W. C. (1984) Scientific explanation and the causal structure of the world. Princeton University Press. [EH]
254. Schimmack, U. (2012) The ironic effect of significant results on the credibility of multiple-study articles. Psychological Methods 17:551-56. [US]
255. Schimmack, U. (2014) The test of insufficient variance (TIVA): A new tool for the detection of questionable research practices. Working paper. Available at: https://replicationindex.wordpress.com/2014/12/30/the-test-of-insufficient-variance-tiva-a-new-tool-for-the-detection-of-questionable-research-practices/. [US]
256. Schimmack, U. (2017) 'Before you know it' by John A. Bargh: A quantitative book review. Available at: https://replicationindex.wordpress.com/2017/11/28/beforeyou-know-it-by-john-a-bargh-a-quantitative-book-review/. [US]
257. Schimmack, U. & Brunner, J. (submittrd for publication) Z-Curve: A method for estimating replicability based on test statistics in original studies. Submitted for Publication. [US]
258. Schimmack, U., Heene, M. & Kesavan, K. (2017) Reconstruction of a train wreck: How priming research went off the rails. Blog post. Available at: https://replicationindex. wordpress.com/2017/02/02/reconstruction-of-a-train-wreck-howpriming-research-went-of-the-rails/. [US]
259. Schmidt, F. L. & Oh, I.-S. (2016) The crisis of confidence in research findings in psychology: Is lack of replication the real problem? Or is it something else? Archives of Scientific Psychology 4(1):32-37. Available at: http://dx.doi.org/10. 1037/arc0000029. [aRAZ]
260. Schmidt, S. (2009) Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology 13:90-100. [aRAZ, AGe]
261. Schnall, S. (2014a) An experience with a registered replication project. Blog post. Available at: http://www.psychol.cam.ac.uk/cece/blog#anchor-2. [WT]
262. Schnall, S. (2014b) Further thoughts on replications, ceiling effects and bullying. Blog post. Available at: http://www.psychol.cam.ac.uk/cece/blog. [WT]
263. Schnall, S. (2014c) Social media and the crowd-sourcing of social psychology. Blog post. Available at: http://www.psychol.cam.ac.uk/cece/blog. [WT]
264. Schönbrodt, F. D. (2018). p-checker: One-for-all p-value analyzer. Available at: http://shinyapps.org/apps/p-checker/. [MBN]
265. Schooler, J. (2011) Unpublished results hide the decline effect. Nature 470:37. [WT]
266. Schooler, J. (2014) Metascience could rescue the 'replication crisis'. Nature 515:9. [WT]
267. Schweinsberg, M., Madan, N., Vianello, M., Sommer, S. A., Jordan, J., Tierney, W., Awtrey, E., Zhu, L. L., Diermeier, D., Heinze, J. E., Srinivasan, M., Tannenbaum, D., Bivolaru, E., Dana, J., Davis-Stober, C. P., du Plessis, C., Gronau, Q. F., Hafenbrack, A. C., Liao, E. Y., Ly, A., Marsman, M., Murase, T., Qureshi, I., Schaerer, M., Thornley, N., Tworek, C. M., Wagenmakers, E.-J., Wong, L., Anderson, T., Bauman, C. W., Bedwell, W. L., Brescoll, V., Canavan, A., Chandler, J. J., Cheries, E., Cheryan, S., Cheung, F., Cimpian, A., Clark, M. A., Cordon, D., Cushman, F., Ditto, P. H., Donahue, T., Frick, S. E., Gamez-Djokic, M., Hofstein Grady, R., Graham, J., Gu, J., Hahn, A., Hanson, B. E., Hartwich, N. J., Hein, K. Inbar, Y., Jiang, L., Kellogg, T., Kennedy, D. M., Legate, N., Luoma, T. P., Maibuecher, H., Meindl, P., Miles, J., Mislin, A., Molden, D. C., Motyl, M., Newman, G., Ngo, H. H., Packham, H., Ramsay, P. S., Ray, J. L., Sackett, A. M., Sellier, A.-L., Sokolova, T., Sowden, W., Storage, D., Sun, X., Van Bavel, J. J., Washburn, A. N., Wei, C., Wetter, E., Wilson, C. T., Darrous, S.-C. & Uhlmann, E. L. (2016) The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline. Journal of Experimental Social Psychology 66:55-67. [aRAZ, WT]
268. Shanks, D. R., Vadillo, M. A., Riedel, B., Clymo, A., Govind, S., Hickin, N., Tamman, A. J. & Puhlmann, L. (2015) Romance, risk, and replication: Can consumer choices and risk-taking be primed by mating motives? Journal of Experimental Psychology: General 144(6):e142-58. [aRAZ]
269. Silberzahn, R., Uhlmann, E. L., Martin, D., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., Fong, N., Gamez-Djokic, M., Glenz, A., Gordon-McKeon, S., Heaton, T. J., Hederos, K., Heene, M., Hofelich Mohr, A. J., Högden, F., Hui, K., Johannesson, M., Kalodimos, J., Kaszubowski, E., Kennedy, D. M., Lei, R., Lindsay, T. A., Liverani, S., Madan, C. R., Molden, D., Molleman, E., Morey, R. D., Mulder, L. B., Nijstad, B. R., Pope, N. G., Pope, B., Prenoveau, J. M., Rink, F., Robusto, E., Roderique, H., Sandberg, A., Schlüter, E., Schönbrodt, F. D., Sherman, M. F., Sommer, S. A., Sotak, K., Spain, S., Spörlein, C., Stafford, T., Stefanutti, L., Tauber, S., Ullrich, J., Vianello, M., Wagenmakers, E.-J., Witkowiak, M., Yoon, S. & Nosek, B. A. (2017) Many analysts, one dataset: Making transparent how variations in analytical choices affect results. PsyArXiv Preprint. Available at: https://psyarxiv.com/qkwst/. [aRAZ]
270. Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22:1359-66. Available at: http://doi.org/10. 1177/0956797611417632. [aRAZ, BE, MBN, BAS]
271. Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2018) False-positive citations. Perspectives on Psychological Science 13(2):255-59. [BE]
272. Simmons, J. & Simonsohn, U. (2015) Power posing: Reassessing the evidence behind themost popular TEDtalk. Blog post. Available at: http://datacolada.org/37. [AGe]
273. Simons, D. J., Holcombe, A. O. & Spellman, B. A. (2014) An introduction to Registered Replication Reports at Perspectives on Psychological Science. Perspectives on Psychological Science 9(5):552-55. [BAS, JLT]
274. Simons, D. J., Shoda, Y. & Lindsay, D. S. (2017) Constraints on generality (COG): A proposed addition to all empirical papers. Perspectives on Psychological Science 12:1123-28. Available at: http://doi.org/10.1177/1745691617708630. [aRAZ, AOH, HI, DJS]
275. Simonsohn, U. (2015) Small telescopes: Detectability and the evaluation of replication results. Psychological Science 26:559-69. [aRAZ, GPa]
276. Simonsohn, U. (2016, March 3) [47] Evaluating replications: 40% full?60% empty. Available at: https://web.archive.org/web/20170709184952/http://datacolada. org/47. [aRAZ]
277. Simonsohn, U., Nelson, L. D. & Simmons, J. P. (2014) P-curve: A key to the file-drawer. Journal of Experimental Psychology: General 143(2):534-47. [TEH]
278. Simpson, E.H. (1951) The Interpretation of interaction in contingency tables. Journal of the Royal Statistical Society: Series B (Methodological) 13(2):238-41. [JPdR]
279. Smaldino, P. E. & McElreath, R. (2016) The natural selection of bad science. Royal Society Open Science 3:160384. [aRAZ]
280. Smart, R. G. (1964) The importance of negative results in psychological research. Canadian Psychologist 5:225-32. [aRAZ]
281. Smith, G. T. McCarthy, D. M. & Anderson, K. G. (2000) On the sins of short-form development. Psychological Assessment 12:102-11. [SOL]
282. Smith, P. L. & Little, D. R. (2018) Small is beautiful: In defence of the small-N design. Psychonomic Bulletin & Review. Available at: https://doi.org/10.3758/s13423-018-1451-8. [DRL]
283. Spellman, B. A. (2015) A short (personal) future history of Revolution 2.0. Perspectives on Psychological Science 10:886-99. [aRAZ, BAS]
284. Sripada, C., Kessler, D. & Jonides, J. (2014) Methylphenidate blocks effort-induced depletion of regulatory control in healthy volunteers. Psychological Science 25:1227-34. [AK]
285. Srivastava, S. S. (2011, December 31) Groundbreaking or definitive? Journals need to pick one. Blog post. Available at: https://spsptalks.wordpress.com/2011/12/31/groundbreaking-or-definitive-journals-need-to-pick-one/. [SS]
286. Srivastava, S. (2012, September 17) A Pottery Barn rule for scientific journals. Blog post. Available at: https://hardsci.wordpress.com/2012/09/27/a-pottery-barnrule-for-scientific-journals/. [rRAZ]
287. Ståhl, T., Zaal, M. P. & Skitka, L. J. (2016) Moralized rationality: Relying on logic and evidence in the formation and evaluation of belief can be seen as a moral issue. PLoS One 11(11):e0166332. Available at: https://doi.org/10.1371/journal.pone. 0166332. [GPe]
288. Stanley, T. D., Carter, E. C. & Doucouliagos, H. (November 2017) What metaanalyses reveal about the replicability of psychological research. Deakin Laboratory for the Meta-Analysis of Research, Working Paper. [JLT]
289. Steegen, S., Tuerlinckx, F., Gelman, A. & Vanpaemel, W. (2016) Increasing transparency through a multiverse analysis. Perspectives on Psychological Science 11 (5):702-12. Available at: http://doi.org/10.1177/1745691616658637. [THE, MBN]
290. Sterling, T.D. (1959)Publicationdecisions andtheir possible effectsoninferences drawn from tests of significance-or vice versa. Journal of the American Statistical Association 54(285):30-34. Available at: http://doi.org/10.2307/2282137. [aRAZ, US]
291. Sterling, T. D., Rosenbaum, W. L. & Weinkam, J. J. (1995) Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. The American Statistician 49:108-12. [aRAZ]
292. Sternberg, S. (1969) The discovery of processing stages: Extensions of Donders' method. Acta Psychologica 30:276-315. [DRL]
293. Stewart, N., Chandler, J. & Paolacci, G. (2017) Crowdsourcing samples in cognitive science. Trends in Cognitive Sciences 21(10):736-48. [GPa]
294. Stewart, N., Ungemach, C., Harris, A. J., Bartels, D. M., Newell, B. R., Paolacci, G. & Chandler, J. (2015). The average laboratory samples a population of 7,300Amazon Mechanical Turk workers. Judgment and Decision Making 10(5):479-91. [GPa]
295. Stirman, S. W., Gamarra, J. M., Bartlett, B. A., Calloway, A. & Gutner, C. A. (2017) Empirical examinations of modifications and adaptations to evidence based psychotherapies: Methodologies, impact, and future directions. Clinical Psychology: Science and Practice 24:396-420. [SOL]
296. Stodden, V., McNutt, M., Bailey, D. H., Deelman, E., Gil, Y., Hanson, B., Heroux, M. A., Ioannidis, J. P. & Taufer, M. (2016) Enhancing reproducibility for computational methods. Science 354(6317):1240-41. [TEH]
297. Strack, F. (2017) From data to truth in psychological science. A personal perspective. Frontiers in Psychology 8:702. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5432643/. [FS]
298. Strevens, M. (2001) The Bayesian treatment of auxiliary hypotheses. The British Journal for the Philosophy of Science 52(3):515-37. [AOH, rRAZ]
299. Strevens, M. (2006)The bayesian approach to thephilosophy of science: In:Encyclopedia of philosophy, ed. D. M. Borchert, pp. 495-502. Macmillan Reference. [TEH]
300. Strevens, M. (2017) Notes on Bayesian confirmation theory [paper]. Available at: http://www.nyu.edu/classes/strevens/BCT/BCT.pdf. [AOH]
301. Stroebe, W. & Strack, F. (2014) The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science 9:59-71. Available at: http://doi.org/10.1177/1745691613514450. [aRAZ, FS, AMT, DTW]
302. Szucs, D. & Ioannidis, J. P. (2017a) Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLoS Biology 15(3):e2000797. Available at: http//doi.org/10.1371/journal.pbio.2000797. [ARK]
303. Szucs, D. & Ioannidis, J. P. A. (2017b) When null hypothesis significance testing is unsuitable for research: A reassessment. Frontiers in Human Neuroscience 11:390. Available at: https://doi.org/10.3389/fnhum.2017.00390. [DMA]
304. Tackett, J. L., Brandes, C. M. & Reardon, K. W. (in press) Leveraging the Open Science Framework in clinical psychological assessment research. Psychological Assessment [JLT]
305. Tackett, J. L., Lilienfeld, S. O., Patrick, C. J., Johnson, S. L, Krueger, R. F, Miller, J. D., Oltmans, T. F. & Shrout, P. E. (2017a) It's time to broaden the replicability conversation: Thoughts for and from clinical psychological science. Perspectives on Psychological Science 12(5):742-56. [JLT, SOL]
306. Tackett, J. L., McShane, B. B., Bockenholt, U. & Gelman, A. (2017b) Large scale replication projects in contemporary psychological research. Technical report, Northwestern University. Available at: arXiv:1710.06031. [JLT]
307. Thomson, K. S. & Oppenheimer, D. M. (2016) Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making 11(1):99-113. [GPa]
308. Tierney, W., Schweinsberg, M., Jordan, J., Kennedy, D. M., Qureshi, I., Sommer, A., Thornley, N., Madan, N., Vianello, M., Awtrey, E., Zhu, L. L., Diermeier, D., Heinze, J. E., Srinivasan, M., Tannenbaum, D., Bivolaru, E., Dana, J., Davis-Stober, C. P., du Plessis, C., Gronau, Q. F., Hafenbrack, A. C., Liao, E. Y, Ly, M. M., Murase, T., Schaerer, M., Tworek, C. M., Wagenmakers, EJ., Wong, L., Anderson, T., Bauman, C. W., Bedwell, W. L., Brescoll, V., Canavan, A., Chandler, J. J., Cheries, E., Cheryan, S., Cheung, F., Cimpian, A., Clark, M. A., Cordon, D., Cushman, F., Ditto, P. H., Amell, A., Frick, S. E., Gamez-Djokic, M., Hofstein Grady, R., Graham, J., Gu, J., Hahn, A., Hanson, B. E., Hartwich, N. J., Hein, K., Inbar, Y., Jiang, L., Kellogg, T., Legate, N., Luoma, T. P., Maibeucher, H., Meindl, P., Miles, J., Mislin, A., Molden, D. C., Motyl, M., Newman, G., Ngo, H. H., Packhan, H., Ramsay, P. S., Ray, J. L., Sackett, A. M., Sellier, A.-L., Sokolova, T., Sowden, W., Storage, D., Sun, X., Van Bavel, J. J., Washburn, A. N., Wei, C., Wetter, E., Wilson, C. T., Darroux, S.-C. & Uhlmann, E. L. (2016) Data from a prepublication independent replication initiative examining ten moral judgement effects. Nature Scientific Data 3:160082. Available at: http://doi.org/10.1038/sdata.2016.82. [WT]
309. Tracy, J. L. & Beall, A. T. (2014) The impact of weather on women's tendency to wear red or pink when at high risk for conception. PLoS One 9(2):e88852. [AGe]
310. Traxler, M. J. & Gernsbacher, M. A. (1992) Improving written communication through minimal feedback. Language and Cognitive Processes 7:1-22. Available at: https://doi.org/10.1080/01690969208409378. [MAG]
311. Traxler, M. J. & Gernsbacher, M. A. (1993) Improving written communication through perspective-taking. Language and Cognitive Processes 8:311-34. Available at: https://doi.org/10.1080/01690969308406958. [MAG]
312. van Aert, R. C. & van Assen, M. A. (2017) Bayesian evaluation of effect size after replicating an original study. PLoS One 12(4):e0175302. [aRAZ]
313. Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J. & Reinero, D. A. (2016) Contextual sensitivity in scientific reproducibility. Proceedings of the National Academy of Sciences of the United States of America 113(23):6454-59. [aRAZ]
314. van Erp, S., Verhagen, A. J., Grasman, R. P. P. P. & Wagenmakers, E.-J. (2017) Estimates of between-study heterogeneity for 705 meta-analyses reported in Psychological Bulletin from 1990-2013. Journal of Open Psychology Data 5 (1):4. DOI: http://doi.org/10.5334/jopd.33. [JLT]
315. Vanpaemel, W., Vermorgen, M., Deriemaecker, L. & Storms, G. (2015) Are we wasting a good crisis? The availability of psychological research data after the storm. Collabra 1(1):1-5. Available at: http://doi.org/10.1525/collabra.13. [MBN]
316. Veldkamp, C. L. S., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A. L. M. & Wicherts, J. M. (2014) Statistical reporting errors and collaboration on statistical analyses in psychological science. PLoS One 9(12):e114876. Available at: http://doi.org/10.1371/journal.pone.0114876. [MBN]
317. Verhagen, A. J. & Wagenmakers, E.-J. (2014) Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General 143:1457-75. [aRAZ]
318. Vohs, K. D. (2018) A pre-registered depletion replication project: The paradigmatic replication approach. Presented at the Symposium at the 2018 Society of Personality and Social Psychology Annual Convention, Atlanta, GA. [BAS]
319. Vul, E., Harris, C., Winkielman, P. & Pashler, H. (2009) Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science 4:274-90. Available at: http://doi.org/10.1111/j.1745-6924.2009.01125.x. [BE]
320. Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R. B., Jr., Albohn, D. N., Allard, E. S., Benning, S. D., Blouin-Hudon, E.-M., Bulnes, L. C., Caldwell, T. L., Calin-Jageman, R. J., Capaldi, C. A., Carfagno, N. S., Chasten, K. T., Cleeremans, A., Connell, L., DeCicco, J. M., Dijkstra, K, Fischer, A. H., Foroni, F., Hess, U., Holmes, K. J., Jones, J. L. H., Klein, O., Koch, C., Korb, S., Lewinski, P., Liao, J. D., Lund, S., Lupianez, J., Lynott, D., Nance, C. N., Oosterwijk, S., Ozdoǧru, A. A., Pacheco-Unguetti, A. P., Pearson, B., Powis, C., Riding, S., Roberts, T.-A., Rumiati, R. I., Senden, M., Shea-Shumsky, N. B., Sobocko, K., Soto, J. A., Steiner, T. G., Talarico, J. M., van Allen, Z. M., Vandekerckhove, M., Wainwright, B., Wayand, J. F., Zeelenberg, R., Zetzer, E. E. & Zwaan, R. A. (2016a) Registered replication report: Strack, Martin & Stepper (1988). Perspectives on Psychological Science 11:917-28. [arRAZ]
321. Wagenmakers, E.-J., Verhagen, A. J. & Ly, A. (2016b) How to quantify the evidence for the absence of a correlation. Behavior Research Methods 48:413-26. [aRAZ]
322. Wagge, J., Johnson, K., Meltzer, A., Baciu, C., Banas, K., Nadler, J. T., Ijzerman, H. & Grahe, J. E. (in preparation). Elliott et al.'s (2011) "Red, rank, and romance" effect: A meta-analysis of CREP replications. [HI]
323. Wald, A. (1947) Sequential analysis. Wiley. [EHW]
324. Wald, A. (1950) Statistical decision functions. Wiley. [NAC]
325. Wallot, S. & Kelty-Stephen, D. G. (2018) Interaction-dominant causation in mind and brain, and its implication for questions of generalization and replication. Minds and Machines 28(2):353-74. Available at: https://doi.org/10.1007/s11023-017-9455-0. [DMA]
326. Washburn, A. N., Hanson, B. E., Motyl, M., Skitka, L., Yantis, C., Wong, K., Sun, J., Prims, J., Mueller, A. B., Melton, Z. J. & Carsel, T. S. (2018) Why do some psychology researchers resist using proposed reforms to research practices? A description of researchers' rationales. Advances in Methods and Practices in Psychological Science. Published online March 7, 2018. Available at: https://doi. org/10.1177/2515245918757427. [TC]
327. Wicherts, J. M., Bakker, M. & Molenaar, D. (2011) Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS One 6(11):e26828. Available at: http://doi.org/10.1371/journal.pone.0026828. [MBN]
328. Wicherts, J. M., Borsboom, D., Kats, J. & Molenaar, D. (2006) The poor availability of psychological research data for reanalysis. American Psychologist 61:726-28. Available at: http://doi.org/10.1037/0003-066X.61.7.726. [MBN]
329. Widaman, K. (2015) Confirmatory theory testing: Moving beyond NHST. The score. Newsletter. Available at: http://www.apadivisions.org/division-5/publications/score/2015/01/issue.pdf. [DMA]
330. Witte, E. H. & Melville, P. (1982) Experimentelle Kleingruppenforschung: Methodologische Anmerkungen und eine empirische Studie. [Experimental small group research: Methodological remarks and an empirical study.] Zeitschrift für Sozialpsychologie 13:109-24. [EHW]
331. Witte, E. H. & Zenker, F. (2016a) Reconstructing recent work on macro-social stress as a research program. Basic and Applied Social Psychology 38(6):301-307. [EHW]
332. Witte, E. H. & Zenker, F. (2016b) Beyond schools - reply to Marsman, Ly & Wagenmakers. Basic and Applied Social Psychology 38(6):313-17. [EHW]
333. Witte, E. H. & Zenker, F. (2017a) Extending a multilab preregistered replication of the ego-depletion effect to a research program. Basic and Applied Social Psychology 39(1):74-80. [EHW]
334. Witte, E. H. & Zenker, F. (2017b) From discovery to justification. Outline of an ideal research program in empirical psychology. Frontiers in Psychology 8:1847. Available at: https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01847/full. [EHW]
335. Wood, J. M., Garb, H. N., Nezworski, M. T., Lilienfeld, S. O. & Duke, M. C. (2015) A second look at the validity of widely used Rorschach indices: Comment on Mihura, Meyer, Dumitrascu, and Bombel (2013). Psychological Bulletin 141:236-49. [SOL]
336. Wrinch, D. & Jeffreys, H. (1921) XLII. On certain fundamental principles of scientific inquiry. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 42(249):369-90. [aRAZ]
337. Yarkoni, T. & Westfall, J. (2017) Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science 1-23. [DRL]
338. Zwaan, R. A. (2017, May 8) Concurrent replication. Blog post. Available at: https://rolfzwaan.blogspot.nl/2017/05/concurrent-replication.html. [rRAZ, MAG]
339. Zwaan, R. A., Pecher, D., Paolacci, G., Bouwmeester, S., Verkoeijen, P., Dijkstra, K. & Zeelenberg, R. (2017) Participant nonnaiveté and the reproducibility of cognitive psychology. Psychonomic Bulletin and Review. Available at: http://doi. org/10.3758/s13423-017-1348-y. [aRAZ, GPa]