Replicating Studies in Which Samples of Participants Respond to Samples of Stimuli

Perspectives on Psychological Science

Volumn 10, Issue 3, 2015, Pages 390-399

  Westfall, Jacob ^a   Judd, Charles M ^a   Kenny, David A ^b  

^a UNIVERSITY OF COLORADO   (United States)

^b UNIVERSITY OF CONNECTICUT   (United States)

Author keywords

replication; statistical power; stimulus sampling

Indexed keywords

HUMAN; METHODOLOGY; PROCEDURES; PSYCHOLOGIC TEST; PSYCHOLOGY; SELECTION BIAS; STATISTICS; STATISTICS AND NUMERICAL DATA;

HUMANS; PSYCHOLOGICAL TESTS; PSYCHOLOGY; RESEARCH DESIGN; SELECTION BIAS; STATISTICS AS TOPIC;

EID: 84930336007     PISSN: 17456916     EISSN: 17456924     Source Type: Journal    
DOI: 10.1177/1745691614564879     Document Type: Article

References (41)

1. Asendorpf J. B.,Conner M.,De Fruyt F.,De Houwer J.,Denissen J. J. A.,Fiedler K.,.. Wicherts J. M.Recommendations for increasing replicability in psychology.European Journal of Personality. 2013;27:108-119
2. Baayen R. H.,Davidson D. J.,Bates D. M.Mixed-effects modeling with crossed random effects for subjects and items.Journal of Memory and Language. 2008;59:390-412
3. Baddeley A. D.,Thomson N.,Buchanan M.Word length and the structure of short-term memory.Journal of Verbal Learning and Verbal Behavior. 1975;14:575-589
4. Bakker M.,van Dijk A.,Wicherts J. M.The rules of the game called psychological science.Perspectives on Psychological Science. 2012;7:543-554
5. Brandt M. J.,IJzerman H.,Dijksterhuis A.,Farach F. J.,Geller J.,Giner-Sorolla R.,.. van’t Veer A.The replication recipe: What makes for a convincing replication?.Journal of Experimental Social Psychology. 2014;50:217-224
6. Button K. S.,Ioannidis J. P. A.,Mokrysz C.,Nosek B. A.,Flint J.,Robinson E. S. J.,Munafò M. R.Power failure: Why small sample size undermines the reliability of neuroscience.Nature Reviews Neuroscience. 2013;14:365-376
7. Cartwright D.Risk taking by individuals and groups: An assessment of research employing choice dilemmas.Journal of Personality and Social Psychology. 1971;20:361-378
8. Cartwright D.Determinants of scientific progress: The case of research on the risky shift.American Psychologist. 1973;28:222-231
9. Clark H. H.The language-as-fixed-effect fallacy: A critique of language statistics in psychological research.Journal of Verbal Learning and Verbal Behavior. 1973;12:335-359
10. Coleman B. E.Generalizing to a language population.Psychological Reports. 1964;14:219-226
11. Doyen S.,Klein O.,Pichon C.-L.,Cleeremans A.Behavioral priming: It’s all in the mind, but whose mind?.PLoS ONE. 2012;7 (1): e29081
12. Enserink M.Final report on Stapel also blames field as a whole.Science. 2012;338:1270-1271
13. Fiedler K.Voodoo correlations are everywhere—Not only in neuroscience.Perspectives on Psychological Science. 2011;6:163-171
14. Francis G.Publication bias and the failure of replication in experimental psychology.Psychonomic Bulletin & Review. 2012;19:975-991
15. Funder D. C.,Levine J. M.,Mackie D. M.,Morf C. C.,Sansone C.,Vazire S.,West S. G.Improving the dependability of research in personality and social psychology: Recommendations for research and educational practice.Personality and Social Psychology Review. 2014;18:3-12
16. Gorman A. M.Recognition memory for nouns as a function of abstractness and frequency.Journal of Experimental Psychology. 1961;61:23-29
17. IJzerman H.,Brandt M. J.,Van Wolferen J.Rejoice! In replication.European Journal of Personality. 2013;27:128-129
18. Ioannidis J. P. A.Why most discovered true associations are inflated.Epidemiology. 2008;19:640-648
19. Ioannidis J. P. A.Why science is not necessarily self-correcting.Perspectives on Psychological Science. 2012;7:645-654
20. John L. K.,Loewenstein G.,Prelec D.Measuring the prevalence of questionable research practices with incentives for truth telling.Psychological Science. 2012;23:524-532
21. Judd C. M.,Westfall J.,Kenny D. A.Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem.Journal of Personality and Social Psychology. 2012;103:54-69
22. Kenny D. A.Handbook of social psychology. (1985). Quantitative methods for social psychology. In LindzeyG.AronsonE. (Eds.), Handbook of social psychology (3rd ed., Vol. 1, pp. 487–508). New York, NY: Random House.Lindzey G.Aronson E., ed. New York, NY: Random House; 1985:487-508.
23. Kogan N.,Wallach M. A.Risk taking: A study in cognition and personality. Oxford, England: Holt, Rinehart & Winston; 1964:
24. Koole S. L.,Lakens D.Rewarding replications: A sure and simple way to improve psychological science.Perspectives on Psychological Science. 2012;7:608-614
25. Monin B.,Oppenheimer D. M.The limits of direct replications and the virtues of stimulus sampling.Social Psychology. 2014;45:299-300
26. Myers D. G.,Lamm H.The group polarization phenomenon.Psychological Bulletin. 1976;83:602-627
27. Registered reports: A method to increase the credibility of published results.Social Psychology. 2014;45:137-141
28. Nosek B. A.,Spies J. R.,Motyl M.Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability.Perspectives on Psychological Science. 2012;7:615-631
29. An open, large-scale, collaborative effort to estimate the reproducibility of psychological science.Perspectives on Psychological Science. 2012;7:657-660
30. Pashler H.,Harris C. R.Is the replicability crisis overblown? Three arguments examined.Perspectives on Psychological Science. 2012;7:531-536
31. Pruitt D. G.Choice shifts in group discussion: An introductory review.Journal of Personality and Social Psychology. 1971;20:339-360
32. Schimmack U.The ironic effect of significant results on the credibility of multiple-study articles.Psychological Methods. 2012;17:551-566
33. Schmidt S.Shall we really do it again? The powerful concept of replication is neglected in the social sciences.Review of General Psychology. 2009;13:90-100
34. Simmons J. P.,Nelson L. D.,Simonsohn U.False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant.Psychological Science. 2011;22:1359-1366
35. Simons D. J.The value of direct replication.Perspectives on Psychological Science. 2014;9:76-80
36. SimonsohnU. (2013). Small telescopes: Detectability and the evaluation of replication results (SSRN Scholarly Paper No. ID 2259879). Rochester, NY: Social Science Research Network. Retrieved from http://papers.ssrn.com/abstract=2259879
37. StonerJ. A. F. (1961). A comparison of individual and group decisions involving risk (Master’s thesis). Massachusetts Institute of Technology. Retrieved from http://dspace.mit.edu/handle/1721.1/11330
38. Stroebe W.,Strack F.The alleged crisis and the illusion of exact replication.Perspectives on Psychological Science. 2014;9:59-71
39. Verhagen A. J.,Wagenmakers E.Bayesian tests to quantify the result of a replication attempt.Journal of Experimental Psychology: General. 2014;143:1457-1475
40. Wells G. L.,Windschitl P. D.Stimulus sampling and social psychological experimentation.Personality and Social Psychology Bulletin. 1999;25:1115-1125
41. Westfall J.,Kenny D. A.,Judd C. M.Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli.Journal of Experimental Psychology: General. 2014;143:2020-2045