Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science

Journal of Personality and Social Psychology

Volumn 108, Issue 2, 2015, Pages 275-297

  Finkel, Eli J ^a   Eastwick, Paul W ^b   Reis, Harry T ^c  

^a NORTHWESTERN UNIVERSITY   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^c UNIVERSITY OF ROCHESTER   (United States)

Author keywords

Best research practices; Error balance; Evidentiary value movement; False positives reduction; Relationship science

Indexed keywords

EMPIRICISM; EVIDENCE BASED PRACTICE; HUMAN; KNOWLEDGE; METHODOLOGY; PSYCHOLOGY; RANDOMIZED CONTROLLED TRIAL; RESEARCH; SCIENCE; STATISTICAL ANALYSIS;

DATA INTERPRETATION, STATISTICAL; EMPIRICISM; EVIDENCE-BASED PRACTICE; HUMANS; KNOWLEDGE; PRAGMATIC CLINICAL TRIALS AS TOPIC; PSYCHOLOGY; RESEARCH; RESEARCH DESIGN; SCIENCE;

EID: 84922350571     PISSN: 00223514     EISSN: None     Source Type: Journal    
DOI: 10.1037/pspi0000007     Document Type: Article

References (120)

1. Alogna, V. K., Attaya, M. K., Aucoin, P., Bahnik, S., Birch, S., Birt, A. R., . Zwaan, R. A. (2014). Registered replication report: Schooler & Engstler-Schooler (1990). Perspectives on Psychological Science, 9, 556-578. http://dx.doi.org/10.1177/1745691614545653
2. American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.
3. American Psychological Association. (2014). Journal of Personality and Social Psychology: Instructions to authors. Retrieved from http://www .apa.org/pubs/journals/psp/
4. Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J., Fiedler, K.,. Wicherts, J. M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27, 108- 119. http://dx.doi.org/10.1002/per.1919
5. Association for Psychological Science. (2014). Registered replication reports. Retrieved from http://www.psychologicalscience.org/index.php/ replication
6. Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543-554. http://dx.doi.org/10.1177/1745691612459060
7. Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods, 43, 666- 678. http://dx.doi.org/10.3758/s13428-011-0089-5
8. Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407-425. http://dx.doi.org/10.1037/ a0021524
9. Berscheid, E. (1999). The greening of relationship science. American Psychologist, 54, 260-266.
10. Bissell, M. (2013). Reproducibility: The risks of the replication drive. Nature, 503, 333-334. http://dx.doi.org/10.1038/503333a
11. Bolger, N., & Laurenceau, J. P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research. New York, NY: Guilford Press.
12. Bolger, N., Stadler, G., & Laurenceau, J. P. (2012). Power analysis for intensive longitudinal studies. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 285-301). New York, NY: Guilford Press.
13. Brandt, M. J., Crawford, M. T., & Giner-Sorolla, R. (2014). Call for submissions: Special issue on pre-registered research. Retrieved from http://www.journals .elsevier.com/journal-of-experimental-social-psychology/call-for-papers/callfor- submissions-special-issue-on-pre-registered-researc/
14. Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R. Van't Veer, A. (2014). The replication recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50, 217-224. http://dx.doi.org/10.1016/j.jesp.2013 .10.005
15. Brown, S. D., Furrow, D., Hill, D. F., Gable, L. P., Porter, L. P., & Jacobs, J. (in press). A duty to describe: Better the devil you know that the devil you don't. Perspectives on Psychological Science.
16. Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376. http://dx.doi.org/10.1038/nrn3475
17. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. http://dx.doi.org/10.1037/h0046016
18. Casey, K., Glennerster, R., & Miguel, E. (2012). Reshaping institutions: Evidence on aid impacts using a preanalysis plan. The Quarterly Journal of Economics, 127, 1755-1812. http://dx.doi.org/10.1093/qje/qje027
19. Chamberlin, J. (2000). A student publishing tradition: The student-run, student-reviewed Representative Research in Social Psychology celebrates 30 years. APA Monitor, 31, 36.
20. Christakis, D. A., & Zimmerman, F. J. (2013). Rethinking reanalysis. Journal of the American Medical Association, 310, 2499-2500. http:// dx.doi.org/10.1001/jama.2013.281337
21. Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. The Journal of Abnormal and Social Psychology, 65, 145-153. http://dx.doi.org/10.1037/h0045186
22. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York, NY: Erlbaum.
23. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. http://dx.doi.org/10.1037/0033-2909.112.1.155
24. Collins, F. S., & Tabak, L. A. (2014). Policy: NIH plans to enhance reproducibility. Nature, 505, 612- 613. http://dx.doi.org/10.1038/ 505612a
25. Cook, T. D. (1990). The generalization of causal connections: Multiple theories in search of clear practice. In L. Sechrest, J. Bunker, & E. Perrin (Eds.), Research methodology: Strengthening causal interpretation of non-experimental data (pp. 9-31). Rockville, MD: Agency for Health Care Policy & Research.
26. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings (pp. 19-21). Boston: Houghton Mifflin.
27. Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist, 12, 671- 684. http://dx.doi.org/10.1037/ h0043943
28. Dafoe, A. (2014). Science deserves better: The imperative to share complete replication files. PS: Political Science & Politics, 47, 60-66.
29. de Groot, A. D. (2014). The meaning of "significance" for different types of research [Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han LJ van der Maas, Trans.]. Acta Psychologica, 148, 188-194. http://dx.doi.org/10.1016/j.actpsy.2014.02.001
30. DeWall, C. N., Lambert, N. M., Slotter, E. B., Pond, R. S., Deckman, T., Finkel, E. J.,. Fincham, F. D. (2011). So far away from one's partner, yet so close to romantic alternatives: Avoidant attachment, interest in alternatives, and infidelity. Journal of Personality and Social Psychology, 101, 1302-1316. http://dx.doi.org/10.1037/a0025497
31. Eastwick, P. W., Luchies, L. B., Finkel, E. J., & Hunt, L. L. (2014). The predictive validity of ideal partner preferences: A review and metaanalysis. Psychological Bulletin, 140, 623- 665. http://dx.doi.org/ 10.1037/a0032432
32. Eastwick, P. W., Neff, L. A., Finkel, E. J., Luchies, L. B., & Hunt, L. L. (2014). Is a meta-analysis a foundation, or just another brick? Comment on Meltzer, McNulty, Jackson, and Karney (2014). Journal of Personality and Social Psychology, 106, 429-434. http://dx.doi.org/10.1037/ a0034767
33. Eich, E. (2014). Business not as usual. Psychological Science, 25, 3-6. http://dx.doi.org/10.1177/0956797613512465
34. Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics, 90, 891-904. http://dx.doi.org/10.1007/ s11192-011-0494-7
35. Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychological science's aversion to the null. Perspectives on Psychological Science, 7, 555-561. http://dx.doi.org/ 10.1177/1745691612459059
36. Fiedler, K., Kutzner, F., & Krueger, J. I. (2012). The long way from α-error control to validity proper problems with a short-sighted false-positive debate. Perspectives on Psychological Science, 7, 661-669. http://dx.doi .org/10.1177/1745691612462587
37. Fine, M. A., & Kurdek, L. A. (1994). Publishing multiple journal articles from a single data set: Issues and recommendations. Journal of Family Psychology, 8, 371-379. http://dx.doi.org/10.1037/0893-3200.8.4.371
38. Finkel, E. J., DeWall, C. N., Slotter, E. B., McNulty, J. K., Pond, R. S., Jr., & Atkins, D. C. (2012). Using I3 theory to clarify when dispositional aggressiveness predicts intimate partner violence perpetration. Journal of Personality and Social Psychology, 102, 533-549. http://dx.doi.org/ 10.1037/a0025651
39. Fiske, S. T. (2014, October). Change is coming: A new day for human subjects research participation. APS Observer, 8, 5-21.
40. Fletcher, G. J., Simpson, J. A., & Thomas, G. (2000). The measurement of perceived relationship quality components: A confirmatory factor analytic approach. Personality and Social Psychology Bulletin, 26, 340- 354. http://dx.doi.org/10.1177/0146167200265007
41. Fraley, R. C., & Vazire, S. (2014). The N-pact factor: Evaluating the quality of empirical journals with respect to sample size and statistical power. PLoS ONE, 9, e109019. http://dx.doi.org/10.1371/journal.pone .0109019
42. Francis, G. (2012). Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review, 19, 151-156. http://dx.doi.org/10.3758/s13423-012-0227-9
43. Funder, D. C., Levine, J. M., Mackie, D. M., Morf, C. C., Sansone, C., Vazire, S., & West, S. G. (2014). Improving the dependability of research in personality and social psychology: Recommendations for research and educational practice. Personality and Social Psychology Review, 18, 3-12. http://dx.doi.org/10.1177/1088868313507536
44. Galak, J., Leboeuf, R. A., Nelson, L. D., & Simmons, J. P. (2012). Correcting the past: Failures to replicate Ψ. Journal of Personality and Social Psychology, 103, 933-948. http://dx.doi.org/10.1037/a0029709
45. Goodman, S., & Greenland, S. (2007). Why most published research findings are false: Problems in the analysis. PLoS Medicine, 4, e168. http://dx.doi.org/10.1371/journal.pmed.0040168
46. Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20. http://dx.doi.org/10.1037/ h0076157
47. Greenwald, A. G. (1976). An editorial. Journal of Personality and Social Psychology, 33, 1-7.
48. Higgins, E. T. (1992). Increasingly complex but less interesting articles: Scientific progress or regulatory problem? Personality and Social Psychology Bulletin, 18, 489-492. http://dx.doi.org/10.1177/0146 167292184014
49. Higgins, E. T. (1997). Beyond pleasure and pain. American Psychologist, 52, 1280-1300. http://dx.doi.org/10.1037/0003-066X.52.12.1280
50. Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York, NY: Psychology Press.
51. Humphreys, M., de la Sierra, R. S., & Van der Windt, P. (2013). Fishing, commitment, and communication: A proposal for comprehensive nonbinding research registration. Political Analysis, 21, 1-20. http://dx.doi .org/10.1093/pan/mps021
52. Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. http://dx.doi.org/10.1371/journal.pmed .0020124
53. Ioannidis, J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology, 19, 640-648. http://dx.doi.org/10.1097/EDE .0b013e31818131e7
54. Jager, L. R., & Leek, J. T. (2014). An estimate of the science-wise false discovery rate and application to the top medical literature. Biostatistics, 15, 1-12. http://dx.doi.org/10.1093/biostatistics/kxt007
55. John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524 -532. http://dx.doi.org/10.1177/ 0956797611430953
56. Kahneman, D. (2012, September 26). A proposal to deal with questions about priming effects [Open letter to social priming researchers].
57. Kahneman, D. (in press). A new etiquette for replication. Social Psychology.
58. Kashy, D. A., Donnellan, M. B., Ackerman, R. A., & Russell, D. W. (2009). Reporting and interpreting research in PSPB: Practices, principles, and pragmatics. Personality and Social Psychology Bulletin, 35, 1131-1142. http://dx.doi.org/10.1177/0146167208331253
59. Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis. New York, NY: Guilford Press.
60. Keppel, G., & Wickens, T. D. (2004). Design and analysis: A researcher's handbook (4th edition). Englewood Cliffs, NJ: Prentice-Hall.
61. Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196-217. http://dx.doi .org/10.1207/s15327957pspr0203_4
62. King, L. A. (2012). A dinosaur comments on the coming apocalypse: Does anybody else see that asteroid? Psychological Inquiry, 23, 274-276. http://dx.doi.org/10.1080/1047840X.2012.704804
63. Kühberger, A., Fritz, A., & Scherndl, T. (2014). Publication bias in psychology: A diagnosis based on the correlation between effect size and sample size. PLoS ONE, 9, e105825.
64. LeBel, E. P., Borsboom, D., Giner-Sorolla, R., Hasselman, F., Peters, K. R., Ratliff, K. A., & Smith, C. T. (2013). PsychDisclosure.org: Grassroots support for reforming reporting standards in psychology. Perspectives on Psychological Science, 8, 424-432. http://dx.doi.org/ 10.1177/1745691613491437
65. Ledgerwood, A. (2014). Introduction to the special section on moving toward a cumulative science: maximizing what our research can tell us. Perspectives on Psychological Science, 9, 610-611. http://dx.doi.org/ 10.1177/1745691614553989
66. Lehrer, J. (2010). The truth wears off: Is there something wrong with the scientific method. The New Yorker, December, 52-57.
67. Lykken, D. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159. http://dx.doi.org/10.1037/ h0026141
68. Maner, J. K. (2014). Let's put our money where our mouth is: If authors are to change their ways, reviewers (and editors) must change with them. Perspectives on Psychological Science, 9, 343-351. http://dx.doi.org/ 10.1177/1745691614528215
69. Marszalek, J. M., Barber, C., Kohlhart, J., & Holmes, C. B. (2011). Sample size in psychological research over the past 30 years. Perceptual and Motor Skills, 112, 331-348. http://dx.doi.org/10.2466/03.11.PMS.112.2 .331-348
70. Medawar, P. B. (1969). Induction and intuition in scientific thought. Philadelphia, PA: American Philosophical Society.
71. Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., . Van der Laan, M. (2014). Social science. Promoting transparency in social science research. Science, 343, 30-31. http://dx.doi.org/10.1126/ science.1245317
72. Mikulincer, M., Shaver, P. R., Gillath, O., & Nitzberg, R. A. (2005). Attachment, caregiving, and altruism: Boosting attachment security increases compassion and helping. Journal of Personality and Social Psychology, 89, 817- 839. http://dx.doi.org/10.1037/0022-3514.89.5 .817
73. Molden, D. C., Lee, A. Y., & Higgins, E. T. (2008). Motivations for promotion and prevention. In J. Shah & W. Gardner (Eds.), Handbook of motivation science (pp. 169-187). New York, NY: Guilford Press.
74. Neff, L. A., & Geers, A. L. (2013). Optimistic expectations in early marriage: A resource or vulnerability for adaptive relationship functioning? Journal of Personality and Social Psychology, 105, 38-60. http:// dx.doi.org/10.1037/a0032600
75. Nosek, B. A., & Bar-Anan, Y. (2012). Scientific utopia: I. Opening scientific communication. Psychological Inquiry, 23, 217-243. http://dx .doi.org/10.1080/1047840X.2012.692215
76. Nosek, B. A., & Lakens, D. (2014). Registered reports. Social Psychology, 45, 137-141.
77. Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7, 615-631.
78. Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7, 657-660. http://dx.doi.org/10.1177/ 1745691612462588
79. Open Science Framework. (2013, September 3). Badges to acknowledge open practices. Retrieved from https://osf.io/tvyxz/wiki/view/
80. Open Science Framework. (2014a, May 21). Open science framework. Retrieved from https://osf.io/4znzp/wiki/home
81. Open Science Framework. (2014b, May 31). Registered reports: A method to increase the credibility of published reports. Retrieved from https:// osf.io/zv2cs/
82. Pashler, H., & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7, 531-536. http://dx.doi.org/10.1177/1745691612463401
83. Popper, K. R. (1959/2002). The logic of scientific discovery. New York, NY: Routledge.
84. Reis, H. T. (2012). A brief history of relationship research in social psychology. In A. W. Kruglanski & W. Stroebe (Eds.), Handbook of the history of social psychology (pp. 363-382). New York, NY: Psychology Press.
85. Reis, H. T., Collins, W. A., & Berscheid, E. (2000). The relationship context of human behavior and development. Psychological Bulletin, 126, 844-872. http://dx.doi.org/10.1037/0033-2909.126.6.844
86. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638-641. http://dx.doi.org/10.1037/ 0033-2909.86.3.638
87. Rosenthal, R. (1990). How are we doing in soft psychology? American Psychologist, 45, 775-777. http://dx.doi.org/10.1037/0003-066X.45.6 .775
88. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284. http://dx.doi.org/10.1037/0003-066X.44.10.1276
89. Sbarra, D. A. (2014). Forward thinking: An introduction. Perspectives on Psychological Science, 9, 443- 444. http://dx.doi.org/10.1177/ 1745691614539905
90. Scargle, J. D. (2000). Publication bias: The "file-drawer" problem in scientific inference. Journal of Scientific Exploration, 14, 91-106.
91. Scherbaum, C. A., & Ferreter, J. M. (2009). Estimating statistical power and required sample sizes for organizational research using multilevel modeling. Organizational Research Methods, 12, 347-367. http://dx.doi .org/10.1177/1094428107308906
92. Schimmack, U. (2012). The ironic effect of significant results on the credibility of multiple-study articles. Psychological Methods, 17, 551- 566. http://dx.doi.org/10.1037/a0029487
93. Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13, 90-100. http://dx.doi.org/10.1037/a0015108
94. Schonbrodt, F. D., & Perugini, M. (2013). At what sample size do correlations stabilize? Journal of Research in Personality, 47, 609-612.
95. Schooler, J. (2011). Unpublished results hide the decline effect. Nature, 470, 437.
96. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316. http://dx.doi.org/10.1037/0033-2909.105.2.309
97. Silva, L. (2014, February 24). PLOS' new data policy: Public access to data. Retrieved from http://blogs.plos.org/everyone/2014/02/24/plosnew- data-policy-public-access-data-2/
98. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. http://dx.doi.org/10.1177/0956797611417632
99. Simons, D. J. (2014). The value of direct replication. Perspectives on Psychological Science, 9, 76-80. http://dx.doi.org/10.1177/1745691613514755
100. Simons, D. J., Holcombe, A. O., & Spellman, B. A. (2014). An introduction to registered replication reports at Perspectives on Psychological Science. Perspectives on Psychological Science, 9, 552-555. http://dx .doi.org/10.1177/1745691614543974
101. Simonsohn, U. (2013). Just post it: The lesson from two cases of fabricated data detected by statistics alone. Psychological Science, 24, 1875-1888. http://dx.doi.org/10.1177/0956797613480366
102. Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). p-curves: A key to the file-drawer. Journal of Experimental Psychology: General, 143, 534-547. http://dx.doi.org/10.1037/a0033242
103. Simpson, J. A., Collins, W. A., Tran, S., & Haydon, K. C. (2007). Attachment and the experience and expression of emotions in romantic relationships: A developmental perspective. Journal of Personality and Social Psychology, 92, 355-367. http://dx.doi.org/10.1037/0022-3514 .92.2.355
104. Snijders, T. A., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London, UK: Sage.
105. Srivastava, S. (2014, February). How can we make psychology less sciencey and more scientific? Paper presented at the Society for Personality and Social Psychology Annual Meeting, Austin, TX.
106. Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance-Or vice versa. Journal of the American Statistical Association, 54, 30-34.
107. Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. The American Statistician, 49, 108-112.
108. Stroebe, W., & Strack, F. (2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9, 59-71. http://dx.doi.org/10.1177/1745691613514450
109. Tanner, W. P., Jr., & Swets, J. A. (1954). A decision-making theory of visual detection. Psychological Review, 61, 401-409. http://dx.doi.org/ 10.1037/h0058700
110. Terman, L. M. (1938). Psychological factors in marital happiness. New York, NY: McGraw-Hill.
111. Tidwell, N. D., Eastwick, P. W., & Finkel, E. J. (2013). Perceived, not actual, similarity predicts initial attraction in a live romantic context: Evidence from the speed-dating paradigm. Personal Relationships, 20, 199-215. http://dx.doi.org/10.1111/j.1475-6811.2012.01405.x
112. Tsang, E. W., & Kwan, K. M. (1999). Replication and theory development in organizational science: A critical realist perspective. The Academy of Management Review, 24, 759-780.
113. Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632-638. http://dx.doi.org/ 10.1177/1745691612463078
114. Wegner, D. M. (1992). The premature demise of the solo experiment. Personality and Social Psychology Bulletin, 18, 504-508. http://dx.doi .org/10.1177/0146167292184017
115. Westfall, J., Judd, C. M., & Kenny, D. A. (in press). Replicating studies in which samples of participants respond to samples of stimuli. Current Opinion in Psychological Science.
116. Wicherts, J. M., & Bakker, M. (2012). Publish (your data) or (let the data) perish! Why not publish your data too? Intelligence, 40, 73-76. http:// dx.doi.org/10.1016/j.intell.2012.01.004
117. Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS ONE, 6, e26828. http://dx.doi.org/ 10.1371/journal.pone.0026828
118. Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726-728. http://dx.doi.org/10.1037/0003-066X.61.7 .726
119. Zhong, C.-B., & Liljenquist, K. (2006). Washing away your sins: Threatened morality and physical cleansing. Science, 313, 1451-1452. http:// dx.doi.org/10.1126/science.1130726
120. Zimmer, M. (2010). "But the data is already public": On the ethics of research in Facebook. Ethics and Information Technology, 12, 313-325. http://dx.doi.org/10.1007/s10676-010-9227-5