"Everybody knows psychology is not a real science": Public perceptions of psychology and how we can improve our relationship with policymakers, the scientific community, and the general public

American Psychologist

Volumn 70, Issue 6, 2015, Pages 527-542

  Ferguson, Christopher J ^a  

^a STETSON UNIVERSITY   (United States)

Author keywords

Bias; Psychological science; Public perceptions; Replication

Indexed keywords

HUMAN; PERCEPTION; PSYCHOLOGY; PUBLIC OPINION; PUBLIC POLICY; SCIENCE;

HUMANS; PERCEPTION; PSYCHOLOGY; PUBLIC OPINION; PUBLIC POLICY; SCIENCE;

EID: 84941084273     PISSN: 0003066X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0039405     Document Type: Article

References (120)

1. Adachi, P. C., & Willoughby, T. (2013). Do video games promote positive youth development? Journal of Adolescent Research, 28, 155-165. http://dx.doi.org/10.1177/0743558412464522
2. American Psychological Association. (1994). Violence in mass media. Retrieved from http://www.apa.org/about/policy/media.aspx
3. American Psychological Association. (2005). Resolution on violence in video games and interactive media. Retrieved from http://www.apa.org/about/governance/council/policy/interactive-media.pdf
4. American Psychological Association. (2015). Abortion resolutions. Retrieved from http://www.apa.org/about/policy/abortion.aspx
5. APA Task Force on Mental Health and Abortion. (2008). Report of the APA Task Force on Mental Health and Abortion. Washington, DC: Author.
6. Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543-554. http://dx.doi.org/10.1177/1745691612459060
7. Bandura, A. (1965). Influence of models' reinforcement contingencies on the acquisition of imitative response. Journal of Personality and Social Psychology, 1, 589-595. http://dx.doi.org/10.1037/h0022070
8. Bargh, J., & Chartrand, T. (1999). The unbearable automaticity of being. American Psychologist, 54, 462-479. http://dx.doi.org/10.1037/0003-066X.54.7.462
9. Bartlett, T. (2014, June 23). Replication crisis in psychology research turns ugly and odd. Chronicle of Higher Education. Retrieved from http:// chronicle.com/article/Replication-Crisis-in/147301
10. Bell, L. (2013, February 12). Why the Centers for Disease Control should not receive gun research funding. Forbes. Retrieved from http://www.forbes .com/sites/larrybell/2013/02/12/why-the-centers-for-disease-controlshould-not-receive-gun-research-funding/
11. Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407-425. http://dx.doi.org/10.1037/ a0021524
12. Bem, D., & Honorton, C. (1994). Does psi exist? Replicable evidence for an anomalous process of information transfer. Psychological Bulletin, 115, 4-18. http://dx.doi.org/10.1037/0033-2909.115.1.4
13. Berezow, A. (2012, July 13). Why psychology isn't science. The L.A. Times. Retrieved from http://articles.latimes.com/2012/jul/13/news/laol-blowback-pscyhology-science-20120713
14. Block, J. J., & Crain, B. R. (2007). Omissions and errors in "media violence and the American public." American Psychologist, 62, 252-253. http://dx.doi.org/10.1037/0003-066X.62.3.252
15. Brigham, F. J., Gustashaw, W. E., III, & Brigham, M. S. (2004). Scientific practice and the tradition of advocacy in special education. Journal of Learning Disabilities, 37, 200-206. http://dx.doi.org/10.1177/ 00222194040370030301
16. Brown, N., MacDonald, D., Samanta, M., Friedman, H., & Coyne, J. (in press). A critical reanalysis of Fredrickson et al.'s study of genomics and well-being. Proceedings of the National Academy of Sciences of the United States of America.
17. Chambers, C. (2014, June 10). Physics envy: Do "hard" sciences hold the solution to the replication crisis in psychology? The Guardian. Retrieved from http://www.theguardian.com/science/head-quarters/2014/ jun/10/physics-envy-do-hard-sciences-hold-the-solution-to-the-replicationcrisis-in-psychology
18. Childstats.gov. (2014). America's children: Key national indicators of well-being, 2010. Retrieved from http://www.childstats.gov/
19. Coyne, J. C., & de Voogd, J. N. (2012). Are we witnessing the decline effect in the Type D personality literature? What can be learned? Journal of Psychosomatic Research, 73, 401-407. http://dx.doi.org/10.1016/j.jpsychores.2012.09.016
20. Craig, J. R., & Reese, S. C. (1973). Retention of raw data: A problem revisited. American Psychologist, 28, 723. http://dx.doi.org/10.1037/ h0035667
21. Crotty, D. (2014, September 10). When crises collide: The tension between null results and reproducibility. The Scholarly Kitchen. Retrieved from http://scholarlykitchen.sspnet.org/2014/09/10/whencrises-collide-the-tension-between-null-results-and-reproducibility/
22. Dienes, Z. (2014, July 29). Using Bayes to get the most out of nonsignificant results. Frontiers in Psychology. Retrieved from http:// journal.frontiersin.org/article/10.3389/fpsyg.2014.00781/full. http:// dx.doi.org/10.3389/fpsyg.2014.00781
23. Dillio, R. (2014). A critical miss: Video games, violence, and ineffective legislation. First Amendment Studies, 48, 110-130. http://dx.doi.org/ 10.1080/21689725.2014.950496
24. Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012). Behavioral priming: It's all in the mind, but whose mind? PLoS ONE, 7, e29081. http://dx.doi.org/10.1371/journal.pone.0029081
25. Eaton, W. O. (1984). On obtaining unpublished data for research integrations. American Psychologist, 39, 1325-1326. http://dx.doi.org/10.1037/ 0003-066X.39.11.1325
26. Edwards, K., & Smith, E. E. (1996). A disconfirmation bias in the evaluation of arguments. Journal of Personality and Social Psychology, 71, 5-24. http://dx.doi.org/10.1037/0022-3514.71.1.5
27. Elson, M. (2014). How reliable is psychological science? Retrieved from http://blog.thingswedontknow.com/2014/07/how-reliable-is-psychologicalscience.html
28. Ertelt, S. (2006). American Psychological Association criticized for proabortion position. Retrieved from http://archive.lifenews.com/nat2090.html
29. Etchells, P. (2014, May 28). Psychology's replication drive: It's not about you. The Guardian. Retrieved from http://www.theguardian.com/science/ head-quarters/2014/may/28/psychology-replication-drive-methods-bullying
30. Fanelli, D. (2010). "Positive" results increase down the Hierarchy of the Sciences. PLoS ONE, 5, e10068. http://dx.doi.org/10.1371/journal.pone .0010068
31. Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences of the United States of America, 109, 17028-17033. http://dx.doi.org/10.1073/pnas.1212247109
32. Ferguson, C. J. (2009). Is psychological research really as good as medical research? Effect size comparisons between psychology and medicine. Review of General Psychology, 13, 130-136. http://dx.doi.org/10.1037/ a0015103
33. Ferguson, C. J. (2014). Is video game violence bad? The Psychologist, 27, 324-327.
34. Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods, 17, 120-128. http://dx.doi.org/10.1037/a0024445
35. Fergusson, D. M., Horwood, L. J., & Boden, J. M. (2008). Abortion and mental health disorders: Evidence from a 30-year longitudinal study. The British Journal of Psychiatry, 193, 444-451. http://dx.doi.org/ 10.1192/bjp.bp.108.056499
36. Fiedler, K. (2011). Voodoo correlations are everywhere: Not only in neuroscience. Perspectives on Psychological Science, 6, 163-171. http:// dx.doi.org/10.1177/1745691611400237
37. Finkelhor, D. (2010, October). The internet, youth deviance and the problem of "juvenoia." Paper presented at the Justice Studies Colloquium. Retrieved from https://vimeo.com/16900027
38. Firth, U., & Firth, C. (2014, May 28). A question of trust: Fixing the replication crisis. The Guardian. Retrieved from http://www.theguardian. com/science/occams-corner/2014/may/28/question-trust-fixing-replication-crisis-experimenter-reputation
39. Fowler, R. (1999). Letter to Tom Delay. Retrieved from http://www.lead ershipcouncil.org/1/rind/apa.html
40. Francis, G., Tanzman, J., & Matthews, W. J. (2014). Excess success for psychology articles in the journal science. PLoS ONE, 9, e114255. Retrieved from http://www.plosone.org/article/info%3Adoi%2F10.1371% 2Fjournal.pone.0114255.,http://dx.doi.org/10.1371/journal.pone.0114255
41. Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345, 1502-1505. http://dx.doi.org/10.1126/science.1255484
42. Freedman, J. L. (1984). Effect of television violence on aggressiveness. Psychological Bulletin, 96, 227-246. http://dx.doi.org/10.1037/0033-2909.96.2.227
43. French, C. (2012, March 15). Precognition studies and the curse of the failed replications. The Guardian. Retrieved from http://www.theguardian.com/ science/2012/mar/15/precognition-studies-curse-failed-replications
44. Fritz, A., Scherndl, T., & Kühberger, A. (2012, April). Correlation between effect size and sample size in psychological research: Sources, consequences, and remedies. Paper presented at the 10th Conference of the Austrian Psychological Society, Graz, Austria.
45. Galván, A. (2013). The teenage brain: Sensitivity to rewards. Current Directions in Psychological Science, 22, 88-93. http://dx.doi.org/ 10.1177/0963721413480859
46. Goffin, S. G., & Myers, M. (1991). A landscape of concerns: A national survey of position papers on children's issues. Child Welfare: Journal of Policy, Practice, and Program, 70, 529-540.
47. Greenwald, A. G. (2012). There is nothing so theoretical as a good method. Perspectives on Psychological Science, 7, 99-108. http://dx .doi.org/10.1177/1745691611434210
48. Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress? Psychological Review, 93, 216-229. http://dx.doi.org/10.1037/0033-295X.93.2.216
49. Grijalva, E., Newman, D. A., Tay, L., Donnellan, M. B., Harms, P. D., Robins, R. W., & Yan, T. (2015). Gender differences in narcissism: A meta-analytic review. Psychological Bulletin, 141, 261-310. http://dx .doi.org/10.1037/a0038231
50. Grisso, T., & Steinberg, L. (2005). Between a rock and a soft place: Developmental research and the child advocacy process. Journal of Child and Adolescent Psychology, 34, 619-627. http://dx.doi.org/ 10.1207/s15374424jccp3404_4
51. Grohol, J. (2014). Emotional contagion on Facebook? More like bad research methods. PsychCentral. Retrieved from http://psychcentral .com/blog/archives/2014/06/23/emotional-contagion-on-facebook-morelike-bad-research-methods/
52. Gutting, G. (2012, May 17). How reliable are the social sciences? The New York Times. Retrieved from http://opinionator.blogs.nytimes.com/ 2012/05/17/how-reliable-are-the-social-sciences/
53. Haggstrom, O. (2015). Intellectual suicide by the journal Basic and Applied Social Psychology. Retrieved from http://haggstrom.blogspot .com/2015/02/intellectual-suicide-by-journal-basic.html
54. Hall, R. C., Day, T., & Hall, R. C. (2011). A plea for caution: Violent video games, the Supreme Court, and the role of science. Mayo Clinic Proceedings, 86, 315-321. http://dx.doi.org/10.4065/mcp.2010.0762
55. Hendricks, V. (2014, November 5). Scientific research can be prone to bubbles too: Neuroscience risks being the next one. The Conversation. Retrieved from http://theconversation.com/scientific-research-can-beprone-to-bubbles-too-neuroscience-risks-being-the-next-one-33797
56. Heusmann, L., & Taylor, L. (2003). The case against the case against media violence. In D. Gentile (Ed.), Media violence and children: A complete guide for parents and professionals (pp. 107-130). New York, NY: Praeger.
57. Hsu, L. M. (2004). Biases of success rate differences shown in binomial effect size displays. Psychological Methods, 9, 183-197. http://dx.doi .org/10.1037/1082-989X.9.2.183
58. Ioannidis, J. P. (2005). Why most published research findings are false. PLoS. Med, 2, e124. Retrieved from http://www.plosmedicine.org/ article/info:doi/10.1371/journal.pmed.0020124
59. Ioannidis, J. P. (2012). Scientific inbreeding and same-team replication: Type D personality as an example. Journal of Psychosomatic Research, 73, 408-410. http://dx.doi.org/10.1016/j.jpsychores.2012.09.014
60. John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524-532. http://dx.doi.org/10.1177/ 0956797611430953
61. Johnson, D. J., Cheung, F., & Donnellan, M. (2014). Does cleanliness influence moral judgments? A direct replication of Schnall, Benton, and Harvey (2008). Social Psychology, 45, 209-215. http://dx.doi.org/ 10.1027/1864-9335/a000186
62. Kazdin, A., & Rotello, C. (2009). I spy daddy giving someone the finger: Your kids will imitate you, use it as a force for good. Retrieved from http://www.slate.com/articles/life/family/2009/01/i_spy_daddy_giving_ someone_the_finger.html
63. Klein, O., Doyen, S., Leys, C., de Saldanha da Gama, P., Miller, S., Questienne, L., & Cleeremans, A. (2012). Low hopes, high expectations: Expectancy effects and the replicability of behavioral experiments. Perspectives on Psychological Science, 7, 572-584. http://dx .doi.org/10.1177/1745691612463704
64. Konrath, S. H., O'Brien, E. H., & Hsing, C. (2011). Changes in dispositional empathy in American college students over time: A meta-analysis. Personality and Social Psychology Review, 15, 180-198. http://dx.doi.org/10.1177/1088868310377395
65. Kraemer, H. C. (2005). A simple effect size indicator for two-group comparisons? A comment on r equivalent. Psychological Methods, 10, 413-419.
66. Kramer, A., Guillory, J., & Hancock, J. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences of the United States of America, 111, 8788-8790. http://dx.doi.org/10.1073/pnas .1320040111
67. Lang, A. (2013). Discipline in crisis? The shifting paradigm of mass communication research. Communication Theory, 23, 10-24. http://dx .doi.org/10.1111/comt.12000
68. Lilienfeld, S. O. (2002). When worlds collide: Social science, politics, and the Rind et al. (1998) child sexual abuse meta-analysis. AmericanPsychologist, 57, 176-188. http://dx.doi.org/10.1037/0003-066X.57.3 .176
69. Lilienfeld, S. O. (2012). Public skepticism of psychology: Why many people perceive the study of human behavior as unscientific. American Psychologist, 67, 111-129. http://dx.doi.org/10.1037/a0023963
70. Lilienfeld, S., Lynn, S., Ruscio, J., & Beyerstein, B. (2009). Mythbusting in introductory psychology courses: The whys and the hows. Essays from e-xcellence in teaching (Vol. 9). Retrieved from http://teachpsych .org/ebooks/eit2009/index.php
71. Males, M. (2009). Does the adolescent brain make risk taking inevitable? A skeptical appraisal. Journal of Adolescent Research, 24, 3-20. http:// dx.doi.org/10.1177/0743558408326913
72. Martel, M. M. (2009). The ethics of psychology's role in politics and the development and institution of social policy. Ethics & Behavior, 19, 103-111. http://dx.doi.org/10.1080/10508420902772694
73. Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., . . . Reed, G. M. (2001). Psychological testing and psychological assessment. A review of evidence and issues. American Psychologist, 56, 128-165.
74. Mitchell, J. (2014). On the emptiness of failed replications. Retrieved from http://wjh.harvard.edu/~jmitchel/writing/failed_science.htm
75. Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379-387. http://dx.doi.org/10.1037/0003-066X.38.4.379
76. Neuroskeptic. (2014). On "on the emptiness of failed replications." Retrieved from http://blogs.discovermagazine.com/neuroskeptic/2014/07/ 07/emptiness-failed-replications/
77. New York Times. (1994). Public health approach to crime urged. Retrieved from http://articles.sun-sentinel.com/1994-10-16/news/9410 150290_1_gun-control-public-health-approach-dr-mark-rosenberg
78. Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2, 175-220. http://dx.doi .org/10.1037/1089-2680.2.2.175
79. Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7, 615-631. http://dx .doi.org/10.1177/1745691612459058
80. Novella, S. (2015, February 27). Psychology journal bans statistics. Real Clear Science. Retrieved from http://www.realclearscience-.com/2015/02/27/psychology_journal_bans_statistics_263538.html
81. Nyhan, B. (2014, September 18). To get more out of science, show the rejected research. New York Times. Retrieved from http://www.nytimes. com/2014/09/19/upshot/to-get-more-out-of-science-show-therejected-research.html?abt_0002&abg_0
82. O'Donohue, W., & Dyslin, C. (1996). Abortion, boxing and Zionism: Politics and the APA. New Ideas in Psychology, 14, 1-10. http://dx.doi .org/10.1016/0732-118X(95)00025-C
83. Olson, B., Soldz, S., & Davis, M. (2008). The ethics of interrogation and the American Psychological Association: A critique of policy and process. Philosophy, Ethics, and Humanities in Medicine, 3, 3. http:// dx.doi.org/10.1186/1747-5341-3-3
84. Pashler, H., Coburn, N., & Harris, C. R. (2012). Priming of social distance? Failure to replicate effects on social and food judgments. PLoS ONE, 7, e42510.
85. Pedersen, W. (2007). Childbirth, abortion and subsequent substance use in young women: A population-based longitudinal study. Addiction, 102, 1971-1978. http://dx.doi.org/10.1111/j.1360-0443.2007.02040.x
86. Pinker, S. (2002). The blank slate: The modern denial of human nature. New York, NY: Penguin.
87. Przybylski, A. K., Rigby, C. S., & Ryan, R. M. (2010). A motivational model of video game engagement. Review of General Psychology, 14, 154-166. http://dx.doi.org/10.1037/a0019440
88. Redding, R. E. (2001). Sociopolitical diversity in psychology. The case for pluralism. American Psychologist, 56, 205-215. http://dx.doi.org/ 10.1037/0003-066X.56.3.205
89. Risen, J. (2014). Pay any price. Boston, MA: Houghton Mifflin Harcourt.
90. Ritter, D., & Eslea, M. (2005). Hot sauce, toy guns and graffiti: A critical account of current laboratory aggression paradigms. Aggressive Behavior, 31, 407-419. http://dx.doi.org/10.1002/ab.20066
91. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638-641. http://dx.doi.org/10.1037/ 0033-2909.86.3.638
92. Rosnow, R. L., & Rosenthal, R. (2003). Effect sizes for experimenting psychologists. Canadian Journal of Experimental Psychology, 57, 221-237. http://dx.doi.org/10.1037/h0087427
93. Rouder, J. N., & Morey, R. D. (2012). Default Bayes factors for model selection in regression. Multivariate Behavioral Research, 47, 877-903. http://dx.doi.org/10.1080/00273171.2012.734737
94. Schimmack, U. (2012). The ironic effect of significant results on the credibility of multiple-study articles. Psychological Methods, 17, 551-566. http://dx.doi.org/10.1037/a0029487
95. Shipstead, Z., Redick, T. S., & Engle, R. W. (2012). Is working memory training effective? Psychological Bulletin, 138, 628-654. http://dx.doi .org/10.1037/a0027473
96. Shooter, J. (2011). Unpublished results hide the decline effect. Nature. Retrieved from http://www.nature.com/news/2011/110223/full/ 470437a.html
97. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. http://dx.doi.org/10.1177/0956797611417632
98. Simons, D. J. (2014). The value of direct replication. Perspectives on Psychological Science, 9, 76-80. http://dx.doi.org/10.1177/ 1745691613514755
99. Stanovich, K. E., West, R. F., & Toplak, M. E. (2013). Myside bias, rational thinking, and intelligence. Current Directions in Psychological Science, 22, 259-264. http://dx.doi.org/10.1177/0963721413480174
100. Steinberg, L., & Monahan, K. C. (2011). Adolescents' exposure to sexy media does not hasten the initiation of sexual intercourse. Developmental Psychology, 47, 562-576. http://dx.doi.org/10.1037/a0020613
101. Stratford, M. (2014). Symbolic slap at social sciences. Inside HigherEd. Retrieved from http://www.insidehighered.com/news/2014/06/02/housepasses-nsf-funding-bill-takes-slap-social-sciences#sthash.hh40CGu6.dpbs
102. Task Force on Women, Poverty, and Public Assistance. (1998). Making "welfare to work" really work: Improving welfare reform for poor women, families and children. Retrieved from http://www.apa.org/pi/women/programs/poverty/welfare-to-work.aspx
103. Tear, M. J., & Nielsen, M. (2013). Failure to demonstrate that playing violent video games diminishes prosocial behavior. PLoS ONE, 8, e68382. http://dx.doi.org/10.1371/journal.pone.0068382
104. Tedeschi, J. T., & Quigley, B. M. (1996). Limitations of laboratory paradigms for studying aggression. Aggression and Violent Behavior, 1, 163-177. http://dx.doi.org/10.1016/1359-1789(95)00014-3
105. The New Republic. (2006, September 4). Fared well. The New Republic. Retrieved from http://www.newrepublic.com/article/fared-well
106. Trafimow, D., & Marks, M. (2015). Editorial. Basic and Applied Social Psychology, 37, 1-2. http://dx.doi.org/10.1080/01973533.2015.1012991
107. Twenge, J. M., & Campbell, W. (2009). The narcissism epidemic: Living in the age of entitlement. New York, NY: Free Press.
108. van Lent, M., Overbeke, J., & Out, H. J. (2014). Role of editorial and peer review processes in publication bias: Analysis of drug trials submitted to eight medical journals. [Advance online publication]. PLoS ONE, 9, e104846. http://dx.doi.org/10.1371/journal.pone.0104846
109. Wagenmakers, E. J., Wetzels, R., Borsboom, D., & van der Maas, H. L. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100, 426-432. http://dx.doi.org/10.1037/a0022790
110. Wagenmakers, E., Wetzels, R., Borsboom, D., van der Maas, H. J., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632-638. http://dx.doi.org/ 10.1177/1745691612463078
111. Wai, J. (2012, June 10). Can psychology be considered a science? Psychology Today. Retrieved from http://www.psychologytoday.com/blog/finding-the-next-einstein/201206/can-psychology-be-considered-science
112. Weeks, N. (2005). A few minutes of your time: Tips on communicating scientific information to the media. The Observer. Retrieved from http://www.psychologicalscience.org/index.php/uncategorized/a-fewminutes-of-your-time-tips-on-communicating-scientific-informationto-the-media.html
113. Wicherts, J., & Bakker, M. (2009). Sharing: Guidelines go one step forwards, two steps back. Nature, 461, 1053. http://dx.doi.org/10.1038/4611053c
114. Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS ONE, 6, e26828. http://dx.doi.org/10.1371/journal.pone.0026828
115. Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726-728. http://dx.doi.org/10.1037/0003-066X.61.7.726
116. Willoughby, T., Good, M., Adachi, P. J., Hamza, C., & Tavernier, R. (2013). Examining the link between adolescent brain development and risk taking from a social-developmental perspective. Brain and Cognition, 83, 315-323. http://dx.doi.org/10.1016/j.bandc.2013.09.008
117. Wilson, A., & Golonka, S. (2012). The small effect size effect: Why do we put up with small effects? Retrieved from http://psychsciencenotes.blogspot.com/2012/08/the-small-effect-size-effect-why-do-we.html
118. Wolfe, J., & Roberts, C. (1993). A further study of the external validity of business games: Five-year peer group indicators. Simulation & Gaming, 24, 21-33. http://dx.doi.org/10.1177/1046878193241004
119. Wolins, L. (1962). Responsibility for raw data. American Psychologist, 17, 657-658. http://dx.doi.org/10.1037/h0038819
120. Yong, E. (2012). Replication studies: Bad copy. Nature. Retrieved from http://www.nature.com/news/replication-studies-bad-copy-1.10634