The international credibility crisis of psychology as an opportunity for growth: Problems and possible solutions;La crisi internazionale di credibilità della psicologia come un'opportunità di crescita: Problemi e possibili soluzioni

Giornale Italiano di Psicologia

Volumn 41, Issue 1, 2014, Pages 23-46

  Perugini, Marco ^a  

^a UNIVERSITY OF MILANO BICOCCA   (Italy)

Author keywords

Credibility; False positives; Methodology; Replicability; Statistical power

Indexed keywords

EID: 84906309833     PISSN: 03905349     EISSN: None     Source Type: Journal    
DOI: 10.1421/77194     Document Type: Article

References (74)

1. Agnoli F., Furlan E.S., Padova U. (2009). Statistiche migliori per decisioni. Giornale Italiano di Psicologia, 36, 849-882.
2. Aldhous P. (2011, 5 May). Journal rejects studies contradicting precognition. New Scientist, http://tinyurl.com/3rsb8hs (11/08/2013).
3. Asendorpf J.B., Conner M., DeFruyt F., DeHouwer J., Denissen J.J.A., Fiedler K., Fiedler S., Funder D.C., Kliegl R., Nosek B.A., Perugini M., Roberts B.W., Schmitt M., Aken M.A.G., Weber H., Wicherts J.M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27 (2), 108-119.
4. Bakker M., van Dijk A., Wicherts J.M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7 (6), 543-554.
5. Bargh J.A., Chen M., Burrows L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 71 (2), 230-244.
6. Begley C.G., Ellis L.M. (2012). Drug development: Raise standards for preclinical cancer research. Nature, 483, 531-533.
7. Bem D.J. (2003). Writing the empirical journal article. In J.M. Darley, M.P. Zanna, H.L. Roediger III (eds.), The complete academic: A career guide. Washington, D.C.: American Psychological Association, pp. 171-201.
8. Bem D.J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100 (3), 407-425.
9. Biederman J. (2008). Credibility crisis in pediatric psychiatry. Nature Neuroscience, 11 (11), 1233.
10. Brandt M.J., Ijzerman H., Dijksterhuis A., Farach F.J., Geller J., Giner-Sorolla R., Grange J.A., Perugini M., Spies J., van 't Veer A. (2013). The Replication Recipe: What Makes for a Convincing Replication? Journal of Experimental Social Psychology, 50, 217-224.
11. Button K.S., Ioannidis J.P.A., Mokrysz C., Nosek B.A., Flint J., Robinson E.S.J., Munafò M.R. (2013). Confidence and precision increase with high statistical power. Nature Reviews Neuroscience, 14 (5), 585.
12. Carpenter S. (2012). Psychology's bold initiative. Science, 335, 1558-1560.
13. Cohen J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65 (3), 145-153.
14. Cumming G. (2008). Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science, 3 (4), 286-300.
15. Cumming G., Fidler F., Leonard M., Kalinowski P., Christiansen A., Kleinig A., Lo N., McMenamin N., Wilson S. (2007). Statistical reform in psychology: Is anything changing? Psychological Science, 18 (3), 230-232.
16. Cumming G., Maillardet R. (2006). Confidence intervals and replication: Where will the next mean fall? Psychological Methods, 11 (3), 217-227.
17. DeHouwer J. (2011). Why the cognitive approach in psychology would profit from a functional approach and vice versa. Perspectives on Psychological Science, 6 (2), 202-209.
18. DeHouwer J., Gawronski B., Barnes-Holmes D. (in stampa). A functional-cognitive framework for attitude research. European Review of Social Psychology.
19. Doyen S., Klein O., Pichon C.-L., Cleeremans A. (2012). Behavioral priming: It's all in the mind, but whose mind? PloS One, 7 (1), e29081.
20. Enserink M. (2011). Dutch university sacks social psychologist over faked data. Science, http://news.sciencemag.org/scienceinsider/2011/09/dutch- universitysacks-social.html (30/07/2013).
21. Fanelli D. (2010). «Positive» results increase down the Hierarchy of the Sciences. PloS One, 5 (4), e10068.
22. Faul F., Erdfelder E., Lang A.-G., Buchner A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39 (2), 175-191.
23. Fiedler K., Kutzner F., Krueger J.I. (2012). The long way from a-error control to validity proper: Problems with a short-sighted false-positive debate. Perspectives on Psychological Science, 7 (6), 661-669.
24. Francis G. (2013). Replication, statistical consistency, and publication bias. Journal of Mathematical Psychology, 57, 153-169.
25. Frank M.C., Saxe R. (2012). Teaching replication. Perspectives on Psychological Science, 7 (6), 600-604.
26. Frick R.W. (1996). The appropriate use of null hypothesis testing. Psychological Methods, 1 (4), 379-390.
27. Friston K. (2012). Ten ironic rules for non-statistical reviewers. NeuroImage, 61 (4), 1300-1310.
28. Galak J., LeBoeuf R.A., Nelson L.D., Simmons J.P. (2012). Correcting the past: Failures to replicate psi. Journal of Personality and Social Psychology, 103 (6), 933-948.
29. Gigerenzer G., Krauss S., Vitouch O. (2004). The null ritual: What you always wanted to know about significance testing but were afraid to ask. In D. Kaplan (ed.), The SAGE handbook of quantitative methodology for the social sciences. Thousand Oaks, CA: Sage, pp. 391-408.
30. Giner-Sorolla R. (2012). Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspectives on Psychological Science, 7 (6), 562-571.
31. Henrich J., Heine S.J., Norenzayan A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33 (2-3), 61-83.
32. Ingre M. (2013). Why small low-powered studies are worse than large high-powered studies and how to protect against «trivial» findings in research: Comment on Friston (2012). NeuroImage, 81, 496-498.
33. Ioannidis J.P.A. (2005). Why most published research findings are false. PLoS Medicine, 2 (8), e124.
34. Iverson G.J., Lee M.D., Zhang S., Wagenmakers E.-J. (2009). P-rep: An agony in five Fits. Journal of Mathematical Psychology, 53 (4), 195-202.
35. Jasny B.R., Chin G., Chong L., Vignieri S. (2011). Again, and again, and again... Science, 334, 1225.
36. John L.K., Loewenstein G., Prelec D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23 (5), 524-532.
37. Judd C.M., Westfall J., Kenny D.A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103 (1), 54-69.
38. Kerr N.L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196-217.
39. Killeen P.R. (2005). An alternative to null-hypothesis significance tests. Psychological Science, 16 (5), 345-353.
40. LeBel EP, Campbell L. (2013). Heightened sensitivity to temperature cues in highly anxiously attached individuals: Real or elusive phenomenon? Psychological Science, 24, 2128-2130.
41. LeBel E.P., Borsboom D., Giner-Sorolla R., Hasselman F., Peters K.R., Ratliff K.A., Smith C.T. (2013). PsychDisclosure.org: Grassroots support for reforming reporting standards in psychology. Perspectives on Psychological Science, 8 (4), 424-432.
42. LeBel E.P, Paunonen S.V. (2011). Sexy but often unreliable: The impact of unreliability on the replicability of experimental findings with implicit measures. Personality and Social Psychology Bulletin, 37 (4), 570-583.
43. LeBel E.P., Peters K.R. (2011). Fearing the future of empirical psychology: Bem's (2011) evidence of psi as a case study of deficiencies in modal research practice. Review of General Psychology, 15 (4), 371-379.
44. Levelt P., Drenth P., Noort E. (2012). Flawed science: The fraudulent research practices of the social psychologist Diederik Stapel, https://www.commissielevelt.nl/ (30/07/2013).
45. Masicampo E.J., Lalande D.R. (2012). A peculiar prevalence of p values just below .05. Quarterly Journal of Experimental Psychology, 65 (11), 2271-2279.
46. Maxwell S.E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9, 147-163.
47. Maxwell S.E., Kelley K., Rausch J.R. (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59, 537-563.
48. Miller J. (2009). What is the probability of replicating a statistically significant effect? Psychonomic Bulletin & Review, 16 (4), 617-640.
49. Miller J., Schwarz W. (2011). Aggregate and individual replication probability within an explicit model of the research process. Psychological Methods, 16 (3), 337-360.
50. Nickerson R.S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5 (2), 241-301.
51. Nosek B.A., Spies J.R., Motyl M. (2012). Scientific utopia: ii. restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7 (6), 615-631.
52. Open ScienceCollaboration (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7 (6), 657-660.
53. Pashler H., Harris C.R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7 (6), 531-536.
54. Pashler H., Wagenmakers E.-J. (2012). Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7 (6), 528-530.
55. Ritchie S.J., Wiseman R., French C.C. (2012). Failing the future: Three unsuccessful attempts to replicate Bem's «retroactive facilitation of recall» effect. PloS One, 7 (3), e33423.
56. Rothwell P.M. (2005). Treating individuals 1 external validity of randomised controlled trials: «to whom do the results of this trial apply ?». Lancet, 365, 82-93.
57. Schimmack U. (2012). The ironic effect of significant results on the credibility of multiple-study articles. Psychological Methods, 17 (4), 551-566.
58. Schönbrodt F.D., Perugini M. (2013). At what sample size do correlations stabilize? Journal of Research in Personality, 47, 609-612.
59. Sears D.O. (1986). College sophomores in the laboratory: Influences of a narrow data base on social psychology's view of human nature. Journal of Personality and Social Psychology, 51 (3), 515-530.
60. Shanks D.R., Newell B.R., Lee E.H., Balakrishnan D., Ekelund L., Cenac Z., Kavvadia F., Moore C. (2013). Priming intelligent behavior: An elusive phenomenon. PloS One, 8 (4), e56515.
61. Simmons J.P., Nelson L.D., Simonsohn U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22 (11), 1359-1366.
62. Simonsohn U. (2013). Just post it: The lesson from two cases of fabricated data detected by statistics alone. Psychological Science, 24, 1875-1888.
63. Tversky A., Kahneman D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76 (2), 105-110.
64. Van Lange P.A.M., Buunk A.P., Ellemers N., Wigboldus D.H.J. (2012). Sharpening scientific policy after stapel. Kurt Lewin Institute, http://www.kurtlewininstituut.nl/research/research/mission-statement/ (05/09/2013).
65. Vidotto G. (2009). Limiti e alternative della verifica dell'ipotesi nulla: Presentazione. Giornale Italiano di Psicologia, 36, 845-848.
66. Vul E., Harris C., Winkielman P., Pashler H. (2009). Puzzlingly high correlations in fMRI atudies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4 (3), 274-290.
67. Wagenmakers E.J., Wetzels R., Borsboom D., van der Maas H.L.J. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100 (3), 426-432.
68. Wagenmakers E.J., Wetzels R., Borsboom D., van der Maas H.L.J., Kievit R.A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7 (6), 632-638.
69. Wicherts J.M., Bakker M., Molenaar D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PloS One, 6 (11), e26828.
70. Wilkinson L., TaskForceStatistical Inference(1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
71. Yong E. (2012a). Bad copy: In the wake of high-profile controversies, psychologists are facing up to problems with replication. Nature, 485, 298-300.
72. Yong E. (2012b). The data detective. Nature, 487, 18-19.
73. Yong E. (2012c). Nobel laureate challenges psychologists to clean up their act. Nature, doi:10.1038/nature.2012.11535.
74. Young N.S., Ioannidis J.P.A., Al-Ubaydli O. (2008). Why current publication practices may distort science. PLoS Medicine, 5 (10), e201.