Questionable research practices among Italian research psychologists

PLoS ONE

Volumn 12, Issue 3, 2017, Pages

  Agnoli, Franca ^a   Wicherts, Jelte M ^b   Veldkamp, Coosje L S ^b   Albiero, Paolo ^a   Cubelli, Roberto ^c  

^a UNIVERSITY OF PADOVA   (Italy)

^b TILBURG UNIVERSITY   (Netherlands)

^c UNIVERSITY OF TRENTO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ADOPTION; GERMANY; HUMAN; HUMAN EXPERIMENT; PSYCHOLOGIST; PSYCHOLOGY; PUBLICATION; SCIENTIST; UNITED STATES; ITALY; METHODOLOGY;

HUMANS; ITALY; PSYCHOLOGY; RESEARCH DESIGN; UNITED STATES;

EID: 85015277026     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0172792     Document Type: Article

References (54)

1. National Academy of Sciences, National Academy of Engineering, & Institute of Medicine. Responsible science: Ensuring the integrity of the research process ( Vol. I). Washington, DC: National Academy Press, 1992.
2. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug 30; 2(8):e124. doi: 10.1371/journal.pmed.0020124 PMID: 16060722
3. Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015 Aug 28; 349(6251):aac4716. doi: 10.1126/science.aac4716 PMID: 26315443
4. Pashler H, Wagenmakers EJ. Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science. 2012 Nov 1; 7(6):528-30. doi: 10.1177/1745691612465253 PMID: 26168108
5. Spellman BA. Introduction to the special section on methods: Odds and end. Perspectives on Psychological Science. 2015 May 1; 10(3):359-60. doi: 10.1177/1745691615582201 PMID: 25987514
6. Franco A, Malhotra N, Simonovits G. Underreporting in psychology experiments: Evidence from a study registry. Social Psychological and Personality Science. 2016; 7(1):8-12.
7. LeBel EP, Borsboom D, Giner-Sorolla R, Hasselman F, Peters KR, Ratliff KA, et al. PsychDisclosure. org: Grassroots support for reforming reporting standards in psychology. Perspectives on Psychological Science. 2013 Jul 1; 8(4):424-32. doi: 10.1177/1745691613491437 PMID: 26173121
8. Fanelli D. Positive results increase down the hierarchy of the sciences. PLOS ONE. 2010 Apr 7; 5(4): e10068. doi: 10.1371/journal.pone.0010068 PMID: 20383332
9. Bakker M, van Dijk A, Wicherts JM. The rules of the game called psychological science. Perspectives on Psychological Science. 2012 Nov 1; 7(6):543-54. doi: 10.1177/1745691612459060 PMID: 26168111
10. Cohen J. Things I have learned (so far). American Psychologist. 1990 Dec; 45(12):1304.
11. Francis G, Tanzman J, Matthews WJ. Excess success for psychology articles in the journal Science. PLOS ONE. 2014 Dec 4; 9(12):e114255. doi: 10.1371/journal.pone.0114255 PMID: 25474317
12. Maxwell SE. The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychological Methods. 2004 Jun; 9(2):147-63. doi: 10.1037/1082-989X.9.2. 147 PMID: 15137886
13. Maxwell SE, Kelley K, Rausch JR. Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology. 2008 Jan 10; 59:537-63. doi: 10.1146/annurev.psych.59. 103006.093735 PMID: 17937603
14. Schimmack U. The ironic effect of significant results on the credibility of multiple-study articles. Psychological Methods. 2012 Dec; 17(4):551-66. doi: 10.1037/a0029487 PMID: 22924598
15. O'Boyle EH, Banks GC, Gonzalez-Mulé E. The Chrysalis effect: How ugly initial results metamorphosize into beautiful articles. Journal of Management. 2014 Mar 19:0149206314527133.
16. Kerr NL. HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review. 1998 Aug 1; 2(3):196-217. doi: 10.1207/s15327957pspr0203-4 PMID: 15647155
17. Wagenmakers EJ, Wetzels R, Borsboom D, van der Maas HL, Kievit RA. An agenda for purely confirmatory research. Perspectives on Psychological Science. 2012 Nov 1; 7(6):632-8. doi: 10.1177/1745691612463078 PMID: 26168122
18. Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science. 2011 Oct 1:0956797611417632.
19. Wagenmakers EJ, Wetzels R, Borsboom D, van der Maas HL. Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). Journal of Personality and Social Psychology. 2011; 100(3):426-432. doi: 10.1037/a0022790 PMID: 21280965
20. Bakker M, Wicherts JM. The (mis) reporting of statistical results in psychology journals. Behavior Research Methods. 2011 Sep 1; 43(3):666-78. doi: 10.3758/s13428-011-0089-5 PMID: 21494917
21. Nuijten MB, Hartgerink CH, Assen MA, Epskamp S, Wicherts JM. The prevalence of statistical reporting errors in psychology (1985-2013). Behavior Research Methods. 2016 Dec; 48(4):1205-26. doi: 10. 3758/s13428-015-0664-2 PMID: 26497820
22. Anderson MS, Martinson BC, de Vries R. Normative dissonance in science: Results from a national survey of US scientists. Journal of Empirical Research on Human Research Ethics. 2007 Dec 1; 2(4):3-14. doi: 10.1525/jer.2007.2.4.3 PMID: 19385804
23. Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-Analysis of survey data. PLOS ONE. 2009 May 29; 4(5):e5738. doi: 10.1371/journal.pone.0005738 PMID: 19478950
24. Martinson BC, Anderson MS, de Vries R. Scientists behaving badly. Nature. 2005 Jun 9; 435 (7043):737-8. doi: 10.1038/435737a PMID: 15944677
25. John LK, Loewenstein G, Prelec D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science. 2012 Apr 16:0956797611430953.
26. Fiedler K, Schwarz N. Questionable research practices revisited. Social Psychological and Personality Science. 2015 Oct 19:1948550615612150.
27. Fanelli D, Ioannidis JP. US studies may overestimate effect sizes in softer research. Proceedings of the National Academy of Sciences. 2013 Sep 10; 110(37):15031-6.
28. Nuijten MB, van Assen MA, van Aert RC, Wicherts JM. Standard analyses fail to show that US studies overestimate effect sizes in softer research. Proceedings of the National Academy of Sciences. 2014 Feb 18; 111(7):E712-3.
29. Publication Manual of the American Psychological Association. Washington, DC: American Psychological Association. Technical and research reports. 2010.
30. Guttman L. A basis for scaling qualitative data. American Sociological Review. 1944 Apr 1; 9(2):139-50.
31. Guest G. Using Guttman scaling to rank wealth: integrating quantitative and qualitative data. Field Methods. 2000 Nov 1; 12(4):346-57.
32. Wagenmakers EJ. A practical solution to the pervasive problem of p values. Psychonomic Bulletin & Review. 2007 Oct 1; 14(5):779-804.
33. Corbin J, Strauss A. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications; 2014 Nov 25.
34. Perugini M. The international credibility crisis of psychology as an opportunity for growth: Problems and possible solutions. Giornale Italiano di Psicologia. 2014; 41(1):23-46.
35. Van Aert RCM, Wicherts JM, van Assen MALM. Conducting meta-Analyses based on p values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science. 2016 Sep; 11(5):713-29. doi: 10.1177/1745691616650874 PMID: 27694466
36. Kline RB. Beyond significance testing: Statistics reform in the behavioral sciences. American Psychological Association; 2013.
37. Pastore M, Nucci M, Bobbio A. Life of P: 16 years of statistics on the Italian Journal of Psychology. Giornale Italiano di Psicologia. 2015; 42(1-2):303-28.
38. Ioannidis JP, Munafó MR, Fusar-Poli P, Nosek BA, David SP. Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends in Cognitive Sciences. 2014 May 31; 18(5):235-41. doi: 10.1016/j.tics.2014.02.010 PMID: 24656991
39. Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012; 90(3):891-904.
40. Necker S. Scientific misbehavior in economics. Research Policy. 2014 Dec 31; 43(10):1747-59.
41. van Dalen HP, Henkens K. Intended and unintended consequences of a publish-or-perish culture: A worldwide survey. Journal of the American Society for Information Science and Technology. 2012 Jul 1; 63(7):1282-93.
42. Marzolla M. Quantitative analysis of the Italian national scientific qualification. Journal of Infometrics. 2015 Apr 30; 9(2):285-316.
43. Bartoli A, Medvet E. Bibliometric evaluation of researchers in the internet age. The Information Society. 2014 Oct 20; 30(5):349-54.
44. Cubelli R, Della Sala S. Write less, write well. Cortex; a journal devoted to the study of the nervous system and behavior. 2015 Dec 1; 73:A1-2. doi: 10.1016/j.cortex.2015.05.008 PMID: 26059475
45. Sarewitz D. The pressure to publish pushes down quality. Nature. 2016 May 11; 533(7602):147. doi: 10.1038/533147a PMID: 27172010
46. Banks GC, Robelberg SG, Woznyj HM, Landis RS, Rupp DE. Editorial: evidence on questionable research practices: the good, the bad, and the ugly. Journal of Business and Psychology. 2016; 33:323-38.
47. Wicherts JM, Bakker M, Molenaar D. Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLOS ONE. 2011 Nov 2; 6(11):e26828. doi: 10.1371/journal.pone.0026828 PMID: 22073203
48. Veldkamp CL, Nuijten MB, Dominguez-Alvarez L, van Assen MA, Wicherts JM. Statistical reporting errors and collaboration on statistical analyses in psychological science. PLOS ONE. 2014 Dec 10; 9 (12):e114876. doi: 10.1371/journal.pone.0114876 PMID: 25493918
49. Nosek BA, Spies JR, Motyl M. Scientific utopia II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science. 2012 Nov 1; 7(6):615-31. doi: 10. 1177/1745691612459058 PMID: 26168121
50. Wagenmakers EJ, Wetzels R, Borsboom D, van der Maas HL, Kievit RA. An agenda for purely confirmatory research. Perspectives on Psychological Science. 2012 Nov 1; 7(6):632-8. doi: 10.1177/1745691612463078 PMID: 26168122
51. Chambers CD. Registered reports: a new publishing initiative at Cortex. Cortex; a journal devoted to the study of the nervous system and behavior. 2013 Mar; 49(3):609-10. doi: 10.1016/j.cortex.2012.12.016 PMID: 23347556
52. De Groot AD. The meaning of significance for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han LJ van der Maas]. Acta Psychologica. 2014 May 31; 148:188-94. doi: 10.1016/j.actpsy.2014.02.001 PMID: 24589374
53. Bakker M, Hartgerink CH, Wicherts JM, van der Maas HL. Researchers' Intuitions About Power in Psychological Research. Psychological Science. 2016 Aug 1; 27(8):1069-77. doi: 10.1177/0956797616647519 PMID: 27354203
54. Asendorpf JB, Conner M, De Fruyt F, De Houwer J, Denissen JJ, Fiedler K, et al. Recommendations for increasing replicability in psychology. European Journal of Personality. 2013 Mar 1; 27(2):108-19.