Outlier removal and the relation with reporting errors and quality of psychological research

PLoS ONE

Volumn 9, Issue 7, 2014, Pages

  Bakker, Marjan ^a   Wicherts, Jelte M ^b  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b TILBURG UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONTROLLED STUDY; DATA ANALYSIS; EVIDENCE BASED MEDICINE; HUMAN; MEDICAL RESEARCH; NULL HYPOTHESIS; PREVALENCE; PSYCHOLOGY; PUBLICATION; QUALITY CONTROL; SAMPLE SIZE; STATISTICAL ANALYSIS; SYSTEMATIC ERROR; RESEARCH; STANDARDS;

HUMANS; PSYCHOLOGY; RESEARCH;

EID: 84905014028     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0103360     Document Type: Article

References (36)

1. Wicherts JM, Borsboom D, Kats J, Molenaar D (2006) The poor availability of psychological research data for reanalysis. American Psychologist 61: 726-728. (Pubitemid 44547473)
2. Wolins L (1962) Responsibility for raw data. American Psychologist 17: 657-658.
3. Firebaugh G (2007) Replication data sets and favored-hypothesis bias. Sociological Methods & Research 36: 200-209.
4. Freese J (2007) Replication standards quantitative social science - Why not sociology. Sociological Methods & Research 36: 153-172.
5. Kyzas PA, Loizou KT, Ioannidis JPA (2005) Selective reporting biases in cancer prognostic factor studies. Journal of the National Cancer Institute 97: 1043-1055. (Pubitemid 41417921)
6. McCullough BD, Vinod HD (2003) Verifying the solution from a nonlinear solver: A case study. American Economic Review 93: 873-892. (Pubitemid 38799579)
7. Piwowar HA, Day RS, Fridsma DB (2007) Sharing detailed research data Is associated with increased citation rate. Plos One 2.
8. Reidpath DD, Allotey PA (2001) Data sharing in medical research: An empirical investigation. Bioethics 15: 125-134. (Pubitemid 32274884)
9. Savage CJ, Vickers AJ (2009) Empirical study of data sharing by authors publishing in PLoS Journals. Plos One 4.
10. Campbell EG, Clarridge BR, Gokhale NN, Birenbaum L, Hilgartner S, et al. (2002) Data withholding in academic genetics - Evidence from a national survey. Jama-Journal of the American Medical Association 287: 473-480. (Pubitemid 34251986)
11. Vines TH, Albert AY, Andrew RL, Débarre F, Bock DG, et al. (2014) The availability of research data declines rapidly with article age. Current Biology 24: 94-97.
12. American Psychological Association (2010) Publication Manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.
13. Wicherts JM, Bakker M, Molenaar D (2011) Willingness to share research data Is related to the strength of the evidence and the quality of reporting of statistical results. Plos One 6.
14. Bakker M, Wicherts JM (2011) The (mis)reporting of statistical results in psychology journals. Behavior Research Methods 43: 666-678.
15. Leggett NC, Thomas NA, Loetscher T, Nicholls ME (2013) The life of p: 'Just significant' results are on the rise. The Quarterly Journal of Experimental Psychology: 1-16.
16. Caperos JM, Pardo A (2013) Consistency errors in p-values reported in Spanish psychology journals. Psicothema 25: 408-414.
17. Fisher RA (1958) Statistical Methods for Research Workers. New York: Hafner.
18. Pashler H, Wagenmakers EJ (2012) Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science 7: 528-530.
19. Asendorpf JB, Conner M, Fruyt FD, Houwer JD, Denissen JJA, et al. (2013) Recommendations for increasing replicability in psychology. European Journal of Personality 27: 108-119.
20. Bakker M, van Dijk A, Wicherts JM (2012) The rules of the game called psychological science. Perspectives on Psychological Science 7: 543-554.
21. Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22: 1359-1366.
22. John LK, Loewenstein G, Prelec D (2012) Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science 23: 524-532.
23. Bakker M, Wicherts JM (in press) Outlier removal, sum scores, and the inflation of the Type I error rate in independent samples t tests. The power of alternatives and recommendations. Psychological Methods.
24. Osborne JW, Overbay A (2004) The power of outliers (and why researchers should always check for them). Practical assessment, research & evaluation 9: 1-12.
25. Simonsohn U, Nelson L, Simmons J (In press) P-curve: A key to the file drawer. Journal of Experimental Psychology: General.
26. Ioannidis JPA (2010) Meta-research: The art of getting it wrong. Research Synthesis Methods 1: 169-184.
27. Wagenmakers EJ, Wetzels R, Borsboom D, van der Maas HLJ, Kievit RA (2012) An agenda for purely confirmatory research. Perspectives on Psychological Science 7: 632-638.
28. Molenaar D, Wicherts JM, Borsboom D (2005) Participant loss in psychological research and the problem of exclusion: A validity issue. [Bachelor's thesis]. Amsterdam: University of Amsterdam.
29. Epskamp S, Nuijten MB (in preparation) statcheck: Extract statistics from articles and recompute p values. R package version 0.1.0.
30. Wicherts JM (2008) Expectancy bias, statistical blunders, and the importance of reanalysis. Invited talk at the 23 IOPS summer conference, June 19, Enschede, The Netherlands.
31. LeBel EP, Borsboom D, Giner-Sorolla R, Hasselman F, Peters KR, et al. (2013) PsychDisclosure.org: Grassroots support for reforming reporting standards in psychology. Perspectives on Psychological Science 8: 424-432.
32. Petrocelli JV, Clarkson JJ, Whitmire MB, Moon PE (2013) When ab not equal c - c ': Published errors in the reports of single-mediator models. Behavior Research Methods 45: 595-601.
33. Rossi JS (1987) How often are our statistics wrong? A statistics class exercise. Teaching of Psychology 14: 98-101.
34. Simmons JP, Nelson LD, Simonsohn U (2012) A 21-word solution. Dialogue; The Official Newsletter of the Society for Personality and Social Psychology 26: 4-7.
35. Eich E (2013) Business not as usual. Psychological Science.
36. Wilcox RR (2012) Modern statistics for the social and behavioral sciences: A practical introduction. Boca Raton: CRC Press.