Assessing the reliability of the M5-120 on Amazon's mechanical Turk

Computers in Human Behavior

Volumn 29, Issue 4, 2013, Pages 1749-1754

  Holden, Christopher J ^a   Dennie, Trevor ^a   Hicks, Adam D ^a  

^a WESTERN CAROLINA UNIVERSITY   (United States)

Author keywords

M5 120; Mechanical Turk; MTurk; Online research; Personality; Replication

Indexed keywords

DATA ACQUISITION;

AMAZON'S MECHANICAL TURKS; DATA COLLECTION; M5-120; MECHANICAL TURKS; ON-LINE SERVICE; ONLINE RESEARCH; PERSONALITY; REPLICATION; SOCIAL SCIENTISTS;

RELIABILITY;

EID: 84875701118     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.chb.2013.02.020     Document Type: Article

References (30)

1. Y.S. Ben-Porath, and A. Tellegen MMPI-2-RF Minnesota multiphasic personality inventory - 2-restructured form: Manual for administration, scoring, and interpretation 2008 University of Minnesota Press by NCS Pearson, Inc. Minneapolis, MN
2. M.D. Buhrmester, T. Kwang, and S.D. Gosling Amazon's mechanical Turk: a new source of inexpensive, yet high quality data Perspectives on Psychological Science 6 1 2011 3 5
3. D.M. Buss From vigilance to violence: Tactics of mate retention in American undergraduates Ethology and Sociobiology 9 1988 291 317
4. D.M. Buss, T.K. Shackelford, and W.F. McKibbin The mate retention inventory-short form (MRI-SF) Personality and Individual Differences 44 2008 322 334
5. J.N. Butcher, J.R. Graham, Y.S. Ben-Porath, A. Tellegen, W.G. Dahlstrom, and B. Kaemmer The Minnesota multiphasic personality inventory-2 (MMPI-2): Manual for administration and scoring 2001 University of Minnesota Press Minneapolis, MN
6. R. Cattell Confirmation and clarification of the primary personality factors Psychometrika 12 1947 197 220 10.1007/BF02289253
7. F.M. Cheung The cultural perspective in personality assessment J.N. Butcher, Oxford handbook of personality assessment 2009 Oxford University Press New York 44 58
8. C.R. Cloninger The temperament and character inventory (TCI): A guide to its development and use 1994 Washington University, Center for the Psychobiology of Personality St. Louis. MO
9. S.R. Conn, and M.L. Rieke The 16PF fifth edition technical manual 1994 Institute for Personality and Ability Testing Champaign, IL
10. P.T. Costa, and R.R. McCrae Four ways five factors are basic Personality and Individual Differences 13 6 1992 653 665 10.1016/0191-8869(92)90236-I
11. P.T. Costa, and R.R. McCrae Revised neopersonality inventory (NEO-PI-R) and neo five-factor inventory (NEO-FFI) professional manual 1992 Psychological Assessment Resources Odessa, FL
12. P.T. Costa, and R.R. McCrae Primary traits of Eysenck's P-E-N system: Three- and five-factor solutions Journal of Personality and Social Psychology 69 2 1995 308 317 10.1037/0022-3514.69.2.308
13. J. Digman Personality structure: Emergence of the five factor model Annual Review of Psychology 41 1990 417 440 10.1146/annurev.ps.41.020190.002221
14. J. Feist, and G.J. Feist Theories of personality 6th ed. 2006 McGraw Hill New York, NY
15. L.R. Goldberg A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models I. Mervielde, I. Deary, F. De Fruyt, F. Ostendorf, Personality psychology in Europe Vol. 7 1999 Tilburg University Press Tilburg, The Netherlands 7 28
16. S.D. Gosling, P.J. Rentfrow, and W.B. Swann Jr. A very brief measure of the big-five personality domains Journal of Research in Personality 37 2003 504 528 10.1016/S0092-6566(03)00046-1 (Pubitemid 37445626)
17. Hicks, A. D., Dennie, T., Taurasi, J., & McCord, D. M. (2011, April). The Association Between Anxiety, Depression, and Personality. Poster session at the North Carolina Psychological Association Spring Conference, Chapel Hill, NC.
18. R. Hogan, and J. Hogan Hogan personality inventory manual 1992 Hogan Assessment Systems Tulsa, OK
19. Holden, Christopher J. (2011, March). Amazon's MTurk: A flawed source of participants in psychology studies? Paper session presented at Western Carolina University's Nineteenth Annual Graduate Research Symposium, Cullowhee, NC.
20. J.J. Horton, D.G. Rand, and R.J. Zeckhauser The online laboratory: Conducting experiments in a real labor market Experimental Economics 14 2011 399 425 10.1007/s10683-011-9273-9
21. O.P. John, E.M. Donahue, and R.L. Kentle The big five inventory-versions 4a and 54 1991 University of California, Berkeley, Institute of Personality and Social Research Berkeley, CA
22. Johnson, J. A. (2001, May). Screening massively large data sets for non-responsiveness in web- based personality inventories. Invited talk to the joint Bielefeld-Groningen Personality Research Group, University of Groningen, The Netherlands.
23. W. Mason, and S. Suri Conducting behavioral research on Amazon's mechanical Turk Behavioral Research Methods 44 1 2012 1 23 10.3758/s13428-011- 0124-6
24. Mueller, P. & Chandler, J. (2012) Emailing workers using python. Available at SSRN: < http://ssrn.com/abstract=2100601 > or < http://dx.doi.org/10.2139/ssrn.2100601 >.
25. G. Paolacci, J. Chandler, and P. Ipeirotis Running experiments on Amazon mechanical Turk Judgment and Decision Making 5 5 2010 411 419
26. A.W. Paradise, and M.H. Kernis Development of the contingent self-esteem scale 1999 Unpublished data University of Georgia
27. E.J. Pedhazur, and L.P. Schmelkin Reliability E.J. Pedhazur, L.P. Schmelkin, Measurement, design and analysis: An integrated approach 1991 Lawrence Erlbaum and Associates Hillsdale, NJ 81 117
28. D.G. Rand The promise of mechanical Turk: How online labor markets can help theorists run behavioral experiments Journal of Theoretical Biology 299 2012 172 179 10.1016/j.jtbi.2011.03.004
29. S. Reimers The BBC internet study: General methodology Archives of Sexual Behavior 36 2 2007 147 161 10.1007/s10508-006-9143-2 (Pubitemid 46669186)
30. M. Rosenberg Society and the adolescent self-image 1965 Princeton University Press Princeton, NJ