A reliability analysis of Mechanical Turk data

Computers in Human Behavior

Volumn 43, Issue , 2015, Pages 304-307

  Rouse, Steven V ^a  

^a PEPPERDINE UNIVERSITY   (United States)

Author keywords

Psychometrics

Indexed keywords

COST EFFECTIVENESS; SURVEYS;

AMAZON'S MECHANICAL TURKS; BEST PRACTICES; DATA COLLECTION; MECHANICAL TURKS; PAYMENT RATES; PSYCHOMETRICS; RESEARCH DATA; RESEARCH NEEDS;

RELIABILITY ANALYSIS;

EID: 84912570663     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.chb.2014.11.004     Document Type: Article

References (19)

1. J.S. Adelman, and Z. Estes Emotion and memory: A recognition advantage for positive and negative words independent of arousal Cognition 129 2013 530 535 10.1016/j.cognition.2013.08.014
2. A.L. Alter, D.M. Oppenheimer, and J.C. Zemla Missing the trees for the forest: A construal level account of the illusion of explanatory depth Journal of Personality and Social Psychology 99 2010 436 451 10.1037/a0020218
3. American Psychological Association Ethical principles of psychologists and code of conduct 2010 American Psychological Association Washington, DC
4. M.H. Birnbaum Psychological experiments on the Internet 2000 Academic Press San Diego, CA
5. D.G. Bonett Sample size requirements for comparing two alpha coefficients Applied Psychological Measurement 27 2003 72 74 10.1177/0146621602239477
6. M. Buhrmester, T. Kwang, and S.D. Gosling Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science 6 2011 3 5 10.1177/1745691610393980
7. J.N. Butcher Computerized psychological assessment; A practitioner's guide 1987 Basic New York, NY
8. N.J. Castellan Jr. The modern minicomputer in laboratory automation American Psychologist 30 1975 205 211
9. R.M. Dailey, N. Brody, L. LeFebvre, and B. Crook Charting changes in commitment: Trajectories of on-again/off-again relationships Journal of Social and Personal Relationships 30 2013 1020 1044 10.1177/0265407513480284
10. K. Eriksson, and B. Simpson Emotional reactions to losing explain gender differences in entering a risky lottery Judgment and Decision Making 53 2010 159 163
11. L.R. Goldberg A broad-bandwidth, public-domain, personality inventory measuring the lower-level facets of several five-factor models I. Mervielde, I. Deary, F. De Fruyt, F. Ostendorf, Personality psychology in Europe Vol. 7 1999 Tilburg University Press Tilburg, The Netherlands 7 28
12. L.R. Goldberg, J.A. Johnson, H.W. Eber, R. Hogan, M.C. Ashton, and C.R. Cloninger The international personality item pool and the future of public-domain personality measures Journal of Research in Personality 40 2006 84 96
13. C.J. Holden, T. Dennie, and A.D. Hicks Assessing the reliability of the M5-120 on Amazon's Mechanical Turk Computers in Human Behavior 29 2013 1749 1754 10.1016/j.chb.2013.02.020
14. D.R. Johnson, and L.A. Borden Participants at your fingertips: Using Amazon's Mechanical Turk to increase student-faculty collaborative research Teaching of Psychology 39 2012 245 251 10.1177/0098628312456615
15. S. Kim, and L.S. Feldt A comparison of tests for equality of two or more independent alpha coefficients Journal of Educational Measurement 45 2008 179 193
16. W. Mason, and S. Suri Conducting behavioral research on Amazon's Mechanical Turk Behavioral Research 44 2012 1 23 10.3758/s13428-011-0124-6
17. G. Paolacci, and J. Chandler Inside the Turk: Understanding Mechanical Turk as a participant pool Current Directions in Psychological Science 23 2014 184 188 10.1177/0963721414531598
18. G. Paolacci, J. Chandler, and P.G. Ipeirotis Running experiments of Amazon Mechanical Turk Judgment and Decision Making 5 2010 411 419
19. S.S. Stevens Handbook of experimental psychology 1951 Wiley New York, NY