Is mechanical turk the answer to our sampling woes?

Industrial and Organizational Psychology

Volumn 9, Issue 1, 2016, Pages 162-167

  Keith, Melissa G ^a   Harms, Peter D ^b  

^a PURDUE UNIVERSITY   (United States)

^b UNIVERSITY OF ALABAMA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84966697349     PISSN: 17549426     EISSN: 17549434     Source Type: Journal    
DOI: 10.1017/iop.2015.130     Document Type: Note

References (15)

1. Barger, P., Behrend, T. S., Sharek, D. J., & Sinar, E. F. (2011). I-O and the crowd: Frequently asked questions about using Mechanical Turk for research. The Industrial-Organizational Psychologist, 49(2), 11-17.
2. Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011). The viability of crowdsourcing for survey research. Behavioral Research Methods, 43(3), 1-14.
3. Bergman, M. E., & Jean, V. A. (2016). Where have all the "workers" gone? A critical analysis of the unrepresentativeness of our samples relative to the labor market in the industrial-organizational psychology literature. Industrial and Organizational Psychology: Perspectives on Science and Practice, 9, 84-113.
4. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5.
5. Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. Computers and Human Behavior, 29, 2156-2160.
6. Chandler, J., Mueller, P., & Paolacci, G. (2014). Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavioral Research, 46, 112-130. doi:10.3758/s13428-013-0365-7
7. Downs, J. S., Holbrook, M. B., Sheng, S., & Cranor, L. F. (2010). Are your participants gaming the system? Screening Mechanical Turk workers. In Proceedings from SIGCHI '10: The 28th International Conference on Human Factors in Computing Systems (pp. 2399-2402). New York, NY: ACM Press.
8. Feitosa, J., Joseph, D. L., & Newman, D. A. (2015). Crowdsourcing and personality measurement equivalence: A warning about countries whose primary language is not English. Personality and Individual Differences, 75, 47-52.
9. Harms, P. D., & DeSimone, J. A. (2015). Caution! MTurk workers ahead-Fines doubled. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8(2), 183-190.
10. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2/3), 61-83.
11. Ipeirotis, P. G. (2010). Demographics of Mechanical Turk (Technical Report CeDER-10-01). New York: New York University.
12. Litman, L., Robinson, J., & Rosenzweig, C. (2015). The relationship between motivation, monetary compensation and data quality among US- and India-based workers on Mechanical Turk. Behavioral Research, 47, 519-528.
13. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), 411-419.
14. Ross, J., Zaldivar, A., Irani, L., & Tomlinson, B. (2010). Who are the crowdworkers? Shifting demographics in Mechanical Turk. In Proceedings from CHI'10: Extended Abstracts on Human Factors in Computing Systems (pp. 2863-2872). Atlanta, GA: ACM Press.
15. Woo, S. E., Keith, M., & Thornton, M. A. (2015). Amazon Mechanical Turk for industrial and organizational psychology: Advantages, challenges, and practical recommendations. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8(2), 171-179.