TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences

Behavior Research Methods

Volumn 49, Issue 2, 2017, Pages 433-442

  Litman, Leib ^a   Robinson, Jonathan ^a   Abberbock, Tzvi ^a  

^a TOURO COLLEGE   (United States)

Author keywords

Crowdsourcing; Mechanical Turk; Online research

Indexed keywords

BEHAVIORAL SCIENCE; CROWDSOURCING; E-MAIL; HUMAN; INFORMATION PROCESSING; INTERNET; LONGITUDINAL STUDY; MONITORING; RUNNING; SCIENTIST; VELOCITY; BEHAVIORAL RESEARCH; PROCEDURES; SOFTWARE;

BEHAVIORAL RESEARCH; CROWDSOURCING; DATA COLLECTION; HUMANS; INTERNET; SOFTWARE;

EID: 84963680162     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-016-0727-z     Document Type: Article

References (21)

1. Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20, 351–368. doi:10.1093/pan/mpr057
2. Buccafusco, C., Burns, Z. C., Fromer, J. C., & Sprigman, C. J. (2014). Experimental tests of intellectual property laws’ creativity thresholds. Texas Law Review, 93, 1921–1980.
3. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3–5. doi:10.1177/1745691610393980
4. Chandler, J., & Mueller, P. (2013). Methodological concerns and advanced uses of crowdsourcing in psychological research. Manuscript submitted for publication.
5. Chandler, J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A. (2015). Using nonnaive participants can reduce effect sizes. Psychological Science, 26, 1131–1139. doi:10.1177/0956797615585115
6. Chilton, L. B., Horton, J. J., Miller, R. C., & Azenkot, S. (2010). Task search in a human computation market. In Proceedings of the ACM SIGKDD Workshop on Human Computation (pp. 1–9). New York, NY: ACM Press.
7. Eysenbach, G. (2004). Improving the quality of Web surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES). Journal of Medical Internet Research, 6, e34.
8. Gosling, S. D., & Johnson, J. A. (2010). Advanced methods for conducting online behavioral research. Washington, DC: American Psychological Association.
9. Gureckis, T. M., Martin, J., McDonnell, J., Rich, A. S., Markant D., Coenen, A.,.. Chan, P. (2015). psiTurk: An open-source framework for conducting replicable behavioral experiments online. Behavior Research Methods. Advance online publication. doi:10.3758/s13428-015-0642-8
10. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33, 61–83. doi:10.1017/S0140525X0999152X. disc. 83–135.
11. Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: Report of Board of Scientific Affairs’ Advisory Group on the Conduct of Research on the Internet. American Psychologist, 59, 105–117. doi:10.1037/0003-066x.59.2.105
12. Litman, L., Robinson, J., & Rosenzweig, C. (2015). The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk. Behavior Research Methods, 47, 519–528. doi:10.3758/s13428-014-0483-x
13. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1–23. doi:10.3758/s13428-011-0124-6
14. Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics: Theory, Research, and Practice, 6, 101–115. doi:10.1037/1089-2699.6.1.101
15. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5, 411–419.
16. Peer, E., Paolacci, G., Chandler, J., & Mueller, P. (2012). Selectively recruiting participants from Amazon Mechanical Turk using Qualtrics. Unpublished paper. Retrieved from ssrn.com/abstract=2100631
17. Peer, E., Vosgerau, J., & Acquisti, A. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46, 1023–1031. doi:10.3758/s13428-013-0434-y
18. Rosenzweig, C., Litman, L., & Robinson, J. (2015, May). Getting representative data on MTurk? Symposium presented at the meeting of the Association for Psychological Science, New York, NY.
19. Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1, 213–220. doi:10.1177/2167702612469015
20. Sprouse, J. (2011). A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory. Behavior Research Methods, 43, 155–167. doi:10.3758/s13428-010-0039-7
21. Von Emden, R. (2015, July 15). Exploratory TurkPrime study: Windfall bias—Young versus old [Web log post]. Retrieved from www.pavlov.io/2015/07/15/the-effects-of-age-on-the-windfall-bias/