Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing

Computers in Human Behavior

Volumn 29, Issue 6, 2013, Pages 2156-2160

  Casler, Krista ^a   Bickel, Lydia ^a   Hackett, Elizabeth ^a  

^a FRANKLIN AND MARSHALL COLLEGE   (United States)

Author keywords

Crowd sourcing; Methodology; MTurk; Recruitment; Social media

Indexed keywords

CROWD-SOURCING; METHODOLOGY; MTURK; RECRUITMENT; SOCIAL MEDIA;

BEHAVIORAL RESEARCH; STUDENTS;

SOCIAL NETWORKING (ONLINE);

EID: 84878584198     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.chb.2013.05.009     Document Type: Article

References (22)

1. A. Berinsky, G. Huber, and G. Lenz Evaluating online labor markets for experimental research: Amazon.com's Mechanical Turk Political Analysis 20 2012 351 368 10.1093/pan/mpr057
2. M. Buhrmester, T. Kwang, and S. Gosling Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science 6 2011 3 5 10.1177/1745691610393980
3. Casler, K.; Hackett, E.; & Bickel, L. (2013). Perceiving parts and fixing functions: Why an object's function is less than the sum of its parts, submitted for publication.
4. E. Davidov, and F. Depner Testing for measurement equivalence of human values across online and paper-and-pencil surveys Quality and Quantity: International Journal of Methodology 45 2011 375 390 10.1007/s11135-009-9297-9
5. K. Eriksson, and B. Simpson Emotional reactions to losing explain gender differences in entering a risky lottery Judgment and Decision Making 5 2010 159 163
6. R. Gardner, D. Brown, and R. Boice Using Amazon's Mechanical Turk website to measure accuracy of body size estimation and body dissatisfaction Body Image 9 2012 532 534
7. S. Gosling, S. Vazire, S. Srivastava, and O. John Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires American Psychologist 59 2 2004 93 104 10.1037/0003-066X.59.2.93 (Pubitemid 38318025)
8. J. Henrich, S. Heine, and A. Norenzayan The weirdest people in the world? Behavioral and Brain Sciences 33 2010 61 83 10.1017/S0140525X0999152X
9. P.J. Henry College sophomores in the laboratory redux: Influences of a narrow data base on social psychology's view of the nature of prejudice Psychological Inquiry 19 2 2008 49 71 10.1080/10478400802049936
10. J. Horton, D. Rand, and R. Zeckhauser The online laboratory: Conducting experiments in a real labor market Experimental Economics 14 2011 399 425 10.1007/s10683-011-9273-9
11. A. Joinson Social desirability, anonymity, and internet-based questionnaires Behavior Research Methods, Instruments, and Computers 31 1999 433 438 10.3758/BF03200723
12. A. Joinson, A. Woodley, and U. Reips Personalization, authentication and self-disclosure in self-administered Internet surveys Computers in Human Behavior 23 2007 275 285 10.1016/j.chb.2004.10.012 (Pubitemid 44278494)
13. R. Kraut, V. Patterson, M. Lundmark, S. Kiesler, T. Mukophadhyay, and W. Scherlis Internet paradox: A social technology that reduces social involvement and psychological well-being? American Psychologist 53 1998 1017 1031 (Pubitemid 128461524)
14. Mason, W.; & Watts, D. (2009). Financial incentives and the "performance of crowds". In: HCOMP'09: Proceedings of the ACM SIGKDD Workshop on Human Computation, (pp. 77-85).
15. W. Mason, and S. Suri Conducting behavioral research on Amazon's Mechanical Turk Behavior Research Methods 44 2012 1 23
16. D. Oppenheimer, T. Meyvis, and N. Davidenko Instructional manipulation checks: Detecting satisficing to increase statistical power Journal of Experimental Social Psychology 45 2009 867 872 10.1016/j.jesp.2009.03.009
17. G. Paolacci, J. Chandler, and P. Ipeirotis Running experiments on Amazon Mechanical Turk Judgment and Decision Making 5 2010 411 419
18. D. Sears College sophomores in the lab: Influences of a narrow data base on social psychology's view of human nature Journal of Personality and Social Psychology 51 1986 515 530 10.1037/0022-3514.51.3.515
19. Z. Simsek, and J. Viega A primer on internet organizational surveys Organizational Research Methods 4 2001 218 235 10.1177/109442810143003 (Pubitemid 33674906)
20. Smyth, J.; & Pearson, J. (2011). Internet survey methods: A review of strengths, weaknesses, and innovations. In: M. Das, P. Ester, & L. Kaczmirek (Eds.), Social and behavioral research and the Internet: Advances in applied methods and, research strategies (pp. 11-44).
21. S. Suri, and D. Watts Cooperation and contagion in web0based, networked public goods experiments PLoS ONE 6 3 2011 e16836 doi: 10.1371/journal.pone. 0016836
22. T. Tuten, D. Urban, and M. Bosnjak Internet surveys and data quality: A review B. Batinik, U. Reips, M. Bosnjak, Online social sciences 2002 Hogrefe & Huber Publishers Ashland, OH, US 7 26