The Demographic and Political Composition of Mechanical Turk Samples

SAGE Open

Volumn 6, Issue 1, 2016, Pages

  Levay, Kevin E ^a   Freese, Jeremy ^b   Druckman, James N ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Amazon s Mechanical Turk; public opinion; research methods; survey sampling

Indexed keywords

EID: 84976514572     PISSN: None     EISSN: 21582440     Source Type: Journal    
DOI: 10.1177/2158244016636433     Document Type: Article

References (25)

1. K.Arceneaux (2012). Cognitive biases and the strength of political arguments. American Journal of Political Science, 56, 271-285.
2. R.BakerS.J.BlumbergJ.M.BrickM.P.CouperM.CourtrightJ.M.DennisD.. Zahs (2010). Research synthesis: AAPOR report on online panels. Public Opinion Quarterly, 74, 711-781.
3. A.J.BerinskyG.A.HuberG.S.Lenz (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20, 351-368.
4. J.Bohannon (2011). Social science for pennies. Science, 334, 307.
5. M.BuhrmesterT.KwangS.D.Gosling (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science, 6, 3-5.
6. M.CallegaroR.BakerJ.BethlehemA.S.GöritzJ.A.KrosnickP.J.Lavrakas (2014). Online panel research: History, concepts, applications, and a look at the future. In (Eds.), Online panel research: A data quality perspective (pp. 1-22). West Sussex, UK: John Wiley.
7. K.CaslerL.BickelE.Hackett (2013). Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29, 2156-2160.
8. M.P.Couper (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464-494.
9. M.P.Couper (2011). The future of modes of data collection. Public Opinion Quarterly, 75, 889-908.
10. A.GelmanD.Rothschild (2014, August4). Modern polling needs innovation, not traditionalism. The Monkey Cage.Retrieved from https://www.washingtonpost.com/news/monkey-cage/wp/2014/08/04/modern-polling-requires-both-sampling-and-adjustment/
11. A.S.GerberG.A.HuberD.DohertyC.M.Dowling (2011). Citizens’ policy confidence and electoral punishment: A neglected dimension of electoral accountability. Journal of Politics, 73, 1206-1224.
12. G.A.HuberS.J.HillG.S.Lenz (2012). Sources of bias in retrospective decision making: Experimental evidence on voters’ limitations in controlling incumbents. American Political Science Review, 106, 720-741.
13. G.A.HuberC.Paris (2013). Assessing the programmatic equivalence assumption in question wording experiments: Understanding why Americans like assistance to the poor more than welfare. Public Opinion Quarterly, 77, 385-397.
14. C.HuffD.Tingley (2015). “Who are these people?”: Evaluating the demographic characteristics and political preferences of MTurk survey respondents. Research and Politics, 2, 1-12.
15. P.G.Ipeirotis (2010). Demographics of Mechanical Turk (CeDER Working Paper No. 10-01). New York University. Retrieved from http://www.ipeirotis.com/wp-content/uploads/2012/02/CeDER-10-01.pdf
16. Y.KrupnikovA.S.Levine (2014). Cross-sample comparisons and external validity. Journal of Experimental Political Science, 1, 59-80.
17. T.J.Leeper (2013). Crowdsourcing with R fand the MTurk API. The Political Methodologist, 2, 2-7.
18. M.LiuY.Wang (2015). Data collection mode effect on feeling thermometer questions: A comparison of face-to-face and web surveys. Computers in Human Behavior, 48, 212-218.
19. W.A.MasonD.J.Watts (2009). Financial incentives and the “performance of crowds.”Association for Computing Machinery Explorations Newsletter, 11, 100-108.
20. K.J.MullinixT.J.LeeperJ.FreeseJ.N.Druckman (2015). The generalizability of survey experiments. Journal of Experimental Political Science, 2, 109-138.
21. D.C.Mutz (2011). Population-based survey experiments. Princeton, NJ: Princeton University Press.
22. G.PaolacciJ.Chandler (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23, 184-188.
23. G.PaolacciJ.ChandlerP.G.Ipeirotis (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5, 411-420.
24. K.L.SchlozmanS.VerbaH.E.Brady (2012). The unheavenly chorus: Unequal voice and the broken promise of American democracy. Princeton, NJ: Princeton University Press.
25. B.StruminskayaL.KaczmirekI.SchaurerW.Bandilla (2014). Assessing representativeness of a probability-based online panel in Germany. In (Eds.), Online panel research: A data quality perspective (pp. 61-85). West Sussex, UK: John Wiley.