psiTurk: An open-source framework for conducting replicable behavioral experiments online

Behavior Research Methods

Volumn 48, Issue 3, 2016, Pages 829-842

  Gureckis, Todd M ^a   Martin, Jay ^a,b   McDonnell, John ^a,c   Rich, Alexander S ^a   Markant, Doug ^d   Coenen, Anna ^a   Halpern, David ^a   Hamrick, Jessica B ^e   Chan, Patricia ^a  

^a NEW YORK UNIVERSITY   (United States)

^b Stitch Fix Inc   (United States)

^c PM Squared Inc   (United States)

^d MAX PLANCK INSTITUTE FOR HUMAN DEVELOPMENT   (Germany)

^e UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Amazon Mechanical Turk; Internet experiments; Online experiments; Open science; psiTurk

Indexed keywords

BEHAVIORAL RESEARCH; CROWDSOURCING; INFORMATION PROCESSING; INTERNET; METHODOLOGY; PROCEDURES;

BEHAVIORAL RESEARCH; CROWDSOURCING; DATA COLLECTION; INTERNET; RESEARCH DESIGN;

EID: 84944691134     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-015-0642-8     Document Type: Article

References (12)

1. Birnbaum, M. H. (2004). Human research and data collection via the internet. Annual Review of Psychology, 55, 803–832.
2. Chandler, J., Mueller, P., & Paolacci, G. (2014). Nonnaïveté among amazon mechanical turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46(1), 112–130.
3. Chen, R., & Sakamoto, Y. (2013). Perspective matters: Sharing of crisis information in social media. In 46th Hawaii 46th Hawaii International conference on systems sciences(pp. 2033–2041). IEEE.
4. Crump, M. J., McDonnell, J.V., & Gureckis, T.M. (2013). Evaluating amazon’s mechanical turk as a tool for experimental behavioral research. PloS One, 8(3), e57, 410.
5. Kramer, A., Guillory, J., & Hancock, J. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences of the United States of America, 111(24), 8788–8790.
6. Mason, W., & Suri, S. (2012). A guide to behavioral experiments on mechanical turk. Behavior Research Methods, 44(1), 1–23.
7. Open Science Collaboration (2012). An open, large-scale collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7, 657–660.
8. Raymond, E. (1999). The Cathedral and the Bazaar.: O’Reilly.
9. Reips, U.D. (2000). The web experiment method: Advantages, disadvantages, and solutions. Psychological Experiments on the Internet, 89–117.
10. Reips, U. D. (2002). Standards for internet-based experimenting. Experimental Psychology, 49(4), 243–256.
11. Reips, U. D., & Neuhaus, C. (2002). Wextor: A web-based tool for generating and visualizing experimental designs and procedures. Behavior Research Methods Instruments, & Computers, 34(2), 234–240.
12. Wu, S., Hofman, J., Mason, W., & Watts, D. (2011). Who says what to whom on twitter. In International conference on the world wide web (WWW). Hyderabad, India.