The experiment factory: Standardizing behavioral experiments

Frontiers in Psychology

Volumn 7, Issue APR, 2016, Pages

  Sochat, Vanessa V ^a   Eisenberg, Ian W ^a   Enkavi, A Zeynep ^a   Li, Jamie ^a   Bissett, Patrick G ^a   Poldrack, Russell A ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Assessment; Behavior; Docker; Experiments; Reproducibility; Web experiments

Indexed keywords

EID: 84974603456     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2016.00610     Document Type: Article

References (15)

1. Baker, M. (2015). Over half of psychology studies fail reproducibility test. Nat. News. doi: 10.1038/nature.2015.18248 Available online at: http://www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.18248
2. Brainard, D. H. (1997). The psychophysics toolbox. Spat. Vis. 10, 433-436. doi: 10.1163/156856897X00357
3. de Leeuw, J. R. (2015). jspsych: a JavaScript library for creating behavioral experiments in a web browser. Behav. Res. Methods 47, 1-12. doi: 10.3758/s13428-014-0458-y
4. Klein, R. A., Ratliff, K. A., Vianello, M., R. B. A. Jr., Bahník, V., Bernstein, M. J., Bocian, K., et al. (2014). Investigating variation in replicability. Soc. Psychol. 45, 142-152. doi: 10.1027/1864-9335/a000178
5. Lange, K., Kühn, S., and Filevich, E. (2015). "Just another tool for online studies" (JATOS): an easy solution for setup and management of web servers supporting online studies. PLoS ONE 10:e0130834. doi: 10.1371/journal.pone.0130834
6. MacWhinney, B., St. James, J., Schunn, C., Li, P., and Schneider, W. (2001). STEP-A system for teaching experimental psychology using E-Prime. Behav. Res. Methods Instrum. Comput. 33, 287-296. doi: 10.3758/BF03195379
7. Makin, S. (2016). Brain training: memory games. Nature 531, S10-S11. doi: 10.1038/531S10a
8. Mason, W., and Suri, S. (2011). Conducting behavioral research on amazons mechanical turk. Behav. Res. 44, 1-23. doi: 10.3758/s13428-011-0124-6
9. McDonnell, J. V., Martin, J. B., Markant, D. B., Coenen, A., Rich, A. S., and Gureckis, T. M. (2012). Psiturk (Version 1.02)[Software]. New York, NY: New York University
10. Open Science Collaboration (2015). PSYCHOLOGY. estimating the reproducibility of psychological science. Science 349:aac4716. doi: 10.1126/science.aac4716
11. Peirce, J. W. (2007). PsychoPy-psychophysics software in python. J. Neurosci. Methods 162, 8-13. doi: 10.1016/j.jneumeth.2006.11.017
12. Poldrack, R. A., Kittur, A., Kalar, D., Miller, E., Seppa, C., Gil, Y., et al. (2011). The cognitive atlas: toward a knowledge foundation for cognitive neuroscience. Front. Neuroinform. 5:17. doi: 10.3389/fninf.2011.00017
13. Schneider, W., Eschman, A., and Zuccolotto, A. (2012). E-Prime User's Guide. Pittsburgh, PA: Psychology Software Tools, Inc
14. Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., et al. (2015). The average laboratory samples a population of 7,300 Amazon Mechanical Turk Workers. Judgm. Decis. Mak. 10, 479-491
15. Woods, A. T., Velasco, C., Levitan, C. A., Wan, X., and Spence, C. (2015). Conducting perception research over the internet: a tutorial review. PeerJ 3:e1058. doi: 10.7717/peerj.1058