Online versus offline: The Web as a medium for response time data collection

Behavior Research Methods

Volumn 48, Issue 3, 2016, Pages 1086-1099

  Chetverikov, Andrey ^a,b   Upravitelev, Philipp ^c  

^a ST PETERSBURG STATE UNIVERSITY   (Russian Federation)

^b RUSSIAN PRESIDENTIAL ACADEMY OF NATIONAL ECONOMY AND PUBLIC ADMINISTRATION   (Russian Federation)

^c Sigma Plus   (Russian Federation)

Author keywords

JavaScript; Online; Reaction times; Response times; Task difficulty; Visual search; Web

Indexed keywords

ADULT; COMPUTER LANGUAGE; FEMALE; HUMAN; INFORMATION PROCESSING; INTERNET; MALE; ONLINE SYSTEM; PROCEDURES; PSYCHOMOTOR PERFORMANCE; REACTION TIME; SAMPLE SIZE; SOFTWARE; YOUNG ADULT;

ADULT; DATA COLLECTION; FEMALE; HUMANS; INTERNET; MALE; ONLINE SYSTEMS; PROGRAMMING LANGUAGES; PSYCHOMOTOR PERFORMANCE; REACTION TIME; SAMPLE SIZE; SOFTWARE; YOUNG ADULT;

EID: 84984628071     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-015-0632-x     Document Type: Article

References (24)

1. Baayen, R. H., Davidson, D. J., & Bates, D. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. doi:10.1016/j.jml.2007.12.005
2. Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278. doi:10.1016/j.jml.2012.11.001
3. Bates, D., Maechler, M., Bolker, B., & Walker, S. (2013). lme4: Linear mixed-effects models using Eigen and S4. R package version 1.0–4. Retrieved from http://cran.r-project.org/package=lme4
4. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57, 289–300.
5. Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorials in Quantitative Methods for Psychology, 1, 42–45.
6. Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS ONE, 8, e57410. doi:10.1371/journal.pone.0057410
7. Damian, M. F. (2010). Does variability in human performance outweigh imprecision in response devices such as computer keyboards? Behavior Research Methods, 42, 205–211. doi:10.3758/BRM.42.1.205
8. de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 47, 1–12. doi:10.3758/s13428-014-0458-y
9. Garaizar, P., & Vadillo, M. A. (2014). Accuracy and precision of visual stimulus timing in psychopy: No timing errors in standard usage. PLoS ONE, 9(e112033), 1–8. doi:10.1371/journal.pone.0112033
10. Keller, F., Gunasekharan, S., Mayo, N., & Corley, M. (2009). Timing accuracy of Web experiments: A case study using the WebExp software package. Behavior Research Methods, 41, 1–12. doi:10.3758/BRM.41.1.12
11. Kristjánsson, Á., & Jóhannesson, Ó. I. (2014). How priming in visual search affects response time distributions: Analyses with ex-Gaussian fits. Attention, Perception, & Psychophysics, 76, 2199–2211. doi:10.3758/s13414-014-0735-y
12. McDonnell, J. V., Martin, J. B., Markant, D. B., Coenen, A., Rich, A. S., & Gureckis, T. M. (2014). psiTurk. New York, NY: New York University.
13. Morey, R. D. (2008). Confidence intervals from normalized data: A correction to Cousineau (2005). Tutorials in Quantitative Methods for Psychology, 4, 61–64.
14. Neath, I., Earle, A., Hallett, D., & Surprenant, A. M. (2011). Response time accuracy in Apple Macintosh computers. Behavior Research Methods, 43, 353–362. doi:10.3758/s13428-011-0069-9
15. Palmer, E. M., Horowitz, T. S., Torralba, A., & Wolfe, J. M. (2011). What are the shapes of response time distributions in visual search? Journal of Experimental Psychology: Human Perception and Performance, 37, 58–71. doi:10.1037/a0020747
16. Peirce, J. W. (2007). PsychoPy—Psychophysics software in Python. Journal of Neuroscience Methods, 162, 8–13. doi:10.1016/j.jneumeth.2006.11.017
17. Plant, R. R., & Quinlan, P. T. (2013). Could millisecond timing errors in commonly used equipment be a cause of replication failure in some neuroscience studies? Cognitive, Affective, & Behavioral Neuroscience, 13, 598–614. doi:10.3758/s13415-013-0166-6
18. Reimers, S., & Maylor, E. A. (2005). Task switching across the life span: Effects of age on general and specific switch costs. Developmental Psychology, 41, 661–671. doi:10.1037/0012-1649.41.4.661
19. Reimers, S., & Stewart, N. (2007). Adobe Flash as a medium for online experimentation: A test of reaction time measurement capabilities. Behavior Research Methods, 39, 365–370. doi:10.3758/BF03193004
20. Reimers, S., & Stewart, N. (2015). Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments. Behavior Research Methods, 47, 309–327. doi:10.3758/s13428-014-0471-1
21. Rigby, R. A., & Stasinopoulos, D. M. (2005). Generalized additive models for location, scale and shape (with discussion). Journal of the Royal Statistical Society, Series C, 54, 507–554. doi:10.1111/j.1467-9876.2005.00510.x
22. Rousseeuw, P., Croux, C., Todorov, V., Ruckstuhl, A., Salibian-Barrera, M., Verbeke, T.,.. Maechler, M. (2015). robustbase: Basic robust statistics [Software]. Retrieved from http://cran.r-project.org/web/packages/robustbase/index.html
23. Schubert, T. W., Murteira, C., Collins, E. C., & Lopes, D. (2013). ScriptingRT: A software library for collecting response latencies in online studies of cognition. PLoS ONE, 8, e67769. doi:10.1371/journal.pone.0067769
24. Simcox, T., & Fiez, J. A. (2014). Collecting response times using Amazon Mechanical Turk and Adobe Flash. Behavior Research Methods, 46, 95–111. doi:10.3758/s13428-013-0345-y