Online psychophysics: reaction time effects in cognitive experiments

Behavior Research Methods

Volumn 49, Issue 4, 2017, Pages 1241-1260

  Semmelmann, Kilian ^a   Weigelt, Sarah ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

Cognitive psychology; Online study; Psychophysics; Reaction time; Replication; Web technology

Indexed keywords

ATTENTIONAL BLINK; COMPUTER PROGRAM; EXPERIMENTAL DESIGN; HUMAN; HUMAN EXPERIMENT; POTENTIAL DIFFERENCE; PSYCHOLOGY; PSYCHOPHYSICS; REACTION TIME; COMPARATIVE STUDY; INTERNET; PROCEDURES;

HUMANS; INTERNET; PSYCHOLOGY, EXPERIMENTAL; PSYCHOPHYSICS; REACTION TIME;

EID: 84982931958     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-016-0783-4     Document Type: Article

References (39)

1. Barnhoorn, J. S., Haasnoot, E., Bocanegra, B. R., & van Steenbergen, H. (2014). QRTEngine: An easy solution for running online reaction time experiments using qualtrics. Behavior Research Methods. doi:10.3758/s13428-014-0530-7
2. Birnbaum, M. H. (2000). Introduction to psychological experiments on the internet. In M. H. Birnbaum (Ed.), Psychological Experiments on the Internet (pp. xv – xx). Academic Press. doi: 10.1016/B978-012099980-4/50001-0
3. Birnbaum, M. H. (2004). Human research and data collection via the internet. Annual Review of Psychology, 55, 803–32. doi:10.1146/annurev.psych.55.090902.141601
4. Brainard, D. (1997). The psychophysics toolbox. Spatial Vision, 433–436. doi: 10.1163/156856897X00357
5. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s mechanical turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5. doi:10.1177/1745691610393980
6. Cohen, J., Collins, R., Darkes, J., & Gwartney, D. (2007). A league of their own: demographics, motivations and patterns of use of 1,955 male adult non-medical anabolic steroid users in the United States. Journal of the International Society of Sports Nutrition, 4(1), 12. doi:10.1186/1550-2783-4-12
7. Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s mechanical Turk as a tool for experimental behavioral research. PloS One, 8(3), e57410. doi:10.1371/journal.pone.0057410
8. de Leeuw, J. R., & Motz, B. a. (2015). Psychophysics in a Web browser? comparing response times collected with JavaScript and Psychophysics Toolbox in a visual search task. Behavior Research Methods, (2014). doi: 10.3758/s13428-015-0567-2
9. Eimer, M., & Schlaghecken, F. (2002). Links between conscious awareness and response inhibition: Evidence from masked priming. Psychonomic Bulletin & Review, 9(3), 514–520. doi:10.3758/BF03196307
10. Elze, T., & Tanner, T. G. (2012). Temporal properties of liquid crystal displays: Implications for vision science experiments. PloS One, 7(9), e44048. doi:10.1371/journal.pone.0044048
11. Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16(1), 143–149.
12. Forster, K. I., & Davis, C. (1984). Repetition priming and frequency attenuation in lexical access. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(4), 680–698. doi:10.1037/0278-7393.10.4.680
13. Germine, L., Nakayama, K., Duchaine, B. C., Chabris, C. F., Chatterjee, G., & Wilmer, J. B. (2012). Is the Web as good as the lab? comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic Bulletin & Review, 19(5), 847–57. doi:10.3758/s13423-012-0296-9
14. Göritz, A. S. (2006). Incentives in Web Studies: Methodological Issues and a Review, 1(1), 58–70.
15. Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2000). Should we trust web-based studies? a comparative analysis of six preconceptions about internet questionnaires. The American Psychologist, 59(2), 93–104. doi:10.1037/0003-066X.59.2.93
16. Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015). The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, 10(5), 479–491. doi:10.1017/CBO9781107415324.004
17. Hilbig, B. E. (2015). Reaction time effects in lab- versus Web-based research: Experimental evidence. Behavior Research Methods. doi:10.3758/s13428-015-0678-9
18. Jarosz, A. F., & Wiley, J. (2014). What are the odds? a practical guide to computing and reporting Bayes Factors. The Journal of Problem Solving, 7, 2–9. doi:10.7771/1932-6246.1167
19. Joinson, A. N. (2001). Self-disclosure in computer-mediated communication: The role of self-awareness and visual anonymity. European Journal of Social Psychology, 31(2), 177–192. doi:10.1002/ejsp.36
20. MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109(2), 163–203. doi:10.1037//0033-2909.109.2.163
21. Meyer, D. E., & Schvaneveldt, R. W. (1971). Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations. Journal of Experimental Psychology, 90(2), 227–234. doi:10.1037/h0031564
22. Neath, I., Earle, A., Hallett, D., & Surprenant, A. M. (2011). Response time accuracy in Apple Macintosh computers. Behavior Research Methods, 43(2), 353–62. doi:10.3758/s13428-011-0069-9
23. Plant, R. R., & Turner, G. (2009). Millisecond precision psychological research in a world of commodity computers: New hardware, new problems? Behavior Research Methods, 41(3), 598–614. doi:10.3758/BRM.41.3.598
24. Ramsey, J. D., & Morrissey, S. J. (1978). Isodecrement curves for task performance in hot environments. Applied Ergonomics, 9(2), 66–72. doi:10.1016/0003-6870(78)90150-3
25. Raymond, J., Shapiro, K., & Arnell, K. (1992). Temporary suppression of visual processing in an RSVP task: An attentional blink? Journal of Experimental …. Retrieved from http://psycnet.apa.org/journals/xhp/18/3/849/
26. Reimers, S. (2007). The BBC internet study: General methodology. Archives of Sexual Behavior, 36(2), 147–61. doi:10.1007/s10508-006-9143-2
27. Reimers, S., & Stewart, N. (2007). Adobe flash as a medium for online experimentation: A test of reaction time measurement capabilities. Behavior Research Methods, 39, 365–370. doi:10.3758/BF03193004
28. Reimers, S., & Stewart, N. (2015). Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments. Behavior Research Methods, 47(2), 309–327. doi:10.3758/s13428-014-0471-1
29. Reips, U.-D. (2000). The web experiment: Advantages, disadvantages, and solutions. Psychology Experiments on the Internet, 89–117.
30. Sanders, A. F., & Lamers, J. M. (2002). The Eriksen flanker effect revisited. Acta Psychologica, 109, 41–56.
31. Schmidt, W. C. (2001). Presentation accuracy of Web animation methods. Behavior Research Methods, Instruments, & Computers: A Journal of the Psychonomic Society, Inc, 33(2), 187–200. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11447672
32. Schubert, T. W., Murteira, C., Collins, E. C., & Lopes, D. (2013). ScriptingRT: A software library for collecting response latencies in online studies of cognition. PloS One, 8(6), e67769. doi:10.1371/journal.pone.0067769
33. Shapiro, K. L., Raymond, J. E., & Arnell, K. M. (1997). The attentional blink. Trends in Cognitive Sciences. doi:10.1016/S1364-6613(97)01094-2
34. Skitka, L. J., & Sargis, E. G. (2006). The internet as psychological laboratory. Annual Review of Psychology, 57, 529–55. doi:10.1146/annurev.psych.57.102904.190048
35. Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D.M., Newell, B. R., Paolacci, G., et al. (2015). The average labo-ratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, 10(5), 479–491. doi:10.1017/CBO9781107415324.004
36. Stroop, J. R. (1935). Stroop color word test. Journal of Experimental Physiology (18), 643–662. doi: 10.1007/978-0-387-79948-3
37. Whelan, R. (2008). Effective analysis of reaction time data. The Psychological Record, 58, 475–482.
38. Windes, J. D. (1968). Reaction time for numerical coding and naming of numerals. Journal of Experimental Psychology, 78(2), 318–22. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/5722448
39. Wolfe, J. M. (1998). Visual search. Attention, Perception, & Psychophysics, 20, 13–73. doi:10.1016/j.tics.2010.12.001