A reminder on millisecond timing accuracy and potential replication failure in computer-based psychology experiments: An open letter

Behavior Research Methods

Volumn 48, Issue 1, 2016, Pages 408-411

  Plant, Richard R ^a  

^a The Black Box ToolKit Ltd ^*  (United Kingdom)

Author keywords

Computer based experiment; Experiment generator; Millisecond timing; Replication failure; Timing error

Indexed keywords

COMPUTER; HUMAN; PROCEDURES; PSYCHOLOGY; REPRODUCIBILITY; RESEARCH; SOFTWARE; TIME FACTOR;

COMPUTERS; HUMANS; PSYCHOLOGY; PSYCHOLOGY, EXPERIMENTAL; REPRODUCIBILITY OF RESULTS; RESEARCH; SOFTWARE; TIME FACTORS;

EID: 84924565111     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-015-0577-0     Document Type: Article

References (16)

1. Barnhoorn, Jonathan, S., Haasnoot, E., Bocanegra, B. R., & van Steenbergen, H. (2014). QRTEngine: An easy solution for running online reaction time experiments using Qualtrics. Behavior Research Methods, November 2014. doi:10.3758/s13428-014-0530-7
2. Crosse M.J. & Lalor E.C. (2014). The cortical representation of the speech envelope is earlier for audiovisual speech than audio speech. Journal of Neurophysiology 111: 1400–1408. http://dx.doi.org/10.1152/jn.00690.2013/Retractionhttp://dx.doi.org/10.1152/jn.z9k-2710-retr.2014
3. Garaizar, P., & Vadillo, M. A. (2014). Accuracy and Precision of Visual Stimulus Timing in PsychoPy: No Timing Errors in Standard Usage. PLoS ONE, 9(11), e112033. doi:10.1371/journal.pone.0112033
4. Garaizar, P., Vadillo, M. A., & López-de-Ipiña, D. (2014). Presentation Accuracy of the Web Revisited: Animation Methods in the HTML5 Era. PLoS ONE, 9(10), e109812. doi:10.1371/journal.pone.0109812
5. Pashler, H., & Wagenmakers, E.-J. (2012). Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7, 528–530. doi:10.1177/1745691612465253
6. Plant, R. R. (2014). Quick, quick, slow: Timing inaccuracy in computer-based studies means we may need to make use of external chronometry to guarantee our ability to replicate. Paper presented at the 44th Annual Meeting of the Society for Computers in Psychology (SCiP), Long Beach, California, November 20.
7. Plant, R., & Hammond, N. V. (2001). Benchmarking the timing characteristics of tools used by behavioural scientists. Abstracts of the Psychonomic Society (42nd Annual Meeting), 6, 109.
8. Plant, R., & Hammond, N. V. (2002). Towards an Experimental Timing Standards Lab: Benchmarking precision in the real world. Behavior Research Methods, Instruments, and Computers, 34, 218–226.
9. Plant, R., Hammond, N. V., & Whitehouse, T. (2003). How choice of mouse may effect response timing in psychological studies. Behavior Research Methods, Instruments and Computers, 35, 276–284.
10. Plant, R. R., & Quinlan, P. T. (2013). Could millisecond timing errors in commonly used equipment be a cause of replication failure in some neuroscience studies? Cognitive, Affective, & Behavioral Neuroscience, 13(3), 598–614. doi:10.3758/s13415-013-0166-6
11. Plant, R., & Turner, G. (2004). Self-validating presentation and response timing in cognitive paradigms: How and why? Behavior Research Methods, Instruments and Computers, 36, 291–303.
12. Plant, R., & Turner, G. (2009). Millisecond precision psychological research in a world of commodity computers: New hardware, new problems? Behavior Research Methods, 41, 598–614. doi:10.3758/BRM.41.3.598
13. Sperdin, H. F., Spierer, L., Becker, R., Michel, C. M., & Landis, T. (2014). Submillisecond unmasked subliminal visual stimuli evoke electrical brain responses. Human Brain Mapping. doi:10.1002/hbm.22716
14. Thurgood, C., Whitfield, T. W., & Patterson, J. (2011). Towards a visual recognition threshold: New instrument shows humans identify animals with only 1ms of visual exposure. Vision Research, 51, 1966–1971. doi:10.1016/j.visres.2011.07.008
15. Verhagen, A. J., & Wagenmakers, E.-J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 143, 1457–1475. doi:10.1037/a0036731
16. Zuccolotto, A., Babjack, D., Cernicky, B., Sobotka, S. S., Basler, L. & Struthers, D. (2014). Methods for assessing and standardizing audio stimulus presentation latencies across heterogeneous hardware and operating system platforms. Poster presented at the 44th Annual Meeting of the Society for Computers in Psychology (SCiP), Long Beach, California, November 20.