Audio-visual speech timing sensitivity is enhanced in cluttered conditions

PLoS ONE

Volumn 6, Issue 4, 2011, Pages

  Roseboom, Warrick ^a   Nishida, Shin'ya ^b   Fujisaki, Waka ^c   Arnold, Derek H ^a  

^a UNIVERSITY OF QUEENSLAND   (Australia)

^b NTT Communication Science Laboratories   (Japan)

^c NATIONAL INSTITUTE OF ADVANCED INDUSTRIAL SCIENCE AND TECHNOLOGY AIST   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ARTICLE; AUDITORY DISCRIMINATION; AUDITORY STIMULATION; AUDOVISUAL SPEECH TIMING SENSITIVITY; CONTROLLED STUDY; HUMAN; HUMAN EXPERIMENT; SENSORY SYSTEM PARAMETERS; SPEECH ARTICULATION; TASK PERFORMANCE; VISUAL DISCRIMINATION; VISUAL STIMULATION; AUDIOVISUAL EQUIPMENT; PATHOPHYSIOLOGY; PHYSIOLOGY; SPEECH; SPEECH DISORDER; TIME;

ACOUSTIC STIMULATION; AUDIOVISUAL AIDS; HUMANS; SPEECH; SPEECH DISORDERS; TIME FACTORS;

EID: 79953734489     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0018309     Document Type: Article

References (51)

1. Arnold DH, Johnston A, Nishida S, (2005) Timing Sight and Sound. Vision Research 45: 1275-1284.
2. Spence C, Squire SB, (2003) Multisensory integration: Maintaining the perception of synchrony. Current Biology 13: R519-R521.
3. King AJ, (2005) Multisensory integration: strategies for synchronization. Current Biology 15: R339-41.
4. Dixon NF, Spitz L, (1980) The detection of auditory visual desynchrony. Perception 9: 719-721.
5. Miner N, Caudell T, (1998) Computational requirements and synchronization issues of virtual acoustic displays. Presence: Teleoperators and Virtual Environments 7: 396-409.
6. Grant KW, van Wassenhove V, Poeppel D, (2004) Detection of auditory (crossspectral) and auditory-visual (cross-modal) synchrony. Journal of the Acoustical Society of America 108: 1197-1208.
7. Navarra J, Vatakis A, Zampini M, Humphreys W, Soto-Faraco S, et al. (2005) Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Research 25: 499-507.
8. Vatakis A, Spence C, (2006) Audiovisual synchrony perception for music, speech, and object actions. Brain Research 1111: 134-142.
9. Vatakis A, Spence C, (2007) Crossmodal binding: Evaluating the "unity assumption" using audiovisual speech stimuli. Perception & Psychophysics 69: 744-56.
10. Fendrich R, Corballis PM, (2001) The temporal cross-capture of audition and vision. Perception & Psychophysics 63: 719-725.
11. Morein-Zamir S, Soto-Faraco S, Kingstone A, (2003) Auditory capture of vision: examining temporal ventriloquism. Cognitive Brain Research 17: 154-163.
12. Vroomen J, Keetels M, (2006) The spatial constraint in intersensory pairing: No role in temporal ventriloquism. Journal of Experimental Psychology: Human Perception and Performance 32: 1063-1071.
13. Jackson CV, (1953) Visual factors in auditory localization. Quarterly Journal of Experimental Psychology 5: 52-65.
14. Radeau M, Bertelson P, (1987) Auditory-visual interaction and the timing of inputs. Thomas (1941) revisited. Psychological Research 49: 17-22.
15. Welch RB, Warren DH, (1980) Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 88: 638-667.
16. Guski R, Troje NF, (2003) Audiovisual phenomenal causality. Perception & Psychophysics 65: 789-800.
17. Fujisaki W, Shimojo S, Kashino M, Nishida S, (2004) Recalibration of audiovisual simultaneity. Nature Neuroscience 7: 773-778.
18. Vroomen J, Keetels M, de Gelder B, Bertelson P, (2004) Recalibration of temporal order perception by exposure to audio-visual asynchrony. Cognitive Brain Research 22: 32-35.
19. Navarra J, Vatakis A, Zampini M, Soto-Faraco S, Humphreys W, et al. (2005) Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Research 25: 499-507.
20. Vatakis A, Navarra J, Soto-Faraco S, Spence C, (2007) Temporal recalibration during asynchronous audiovisual speech perception. Experimental Brain Research 181: 173-181.
21. Powers R, Hillock AR, Wallace MT, (2009) Perceptual Training Narrows the Temporal Window of Multisensory Binding. Journal of Neuroscience 29: 12265-12274.
22. Navarra J, Alsius A, Velasco I, Soto-Faraco S, Spence C, (2010) Perception of audiovisual speech synchrony for native and non-native language. Brain Research 1323: 84-93.
23. Lewald J, Ehrenstein WH, Guski R, (2001) Spatio-temporal constraints for auditory-visual integration. Behavioural Brain Research 121: 69-79.
24. Lewald J, Guski R, (2003) Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. Cognitive Brain Research 16 (3): 468-478.
25. Roseboom W, Nishida S, Arnold DH, (2009) The sliding window of audio-visual simultaneity. Journal of Vision 9 (12): 4, 1-8.
26. McGurk H, MacDonald J, (1976) Hearing lips and seeing voices. Nature 265: 746-748.
27. Arnold DH, Tear M, Schindel R, Roseboom W, (2010) Audio-Visual Speech Cue Combination. PLoS One 5 (4): e10217.
28. van Wassenhove V, Grant KW, Poeppel D, (2005) Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences U S A 102: 1181-1186.
29. Stekelenburg JJ, Vroomen J, (2007) Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience 19: 1964-1973.
30. Howard IP, Templeton WB, (1966) Human Spatial Orientation London Wiley.
31. Alais D, Burr D, (2004) The Ventriloquist Effect Results from Near-Optimal Bimodal Integration. Current Biology 14: 257-262.
32. Wichmann FA, Hill NJ, (2001) The psychometric function: I. Fitting, sampling and goodness-of-fit. Perception and Psychophysics 63 (8): 1293-1313.
33. Green DM, Swets JA, (1966) Signal detection theory and psychophysics New York Wiley.
34. Macmillan NA, Creelman CD, (1991) Detection theory: A user's guide Cambridge Cambridge Univ. Press.
35. Fujisaki W, Koene A, Arnold D, Johnston A, Nishida S, (2006) Visual search for a target changing in synchrony with an auditory signal. Proceedings of the Royal Society B: Biological Sciences 273 (1588): 865-874.
36. Watanabe K, Shimojo S, (2001) When Sound Affects Vision: Effects of Auditory Grouping on Visual Motion Perception. Psychological Science 12 (2): 109-116.
37. Sanabria D, Soto-Faraco S, Chan J, Spence C, (2005) Intramodal perceptual grouping modulates multisensory integration: evidence from the crossmodal dynamic capture task. Neuroscience Letters 377: 59-64.
38. Keetels M, Stekelenburg J, Vroomen J, (2007) Auditory grouping occurs prior to intersensory pairing: Evidence from temporal ventriloquism. Experimental Brain Research 180: 449-456.
39. Cook LA, Van Valkenburg DL, (2009) Audio-visual organisation and the temporal ventriloquism effect between grouped sequences: evidence that unimodal grouping precedes cross-modal integration. Perception 38 (8): 1220-33.
40. Alsius A, Navarra J, Campbell R, Soto-Faraco S, (2005) Audiovisual integration of speech falters under high attention demands. Current Biology 15: 839-843.
41. Van der Burg E, Olivers CNL, Bronkhorst AW, Theeuwes J, (2008) Pip and pop: Non-spatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance 34 (5): 1053-1065.
42. Van der Burg E, Olivers CN, Bronkhorst AW, Theeuwes J, (2009) Poke and pop: tactile-visual synchrony increases visual saliency. Neuroscience Letters 450 (1): 60-4.
43. Van der Burg E, Cass J, Olivers CNL, Theeuwes J, Alais D, (2010) Efficient Visual Search from Synchronized Auditory Signals Requires Transient Audiovisual Events. PLoS ONE 5 (5): e10664.
44. Fujisaki W, Nishida S, (2005) Temporal frequency characteristics of synchrony-asynchrony discrimination of audio-visual signals. Experimental Brain Research 166: 445-464.
45. Hirsh IJ, Sherrick CE, (1961) Perceived order in different sense modalities. Journal of Experimental Psychology 62: 423-432.
46. Zampini M, Brown T, Shore DI, Maravita A, Roder B, et al. (2005) Audiotactile temporal order judgments. Acta Psychologica 118 (3): 277-291.
47. Fujisaki W, Nishida S, (2009) Audio-tactile superiority over visuo-tactile and audio-visual combinations in the temporal resolution of synchrony perception. Experimental Brain Research 198 (2-3): 245-259.
48. Nishida S, Johnston A, (2002) Marker correspondence, not processing latency, determines temporal binding of visual attributes. Current Biology 5: 359-68.
49. Arnold DH, (2005) Perceptual pairing of colour and motion. Vision Research 45: 3015-3026.
50. Arnold DH, Wilcock P, (2007) Cortical processing and perceived timing. Proceedings of the Royal Society of London, Series B: Biological Sciences 274: 2331-2336.
51. Arnold DH, Ong Y, Roseboom W, (2009) Simple differential latencies modulate, but do not cause the flash-lag effect. Journal of Vision 9 (5): 4, 1-8.