The effect of a concurrent working memory task and temporal offsets on the integration of auditory and visual speech information

Seeing and Perceiving

Volumn 25, Issue 1, 2012, Pages 87-106

  Buchan, Julie N ^a   Munhall, Kevin G ^a  

^a QUEEN S UNIVERSITY   (Canada)

Author keywords

Audiovisual speech; cognitive load; eye tracking; McGurk effect; multisensory integration; working memory

Indexed keywords

AUDIO-VISUAL SPEECH; COGNITIVE LOADS; EYE-TRACKING; MCGURK EFFECT; MULTISENSORY INTEGRATION; WORKING MEMORIES;

BEHAVIORAL RESEARCH; SPEECH PROCESSING;

INTEGRATION;

ADOLESCENT; ADULT; ARTICLE; COGNITION; FEMALE; HEARING; HUMAN; MALE; MEMORY; PHYSIOLOGY; SPEECH PERCEPTION; VISION;

ADOLESCENT; ADULT; AUDITORY PERCEPTION; COGNITION; FEMALE; HUMANS; MALE; MEMORY; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84858253014     PISSN: 18784755     EISSN: 18784763     Source Type: Journal    
DOI: 10.1163/187847611X620937     Document Type: Article

References (34)

1. Alsius, A., Navarra, J. and Soto-Faraco, S. (2007). Attention to touch weakens audiovisual speech integration, Exper. Brain Res. 183, 399-404.
2. Alsius, A., Navarra, J., Campbell, R. and Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attentional demands, Curr. Biol. 15, 839-843.
3. Andersen, T. S., Tiippana, K., Laarni, J., Kojo, I. and Sams, M. (2009). The role of visual spatial attention in audiovisual speech perception, Speech Commun. 51, 184-193.
4. Arrighi, R., Alais, D. and Burr, D. (2006). Perceptual synchrony of audiovisual streams for natural and artificial motion sequences, J. Vision 6, 260-268.
5. Baart, M. and Vrooment, J. (2010). Phonetic recalibration does not depend on working memory, Exper. Brain Res. 203, 575-582.
6. Buchan, J. N., Paré, M. and Munhall, K. G. (2007). Spatial statistics of gaze fixations during dynamic face processing, Soc. Neurosci. 2, 1-13.
7. Buchan, J. N., Paré, M. and Munhall, K. G. (2008). The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception, Brain Res. 1242, 162-171.
8. Buchan, J. N., Wilson, A., Paré, M. and Munhall, K. G. (2005). The effect of spatial frequency information in central and peripheral vision on natural gaze patterns and audiovisual speech perception, J. Acoustic. Soc. Amer. 117, 2620.
9. Conrey, B. and Pisoni, D. B. (2006). Auditory-visual speech perception and synchrony detection for speech and nonspeech signals, J. Acoustic. Soc. Amer. 119, 4065-4073.
10. de Fockert, J. W., Rees, G., Frith, C. D. and Lavie, N. (2001). The role of working memory in visual selective attention, Science 291, 1803-1806.
11. Dixon, N. and Spitz, L. (1980). The detection of audiovisual desynchrony, Perception 9, 719-721.
12. Doherty-Sneddon, G. and Phelps, F. G. (2005). Gaze aversion: A response to a cognitive or social difficulty? Memory Cognit. 33, 727-733.
13. Erber, N. P. (1969). Interaction of audition and vision in the recognition of oral speech stimuli, J. Speech Hearing Res. 12, 423-425.
14. Grant, K. W., van Wassenhove, V. and Poeppel, D. (2004). Detection of auditory (cross-spectral) and auditory-visual (cross-modal) synchrony, Speech Comm. 44, 43-53.
15. Jones, J. A. and Jarick, M. (2006).Multisensory integration of speech signals: The relationship between space and time, Exper. Brain Res. 174, 588-594.
16. Lewkowicz, D. J. (1996). Perception of auditory-visual temporal synchrony in human infants, J. Exper. Psychol.: Human Percept. Perform. 22, 1094-1106.
17. Liberman, A. M. (1982). On the finding that speech is special, Amer. Psychol. 37, 148-167.
18. MacDonald, J., Andersen, S. and Bachmann, T. (2000). Hearing by eye: How much spatial degradation can be tolerated? Perception 29, 1155-1168.
19. McGurk, H. and MacDonald, J. (1976). Hearing lips and seeing voices, Nature 264, 746-748.
20. Munhall, K. G., Gribble, P., Sacco, L. and Ward, M. (1996). Temporal constraints on the McGurk effect, Percept. Psychophys. 58, 351-362.
21. Munhall, K. G., Kroos, C., Jozan, G. and Vatikiotis-Bateson, E. (2004). Spatial frequency requirements of audiovisual speech perception, Percept. Psychophys. 66, 574-583.
22. Paré, M., Richler, R. C., ten Hove, M. and Munhall, K. G. (2003). Gaze behavior in audiovisual speech perception: The influence of ocular fixations on the McGurk effect, Percept. Psychophys. 65, 533-567.
23. Parkhurst, D. J. and Neibur, E. (2003). Scene content selected by active vision, Spatial Vision 16, 125-154.
24. Parkhurst, D., Law, K. and Neibur, E. (2002). Modeling the role of salience in the allocation of overt visual attention, Vision Res. 42, 107-123.
25. Reisberg, D., McLean, J. and Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli, in:Hearing by Eye: The Psychology of Lip-Reading , B. Dodd and R. Campbell (Eds). Lawrence Erlbaum Associates, Hillsdale, NJ, USA.
26. Rosenblum, L. D., Schmuckler, M. A. and Johnson, J. A. (1997). The McGurk effect in infants, Percept. Psychophys. 59, 347-357.
27. Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C. and Foxe, J. J. (2007). Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments, Cerebral Cortex 17, 1147-1153.
28. Soto-Faraco, S., Navarra, J. and Alsius, A. (2004). Assessing automaticity in audiovisual speech integration: Evidence from the speeded classification task, Cognition 92, B13-B23.
29. Sumby, W. H. and Pollack, I. (1954). Visual contributions to speech intelligibility in noise, J. Acoust. Soc. Amer. 26, 212-215.
30. Summerfield, Q. and McGrath, M. (1984). Detection and resolution of audio-visual incompatibility in the perception of vowels, Qtrly J. Exper. Psychol. A 36, 51-74.
31. Tiippana, K., Andersen, T. S. and Sams, M. (2004). Visual attention modulates audiovisual speech perception, Eur. J. Cognit. Psychol. 16, 457-472.
32. van Wassenhove, V., Grant, K. and Poeppel, D. (2007). Temporal window of integration in auditoryvisual speech perception, Neuropsychologia 45, 598-607.
33. Vatikiotis-Bateson, E., Eigsti, I. M., Yano, S. and Munhall, K. G. (1998). Eye movement of perceivers during audiovisual speech perception, Percept. Psychophys. 60, 926-940.
34. Wilson, A., Wilson, A., ten Hove, M., Paré, M. and Munhall, K. G. (2008). Loss of central vision and audiovisual speech perception, Visual Impairment Res. 10, 23-34.