Deconstructing the McGurk-MacDonald Illusion

Journal of Experimental Psychology: Human Perception and Performance

Volumn 35, Issue 2, 2009, Pages 580-587

  Soto Faraco, Salvador ^a,b   Alsius, Agnés ^a  

^a UNIVERSITY OF BARCELONA   (Spain)

^b INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

Author keywords

attention; audiovisual speech; multisensory; speech perception; temporal order

Indexed keywords

ARTICLE; ATTENTION; AUDITORY STIMULATION; HUMAN; ILLUSION; PHONETICS; PHOTOSTIMULATION; REFERENCE VALUE; SPEECH PERCEPTION; TIME; TIME PERCEPTION; VIDEORECORDING;

ACOUSTIC STIMULATION; ATTENTION; HUMANS; ILLUSIONS; PHONETICS; PHOTIC STIMULATION; REFERENCE VALUES; SPEECH PERCEPTION; TIME FACTORS; TIME PERCEPTION; VIDEOTAPE RECORDING;

EID: 65349117504     PISSN: 00961523     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0013483     Document Type: Article

References (59)

1. Alsius A., Navarra J., Campbell R., and Soto-Faraco S. Audiovisual integration of speech falters under high attention demands. Current Biology 15 (2005) 839-843
2. Alsius A., Navarra J., and Soto-Faraco S. Attention to touch reduces audiovisual speech integration. Experimental Brain Research 183 (2007) 399-404
3. Bertelson P., and Aschersleben G. Automatic visual bias of perceived auditory location. Psychonomic Bulletin & Review 5 (1998) 482-489
4. Bertelson P., and de Gelder B. The psychology of multimodal perception. In: Spence C., and Driver J. (Eds). Crossmodal space and crossmodal attention (2004), Oxford University Press, Oxford, England 141-179
5. Bertelson P., Vroomen J., Wiegeraad G., and de Gelder B. Exploring the relation between McGurk interference and ventriloquism. Proceedings of the International Congress on Spoken Language Processing (1994) 559-562
6. Campbell R., Dodd B., and Burnham D. Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (1998), Psychology Press, Hove, England
7. Colin C., Radeau M., Deltenre P., and Morais J. Rules of intersensory integration in spatial scene analysis and speech reading. Psychologica Belgica 41 (2001) 131-144
8. Colin C., Radeau M., Soquet A., Demolin D., Colin F., and Deltenre P. Mismatch negativity evoked by the McGurk-MacDonald effect: A phonetic representation within short-term memory. Clinical Neurophysiology 113 (2002) 495-506
9. Coltheart M. Modularity and cognition. Trends in Cognitive Sciences 3 (1999) 115-120
10. de Gelder B., and Bertelson P. Multisensory integration, perception and ecological validity. Trends in Cognitive Sciences 7 (2003) 460-467
11. Dixon N.F., and Spitz L. The detection of auditory visual desynchrony. Perception 9 (1980) 719-721
12. Driver J. Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading. Nature 381 (1996, May 2) 66-68
13. Driver J., and Noesselt T. Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 57 (2008) 11-23
14. Easton R.D., and Basala M. Perceptual dominance during lipreading. Perception & Psychophysics 32 (1982) 562-570
15. Erlebacher A., and Sekuler R. Response frequency equalization: A bias model for psychophysics. Perception & Psychophysics 9 (1971) 315-320
16. Fodor J. The modularity of mind (1983), MIT Press, Cambridge, MA
17. Forster K.L., and Forster J.C. A Windows display program with millisecond accuracy. Behavior Research Methods, Instruments, & Computers 35 (2003) 116-124
18. Green K.P., Kuhl P.K., Meltzoff A.N., and Stevens E.B. Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect. Perception & Psychophysics 50 (1991) 524-536
19. Hirsh I.J., and Sherrick Jr. C.E. Perceived order in different sense modalities. Journal of Experimental Psychology 62 (1961) 423-432
20. Jackson C.V. Visual factors in auditory localization. Quarterly Journal of Experimental Psychology 5 (1953) 52-65
21. Jones J.A., and Jarick M. Multisensory integration of speech signals: The relationship between space and time. Experimental Brain Research 174 (2006) 588-594
22. Jones J.A., and Munhall K.G. The effects of separating auditory and visual sources on audiovisual integration of speech. Canadian Acoustics 25 (1997) 13-19
23. Keetels M., Stekelenburg J., and Vroomen J. Auditory grouping occurs prior to intersensory pairing: Evidence from temporal ventriloquism. Experimental Brain Research 180 (2007) 449-456
24. Kislyuk D.S., Möttönen R., and Sams M. Visual processing affects the neural basis of auditory discrimination. Journal of Cognitive Neuroscience (2008) [Epub ahead of print, doi: 10.1162/jocn.2008.20152]
25. Kohlrausch A., and van de Par S. Audio-visual interaction in the context of multi-media applications. In: Blauert J. (Ed). Communication acoustics (2005), Springer, Berlin, Germany 109-138
26. Koppen C., and Spence C. Audiovisual asynchrony modulates the Colavita visual dominance effect. Brain Research 1186 (2007) 224-232
27. Massaro D.W. Speech perception by ear and eye: A paradigm for psychological enquiry (1987), Erlbaum, Hillsdale, NJ
28. Massaro D.W. Perceiving talking faces: From speech perception to a behavioral principle (1998), MIT Press, Cambridge, MA
29. Massaro D.W., and Cohen M.M. Perceiving asynchronous speech in consonant-vowel and vowel syllables. Speech Communication 13 (1993) 127-134
30. Massaro D.W., Cohen M.M., and Smeele P.M.T. Perception of asynchronous and conflicting visible and auditory speech. Journal of the Acoustical Society of America 100 (1996) 1777-1786
31. McGrath M., and Summerfield Q. Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. Journal of the Acoustical Society of America 77 (1985) 678-685
32. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976, December 23) 746-748
33. Miller L.M., and D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. Journal of Neuroscience 25 (2005) 5884-5893
34. Moutoussis K., and Zeki S. A direct demonstration of perceptual asynchrony in vision. Proceedings of the Royal Society of London: Series B, Biological Sciences 264 (1997) 393-399
35. Moutoussis K., and Zeki S. The relationship between cortical activation and perception investigated with invisible stimuli. Proceedings of the National Academy of Sciences, USA 99 (2002) 9527-9532
36. Munhall K.G., Gribble P., Sacco L., and Ward M. Temporal constraints on the McGurk effect. Perception & Psychophysics 58 (1996) 351-362
37. Navarra, J., Alsius, A., Soto-Faraco, S., & Spence, C. (in press). Assessing the role of attention in the audiovisual integration of speech. Information Fusion
38. Navarra J., Vatakis A., Zampini M., Soto-Faraco S., Humphreys W., and Spence C. Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Research 25 (2005) 499-507
39. Pylyshyn Z. Is vision continuous with cognition? The case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences 22 (1999) 341-365
40. Radeau, M. (1998). Auditory-visual interactions in spatial scene analysis: Development and neural bases. In D. Burnham, J. Robert-Ribes, & E. Vatikiotis-Bateson (Eds.), Proceedings of the International Conference on Audio-Visual Speech Processing (pp. 97-102). Terrigal-Sydney: Causal Productions. Retrieved from http://www.isca-speech.org/archive/avsp98/av98_097.html
41. Sanabria D., Soto-Faraco S., Chan J., and Spence C. Intramodal perceptual grouping modulates multisensory integration: Evidence from the crossmodal dynamic capture task. Neuroscience Letters 377 (2005) 59-64
42. Sanabria D., Soto-Faraco S., and Spence C. Exploring the role of visual perceptual grouping on the audiovisual integration of motion. Neuroreport 15 (2004) 2745-2749
43. Soto-Faraco S., and Alsius A. Conscious access to the uni-sensory components in a cross-modal illusion. Neuroreport 18 (2007) 347-350
44. Soto-Faraco S., Navarra J., and Alsius A. Assessing automaticity in audiovisual speech integration: Evidence from the speeded classification task. Cognition 92 (2004) B13-B23
45. Soto-Faraco S., Spence C., and Kingstone A. Assessing automaticity in the audiovisual integration of motion. Acta Psychologica 118 (2005) 71-92
46. Spence C., Sanabria D., and Soto-Faraco S. Intersensory Gestalten and crossmodal scene perception. In: Noguchi K. (Ed). The psychology of beauty and Kansei: New horizons of Gestalt perception (2007), Nihon University College of Humanities and Sciences, Tokyo 519-579
47. Stelmach L.B., and Herdman C.M. Directed attention and perception of temporal order. Journal of Experimental Psychology: Human Perception and Performance 17 (1991) 539-550
48. Sumby W.H., and Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America 26 (1954) 212-215
49. Summerfield Q., and McGrath M. Detection and resolution of audio-visual incompatibility in the perception of vowels. Quarterly Journal of Experimental Psychology 36 (1984) 51-74
50. van Eijk, P. L. J. (2008). Audio-visual synchrony perception. Unpublished doctoral dissertation, Technische Universiteit Eindhoven, Eindhoven, the Netherlands. Available from http://www.tue.nl/bib
51. van Eijk, P. L. J., Kohlrausch, A., van de Par, S., & Juola, J. F. (2006, June). The influence of psychophysical procedure and stimulus type on estimates of human performance in detecting audio-visual asynchrony. Poster session presented at the annual meeting of the International Multisensory Research Forum, Dublin, Ireland.
52. van Wassenhove V., Grant K.W., and Poeppel D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45 (2007) 598-607
53. Vatakis A., Navarra J., Soto-Faraco S., and Spence C. Audiovisual temporal adaptation of speech: Temporal order versus simultaneity judgments. Experimental Brain Research 185 (2008) 521-529
54. Vatakis, A., & Spence, C. (2007a, April). An assessment of the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception. Poster session presented at the One Day Meeting for Young Speech Researchers, University College London, London, England.
55. Vatakis A., and Spence C. Crossmodal binding: Evaluating the "unity assumption" using audiovisual speech stimuli. Perception & Psychophysics 69 (2007) 744-756
56. Walker S., Bruce V., and O'Malley C. Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect. Perception & Psychophysics 57 (1995) 1124-1133
57. Welch R.B. Meaning, attention, and the "unity assumption" in the intersensory bias of spatial and temporal perceptions. In: Aschersleben G., Bachmann T., and Müsseler J. (Eds). Cognitive contributions to the perception of spatial and temporal events (1999), Elsevier, Amsterdam 371-387
58. Welch R.B., and Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 3 (1980) 638-667
59. Zampini M., Guest S., Shore D.I., and Spence C. Audio-visual simultaneity judgments. Perception & Psychophysics 67 (2005) 531-544