Facilitation of multisensory integration by the "unity effect" reveals that speech is special

Journal of Vision

Volumn 8, Issue 9, 2008, Pages

  Vatakis, Argiro ^a   Ghazanfar, Asif A ^b   Spence, Charles ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b PRINCETON UNIVERSITY   (United States)

Author keywords

Cross modal; Monkey vocalizations; Multisensory; Temporal perception

Indexed keywords

ANIMAL; ARTICLE; AUDITORY STIMULATION; COMPARATIVE STUDY; FEMALE; HEARING; HUMAN; MACACA; MALE; PHOTOSTIMULATION; PHYSIOLOGY; SPEECH PERCEPTION; VISION; VOCALIZATION;

ACOUSTIC STIMULATION; ANIMALS; AUDITORY PERCEPTION; FEMALE; HUMANS; MACACA MULATTA; MALE; PHOTIC STIMULATION; SPEECH PERCEPTION; VISUAL PERCEPTION; VOCALIZATION, ANIMAL;

EID: 48249091584     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/8.9.14     Document Type: Article

References (44)

1. Bernstein, L. E., Auer, E. T., & Moore, J. K. (2004). Audiovisual speech binding: Convergence or association? In G. A. Calvert, C. Spence, & B. E. Stein (Eds.), The handbook of multisensory processing (pp. 203-223). Cambridge, MA: MIT Press.
2. Bertelson, P., & de Gelder, B. (2004). The psychology of multimodal perception in crossmodal space and crossmodal attention. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodal attention (pp. 141-177). Oxford: Oxford University Press.
3. Bindemann, M., Burton, A. M., Hooge, I. T., Jenkins, R., & de Haan, E. H. (2005). Faces retain attention. Psychonomic Bulletin & Review, 6, 1048-1053.
4. Coren, S., Ward, L. M., & Enns, J. T. (2004). Sensation & perception (6th ed.). Fort Worth: Harcourt Brace.
5. Diamond, R., & Carey, S. (1986). Why faces are and are not special: An effect of expertise. Journal of Experimental Psychology: General, 115, 107-117.
6. Easton, R. D., & Basala, M. (1982). Perceptual dominance during lipreading. Perception & Psychophysics, 32, 562-570.
7. Finney, D. J. (1964). Probit analysis: Statistical treatment of the sigmoid response curve. London, UK: Cambridge University Press.
8. Fitch, W. T. (1997). Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaques. Journal of the Acoustical Society of America, 102, 1213-1222.
9. Fitch, W. T., & Hauser, M. D. (1995). Vocal production in nonhuman primates: Acoustics, physiology and functional constraints on 'honest' advertising. American Journal of Primatology, 37, 191-219.
10. Frick, R. W. (1995). Accepting the null hypothesis. Memory & Cognition, 23, 132-138.
11. Ghazanfar, A. A., & Logothetis, N. K. (2003). Neuroperception: Facial expressions linked to monkey calls. Nature, 423, 937-938.
12. Ghazanfar,. A.A., Maier, J. X., Hoffman, K. L., & Logothetis, N. K. (2005). Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience, 25, 5004-5012.
13. Ghazanfar, A. A., Turesson, H. K., Maier J. X., van Dinther, R., Patterson, R. D., & Logothetis, N. K. (2007), Vocal-tract resonances as indexical cues in rhesus monkeys. Current Biology, 17, 425-430.
14. Guski, R., & Troje, N. F. (2003). Audiovisual phenomenal causality. Perception & Psychophysics, 65, 789-800.
15. Hauser, M. D., Evans, C. S., & Marler, P. (1993). The role of articulation in the production of rhesus monkeys, Macaca mulatta, vocalizations. Animal Behaviour, 45, 423-433.
16. Jackson, C. V. (1953). Visual factors in auditory localization. Quarterly Journal of Experimental Psychology, 5, 52-65.
17. Jones, J. A., & Jarick, M. (2006). Multisensory integration of speech signals: The relationship between space and time. Experimental Brain Research, 174, 588-594.
18. Lewkowicz, D. J. (1996). Perception of auditory-visual temporal synchrony in human infants. Journal of Experimental Psychology: Human Perception and Performance, 22, 1094-1106.
19. Lewkowicz, D. J. (2000). Infants' perception of the audible, visible, and bimodal attributes of multimodal syllables. Child Development, 71, 1241-1257.
20. Lewkowicz, D. J., & Ghazanfar, A. A. (2006) The decline of cross-species intersensory perception in human infants. Proceedings of the National Academy of Sciences of the United States of America, 103, 6771-6774.
21. Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revisited. Cognition, 21, 1-36.
22. Massaro, D. W. (2004). From multisensory integration to talking heads and language learning. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.), The handbook of multisensory processing (pp. 153-176). Cambridge, MA: MIT Press.
23. Morein-Zamir, S., Soto-Faraco, S., & Kingstone, A. (2003). Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research, 17, 154-163.
24. Munhall, K. G., & Vatikiotis-Bateson, E. (2004). Spatial and temporal constraints on audiovisual speech perception. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.), The handbook of multisensory processing (pp. 177-188). Cambridge, MA: MIT Press.
25. Radeau, M., & Bertelson, P. (1977). Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations. Perception & Psychophysics, 22, 137-146.
26. Radeau, M., & Bertelson, P. (1987). Auditory-visual interaction and the timing of inputs. Thomas (1941) revisited. Psychological Research, 49, 17-22.
27. Rendall, D., Owren, M. J., & Rodman, P. S. (1998). The role of vocal tract filtering in identity cueing in rhesus monkey (Macaca mulatta) vocalizations. Journal of the Acoustical Society of America, 103, 602-614.
28. Saldaña, H. M., & Rosenblum, L. D. (1993). Visual influences on auditory pluck and bow judgments. Perception & Psychophysics, 54, 406-416.
29. Smith, D. R., & Patterson, R. D. (2005). The interaction of glottal-pulse rate and vocal-tract length in judgements of speaker size, sex and age. Journal of the Acoustical Society of America, 118, 3177-3186.
30. Spence, C. (2007). Audiovisual multisensory integration. Acoustical Science and Technology, 28, 61-70.
31. Spence, C., Shore, D. I., & Klein, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130, 799-832.
32. Summerfield, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3-51). London: LEA.
33. Theeuwes, J., & Van der Stigchel, S. (2007). Faces capture attention: Evidence from inhibition-of-return. Visual Cognition, 13, 657-665.
34. Thomas, G. J. (1941). Experimental study of the influence of vision on sound localization. Journal of Experimental Psychology, 28, 163-177.
35. Tuomainen, J., Andersen, T. S., Tiippana, K., & Sams, M. (2005). Audio-visual speech perception is special. Cognition, 96, B13-B22.
36. Vatakis, A., & Spence, C. (2007). Crossmodal binding: Evaluating the "unity assumption" using audiovisual speech stimuli. Perception & Psychophysics, 69, 744-756.
37. Vatakis, A., & Spence, C. (2008). Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli. Acta Psychologica, 127, 12-23.
38. Vroomen, J., & Keetels, M. (2006). The spatial constraint in intersensory pairing: No role in temporal ventriloquism. Journal of Experimental Psychology: Human Perception and Performance, 32, 1063-1071.
39. Walker, S., Bruce, V., & O'Malley, C. (1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect. Perception & Psychophysics, 57, 1124-1133.
40. Warren, D. H., Welch, R. B., & McCarthy, T. J. (1981). The role of visual-auditory 'compellingness' in the ventriloquism effect: Implications for transitivity among the spatial senses. Perception & Psychophysics, 30, 557-564.
41. Welch, R. B. (1999a). Meaning, attention, and the "unity assumption" in the intersensory bias of spatial and temporal perceptions. In G. Aschersleben, T. Bachmann, & J. Müsseler (Eds.), Cognitive contributions to the perception of spatial and temporal events (pp. 371-387). Amsterdam: Elsevier Science, BV.
42. Welch, R. B. (1999b). The advantages and limitations of the psychophysical staircase procedure in the study of intersensory bias: Commentary on Bertelson in Cognitive contributions to the perception of spatial and temporal events. In G. Aschersleben, T. Bachmann, & J. Müsseler (Eds.), Cognitive contributions to the perception of spatial and temporal events (pp. 363-369). Amsterdam: Elsevier Science, BV.
43. Welch, R. B., & Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638-667.
44. Witkin, H. A., Wapner, S., & Leventhal, T. (1952). Sound localization with conflicting visual and auditory cues. Journal of Experimental Psychology, 43, 58-67.