The perceptual basis for audiovisual speech integration

7th International Conference on Spoken Language Processing, ICSLP 2002

Volumn , Issue , 2002, Pages 1461-1464

  Rosenblum, Lawrence D ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INTEGRATION;

AUDIO-VISUAL SPEECH; MULTI-MODAL; NEUROPSYCHOLOGICAL; SPEECH PERCEPTION;

SPEECH;

EID: 85009242646     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (43)

1. Summerfield, Q. "Some preliminaries to a comprehensive account of audiovisual speech perception. " In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 53-83). London: Lawrence Erlbaum Associates, Inc., 1987.
2. Green, K. P. "The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. " In R. Campbell, B. Dodd & D. Burnham (Eds.), Hearing by Eye II: Advances in the Psychology of Speechreading and Auditory-Visual Speech (pp. 3-25). London, England: Lawrence Erlbaum Assoc. 1998.
3. Massaro, D. W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Lawrence Erlbaum Assoc, Inc.
4. Bernstein, L. E., Auer, E. T., Moore, J. K. "Modality-Specific Perception of Auditory and Visual Speech". To appear in B. Stein (Ed.) The Handbook of Multimodal Processing. In press.
5. Gibson, J. J. The ecological approach to visual perception. Boston: Houghton-Mifflin, 1979.
6. Stoffregen T. A. & Bardy, B. G. "On specification and the senses." Behavioral & Brain Sciences, 24(2), 195-261. 2001.
7. Rosenblum, L. D. & Gordon, M. S. "The generality of specificity: Some lessons from audiovisual speech." Behavioral & Brain Sciences, 24(2), 239-240. 2001
8. McGurk, H. & MacDonald, J. W. "Hearing lips and seeing voices." Nature, 264, 746-748. 1976.
9. Green, K. P., Kuhl, P. K., Meltzoff, A. M., & Stevens, E. B. "Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect" Perception and Psychophysics, 50, 524-536. 1991.
10. Rosenblum, L. D. and Saldaña, H. M. (1996). "An audiovisual test of kinematic primitives for visual speech perception. " Journal of Experimental Psychology: Human Perception and Performance. 22(2), 318-331. 1996.
11. Massaro, D. W., Cohen, M. M., Gesi, A., Heredia, R., & Tsuzaki, M. (1993). Bimodal speech perception: An examination across languages. Journal of Phonetics, 21, 445-478.
12. Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997). The McGurk effect in infants. Perception & Psychophysics, 59(3), 347-357.
13. Fowler, C. A. & Dekle, D. J. "Listening with eye and hand: Cross-modal contributions to speech perception. " Journal of Experimental Psychology: Human Perception & Performance, 17, 816-828. 1991.
14. Green, K. P. & Miller, J. L. "On the role of visual rate information in phonetic perception. " Perception and Psychophysics, 38, 269-276. 1985.
15. Green, K. P., & Kuhl, P. K. "The role of visual information in the processing of place and manner features in speech perception. " Perception & Psychophysics, 45, 34-42. 1989.
16. Whalen, D. H. "Effects of vocalic formant transitions and vowel quality on the English [s]-[S] boundary." Journal of the Acoustical Society of America, 69, 275-282. 1981.
17. Green, K. P. & Gerdman, A. "Cross-modal discrepancies in coarticulation and the integration of speech information: The McGurk effect with mismatched vowels." Journal of Experimental Psychology: Human Perception and Performance, 21, 1409-1426. 1995.
18. Green, K. P. & Norrix, L. "The Perception of/r/and/l/in a Stop Cluster: Evidence of Cross-Modal Context Effects". Journal of Experimental Psychology: Human Perception & Performance. 27(1): p. 166-177. 2001.
19. Calvert, G. A., Bullmore, E., Brammer, M. J., Campbell, R., Iversen, S. D., Woodruff, P., McGuire, P., Williams, S., David, A. S. "Silent lipreading activates the auditory cortex." Science, 276, 593-596. 1997.
20. MacSweeney, M., Amaro, E., Calvert, G. A. & Campbell, R. "Silent speechreading in the absence of scanner noise: An event-related fMRI study." Neuroreport: For Rapid Communication of Neuroscience Research, 11, 1729-1733. 2000.
21. Bernstein, L. E., Auer, E. T., Moore, J. K., Ponton, C., Don, M., and Singh, M., 2002. "Visual speech perception without primary auditory cortex activation. " NeuroReport, 13:311-315. 2002.
22. Sams, M., Aulanko, R., Hamalainen, M., Hari., R., Lounasmaa, O. V., Lu, S.-T., & Simola, J. (1991). "Seeing speech: visual information from lip movements modifies activity in the human auditory cortex." Neuroscience Letters, 127, 141-145. 1991.
23. Munhall, K. G. & Vatikiotis-Bateson, E. "The moving face during speech communication. " In R. Campbell & B. Dodd (Eds.), Hearing by Eye II: Advances in the Psychology of Speechreading and Auditory-Visual Speech (pp. 123-139). London, England: Lawrence Erlbaum Assoc. 1998.
24. Vatikiotis-Bateson, E. "The stimulus as basis for audiovisual integration. " Proceedings of the 7th International Conference on Spoken Language Comprehension. 2002.
25. Remez, R., Rubin, P., Pisoni, D., & Carrell, T. "Speech perception without traditional speech cues." Science, 212, 947-950. 1981.
26. Rosenblum, L. D., Johnson, J. A., & Saldaña, H. M. "Visual kinematic information for embellishing speech in noise." Journal of Speech and Hearing Research 39(6), 1159-1170. 1996.
27. Jenkins, J. J., Strange, W. & Edman. T. R. "Identification of vowels in "vowelless" syllables." Perception & Psychophysics, 34, 441-450. 1983.
28. Strange, W., Jenkins, J. J. & Johnson, T. L. "Dynamic specification of coarticulated vowels." Journal of the Acoustical Society of America, 74, 695-705. 1983.
29. Yakel, D. A. Effects of time-varying information on vowel identification accuracy in visual speech perception. Doctoral dissertation, University of California, Riverside, USA. 2000.
30. Remez, R. E., Fellowes, J. M. & Rubin, P. E. "Talker identification based on phonetic information. " Journal of Experimental Psychology: Human Perception & Performance 23(3), 651-666. 1997.
31. Mullennix, J. W., Pisoni, D. B., & Martin, C. S. Some effects of talker variability on spoken word recognition. Journal of the Acoustical Society of America, 85(1), 365-378. 1989.
32. Nygaard, L. C., Sommers, M. S., & Pisoni, D. B. "Speech perception as a talker-contingent process." Psychological Science, 5(1), 42-46. 1994.
33. Palmeri, T. J., Goldinger, S. D., & Pisoni, D. B. "Episodic encoding of voice attributes and recognition memory for spoken words." Journal of Experimental Psychology: Learning, Memory, and Cognition, 19(2), 309-328. 1993.
34. Saldaña, H. M., & Rosenblum, L. D. "Voice information in auditory form-based priming." Journal of the Acoustical Society of America, 95(5), 2870. 1994.
35. Schweinberger, S. R. & Soukup, G. R. "Asymmetric relationships among perceptions of facial identity, emotion, and facial speech." Journal of Experimental Psychology: Human Perception and Performance, 24, 1748-1765. 1998.
36. Yakel, D. A., Rosenblum, L. D., & Fortier, M. A. "Effects of talker variability on speechreading." Perception & Psychophysics, 62, 1405-1412. 2000.
37. Sheffert, S. M. & Fowler, C. A. (1995). "The effects of voice and visible speaker change on memory for spoken words." Journal of Memory and Language, 34, 665-685. 1995.
38. Walker, S., Bruce, V., & O'Malley, C. "Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect." Perception & Psychophysics, 57, 1124-1133. 1995.
39. Jordan, T. R., & Bevan, K. "Seeing and hearing rotated faces: Influences of facial orientation on visual and audio-visual speech recognition. " Journal of Experimental Psychology: Human Perception and Performance., 23, 388-403. 1997.
40. Rosenblum, L. D., Yakel, D. A., & Greene, K. G. "Face and mouth inversion affects on visual and audiovisual speech perception. " Journal of Experimental Psychology: Human Perception and Performance. 26(3), 806-819. 2000.
41. Rosenblum, L. D., Yakel, D. A., Baseer, N., Panchal, A., Nordarse, B. C. & Niehus, R. P. "Visual speech information for face recognition. " Perception & Psychophysics, 64(2), 220-229. 2002.
42. Rosenblum, L. D.,., & Smith, N. "Look who's talking: Recognizing friends from visible articulation. " To be submitted to Psychological Science. In preparation.
43. Rosenblum, L. D., Nichols, S, Lee, J., "Matching voices to visible speech movements: Evidence for crossmodal idiolectic information. " To be submitted to Journal of Experimental Psychology: Human Perception & Performance. In preparation.