Primacy of Multimodal Speech Perception

The Handbook of Speech Perception

Volumn , Issue , 2008, Pages 51-78

  Rosenblum, Lawrence D ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Automaticity; Haptic apprehension; Multimodal speech; Multimodal speech perception; Sensory modalities

Indexed keywords

EID: 84890640150     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9780470757024.ch3     Document Type: Chapter

References (177)

1. Armstrong, D., Stokoe, W. C., & Wilcox, S. (1994). Gesture and the Nature of Language. Cambridge: Cambridge University Press.
2. Arnold, P. & Hill, F. (2001). Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92, 339-55.
3. Bahrick, L. E. & Pickens, J. N. (1994). Amodal relations: The basis for intermodal perception and learning in infancy. In D. J. Lewkowicz & R. Lickliter (eds.), The Development of Intersensory Perception: Comparative Perspectives (pp. 205-32). Hillsdale, NJ: Lawrence Erlbaum.
4. Bartlett, J. C. & Searcy, J. (1993). Inversion and configuration of faces. Cognitive Psychology, 25, 281-316.
5. Bassili, J. N. (1978). Facial motion in the perception of faces and of emotional expression. Journal of Experimental Psychology: Human Perception and Performance, 4, 373-9.
6. Baynes, K., Funnell, M. G., & Fowler, C. A. (1994). Hemispheric contributions to the integration of visual and auditory information in speech perception. Perception & Psychophysics, 55, 633-41.
7. Bermant, R. I. & Welch, R. B. (1976). Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition. Perceptual & Motor Skills, 43, 487-93.
8. Bernstein, L. E., Auer, E. T., Moore, J. K., Ponton, C., Don, M., & Singh, M. (2002). Visual speech perception without primary auditory cortex activation. NeuroReport, 13, 311-15.
9. Bernstein, L. E., Auer, E. T., & Moore, J. K. (2004). Modality-specific perception of auditory and visual speech. In G. A. Calvert, C. Spence, & B. E. Stein (eds.), The Handbook of Multisensory Processing (pp. 203-23). Cambridge, MA: MIT Press.
10. Berry, D. S. (1990). What can a moving face tell us? Journal of Personality and Social Psychology, 58, 1004-14.
11. Berry, D. S. (1991). Child and adult sensitivity to gender information in patterns of facial motion. Ecological Psychology, 3, 349-66.
12. Bertelson, P. & Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & Psychophysics, 29, 578-84.
13. Bertelson, P., Vroomen, J., Wiegeraad, G., & de Gelder, B. (1994). Exploring the relation between McGurk interference and ventriloquism. Proceedings of the 1994 International Conference on Spoken Language Processing (ICSLP94), 2, 559-62.
14. Blumstein, S., Isaacs, E., & Mertus, J. (1982). The role of gross spectral shape as a perceptual cue to place of articulation in initial stop consonants. Journal of the Acoustical Society of America, 72, 43-50.
15. Bradlow, A. R., Nygaard, L. C., & Pisoni, D. B. (1999). Effects of talker, rate, and amplitude variation on recognition memory for spoken words. Perception & Psychophysics, 61, 206-19.
16. Braida, L. D. (1991). Crossmodal integration in the identification of consonant segments. Quarterly Journal of Experimental Psychology, 43, 647-77.
17. Browman, C. P. & Goldstein, L. M. (1985). Dynamic modeling of phonetic structure. In V. A. Fromkin (ed.), Phonetic Linguistics: Essays in Honor of Peter Ladefoged (pp. 35-53). New York: Academic Press.
18. Bruce, V. & Valentine, T. (1988). When a nod's as good as a wink: The role of dynamic information in facial recognition. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1: Memory in Everyday Life (pp. 169-74). New York: John Wiley & Sons.
19. Buechel, C., Price, C., Frackowiak, R. S. J., & Friston, K. (1998) Different activation patterns in the visual cortex of late and congenitally blind subjects. Brain, 121, 409-19.
20. Burnham, D. (1998). Language specificity in the development of auditory-visual speech perception. In R. Campbell, B. Dodd, & D. Burnham (eds.), Hearing by Eye II: The Psychology of Speechreading and Audiovisual Speech (pp. 27-60). Hove, UK: Psychology Press.
21. Burnham, D. & Dodd, B. (1996). Auditory-visual speech perception as a direct process: The McGurk effect in infants and across languages. In D. Stork & M. Hennecke (eds.), Speechreading by Humans and Machines (pp. 103-14). Berlin: Springer-Verlag.
22. Callan, D. E., Callan, A. M., Kroos, C., & Vatikiotis-Bateson, E. (2001). Multimodal contribution to speech perception revealed by independent component analysis: A single-sweep EEG case study. Cognitive Brain Research, 10, 349-53.
23. Calvert, G. A. (2001). Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex, 1, 1110-23.
24. Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., & David, A. S. (1999). Response amplification in sensory-specific cortices during crossmodal binding. NeuroReport, 10, 2619-23.
25. Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Iversen, S. D., Woodruff, P., McGuire, P., Williams, S., & David, A. S. (1997). Silent lipreading activates the auditory cortex. Science, 276, 593-6.
26. Campbell, R. (1986). The lateralization of lip-read sounds: A first look. Brain and Cognition, 5, 1-21.
27. Campbell, R. (1992). The neuropsychology of lipreading. Philisophical Transcriptions of the Royal Society of London, 335, 39-45.
28. Campbell, R. (1998). Everyday speechreading: Understanding seen speech in action. Scandinavian Journal of Psychology, 39, 163-7.
29. Campbell, R. T., Landis, T., & Regard, M. (1986). Face recognition and lip-reading: A neurological dissociation. Brain, 109, 509-21.
30. Christie, F. & Bruce, V. (1998). The role of dynamic information in the recognition of unfamiliar faces. Memory & Cognition, 26, 780-90.
31. Church, B. A. & Schacter, D. L. (1994). Perceptual specificity of auditory priming: Implicit memory for voice intonation and fundamental frequency. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 521-33.
32. Clark, B. & Graybiel, A. (1966). Influence of contact cues on the perception of the oculogravic illusion. US Navy Aerospace Medical Institute & National Aeronautics & Space Administration Joint Report.
33. Clements, G. N. (1985). The geometry of phonological features. Phonological Yearbook, 2, 225-52.
34. Cobb, V. & Lewis, C. C. (2001). Perk Up Your Ears: Discovering Your Sense of Hearing. New York: Millbrook Press.
35. Cohen, L. G., Celnik, P., Pascual-Leone, A., Faiz, L., Corwell, B., Honda, M., Sadato, H., Gerloff, C., Catala, M. D., & Hallett, M. (1997). Functional relevance of crossmodal plasticity in the blind. Nature, 389, 180-2.
36. Corballis, M. C. (2002). From Hand to Mouth: The Origins of Language. Princeton: Princeton University Press.
37. Diehl, R. L. & Kluender, K. R. (1989). On the objects of speech perception. Ecological Psychology, 1, 121-44.
38. Diehl, R. L., Souther, A. F., & Convis, C. L. (1980). Conditions on rate normalization in speech perception. Perception & Psychophysics, 27, 435-43.
39. Diesch, E. (1995). Left and right hemifield advantages of fusions and combinations in audiovisual speech perception. Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 48, 320-33.
40. Donald, M. (1991). Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition. Cambridge, MA: Harvard University Press.
41. Eimer, M. (2001). Crossmodal links in spatial attention between vision, audition, and touch: Evidence from event-related brain potentials. Neuropsychologia, 39, 1292-1303.
42. Ellis, H. D. (1989). Processes underlying face recognition. In R. Bruyer (ed.), Neuropsychology of Face Perception and Facial Expression (pp. 1-27). Hillsdale, NJ: Lawrence Erlbaum.
43. Fellowes, J. M., Remez, R. E., & Rubin, P. E. (1997). Perceiving the sex and identity of a talker without natural vocal timbre. Perception & Psychophysics, 59, 839-49.
44. Fodor, J. (1983). The Modularity of Mind. Cambridge, MA: MIT Press.
45. Fowler, C. A. (1986). An event approach to the study of speech perception from a direct-realist perspective. Journal of Phonetics, 14, 3-28.
46. Fowler, C. A. (1989). Real objects of speech perception: A commentary on Diehl and Kluender. Ecological Psychology, 1, 145-60.
47. Fowler, C. A. (1996). Listeners do hear sounds, not tongues. Journal of the Acoustical Society of America, 99, 1730-41.
48. Fowler, C. A. & Dekle, D. J. (1991). Listening with eye and hand: Cross-modal contributions to speech perception. Journal of Experimental Psychology: Human Perception and Performance, 17, 816-28.
49. Fowler, C. A. & Rosenblum, L. D. (1991). Perception of the phonetic gesture. In I. G. Mattingly & M. Studdert-Kennedy (eds.), Modularity and the Motor Theory of Speech Perception (pp. 33-59). Hillsdale, NJ: Lawrence Erlbaum.
50. Geers, A. & Brenner, C. (1994). Speech perception results: Audition and lipreading enhancement. Volta Review, 96, 97-108.
51. Ghanzafar, A. A. & Logothetis, N. K. (2003). Facial expressions linked to monkey calls. Nature, 423, 937-8.
52. Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Boston, MA: Houghton Mifflin.
53. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston, MA: Houghton-Mifflin.
54. Goldinger, S. D. (1992). Words and voices: Implicit and explicit memory for spoken words. Research on Speech Perception Technical Report No. 7. Bloomington, IN: Speech Research Laboratory, Indiana University.
55. Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105, 251-79.
56. Gordon, M. S. & Rosenblum, L. D. (2001). Audiovisual Speech Web-Lab: An internet teaching and research laboratory. Behavioral Research Methods, Instruments, & Computers, 33, 267-9.
57. Grant, K. W. & Seitz, P.-F. (2000). The use of visible speech cues for improving auditory detection of spoken sentences. Journal of the Acoustical Society of America, 108, 1197-1208.
58. Green, K. P. (1994). The influence of an inverted face on the McGurk effect. Poster presented at the Spring, 1994, meeting of the Acoustical Society of America, Cambridge, MA. Journal of the Acoustical Society of America, 95, 3014 (abstract).
59. Green, K. P. (1998). The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. In R. Campbell & B. Dodd (eds.), Hearing by Eye II: Advances in the Psychology of Speechreading and Audiovisual Speech (pp. 3-25). Hove, UK: Psychology Press.
60. Green, K. P. & Gerdman, A. (1995). Cross-modal discrepancies in coarticulation and the integration of speech information: The McGurk effect with mismatched vowels. Journal of Experimental Psychology: Human Perception and Performance, 21, 1409-26.
61. Green, K. P. & Kuhl, P. K. (1989). The role of visual information in the processing of place and manner features in speech perception. Perception & Psychophysics, 45, 34-42.
62. Green, K. P. & Miller, J. L. (1985). On the role of visual rate information in phonetic perception. Perception & Psychophysics, 38, 269-76.
63. Green, K. P. & Norrix, L. (2001). The perception of /r/and /l/in a stop cluster: Evidence of cross-modal context effects. Journal of Experimental Psychology: Human Perception and Performance, 27, 166-77.
64. Green, K. P., Kuhl, P. K., Meltzoff, A. M., & Stevens, E. B. (1991). Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect. Perception & Psychophysics, 50, 524-36.
65. Hellige, J. B. (1993). Hemispheric Asymmetry: What's Right and What's Left. Cambridge, MA: Harvard University Press.
66. Hewes, G. W. (1992). Primate communication and the gestural origin of language. Current Anthropology, 33, 65-84.
67. Jenkins, J. J., Strange, W., & Edman, T. R. (1983). Identification of vowels in "vowelless" syllables. Perception & Psychophysics, 34, 441-50.
68. Jordan, T. R. & Bevan, K. (1997). Seeing and hearing rotated faces: Influences of facial orientation on visual and audio-visual speech recognition. Journal of Experimental Psychology: Human Perception and Performance, 23, 388-403.
69. Kaiser, A. R., Kirk, K. I., Lachs, L., & Pisoni, D. B. (2003). Talker and lexical effects on audiovisual word recognition by adults with cochlear implants. Journal of Speech, Language, & Hearing Research, 46, 390-404.
70. Kamachi, M., Harold, H., Karen, L., & Vatikiotis-Bateson, E. (2003). Putting the face to the voice: Matching identity across modality. Current Biology, 13, 1709-14.
71. Kinney, J. A. & Luria, S. M. (1970). Conflicting visual and tactual-kinesthetic stimulation. Perception & Psychophysics, 8, 189-92.
72. Kishon-Rabin, L., Boothroyd, A., & Hanin, L. (1996). Speechreading enhancement: A comparison of spatial-tactile display of voice fundamental frequency (F0) with auditory F0. Journal of the Acoustical Society of America, 100, 593-602.
73. Kitagawa, N. & Ichihara, S. (2002). Hearing visual motion in depth. Nature, 416, 172-4.
74. Kluender, K. R. (1994). Speech perception as a tractable problem in cognitive science. In M. A. Gernsbacher (ed.), Handbook of Psycholinguistics (pp. 173-217). San Diego, CA: Academic Press.
75. Knight, B. & Johnston, A. (1997). The role of movement in face recognition. Visual Cognition, 4, 265-73.
76. Kuhl, P. K. & Meltzoff, A. N. (1984). The intermodal representation of speech in infants. Infant Behavior and Development, 7, 361-81.
77. Lachs, L. (1999). A voice is a face is a voice: Cross-modal source identification of indexical information in speech. Research on Spoken Language Processing Progress Report No. 23, Indiana University, pp. 241-58.
78. Lachs, L. (2002). Vocal tract kinematics and crossmodal speech information. Research on Spoken Language Processing Technical Report No. 10. Bloomington, IN: Speech Research Laboratory, Indiana University.
79. Lachs, L. & Pisoni, D. B. (2004). Crossmodal source information and spoken word recognition. Journal of Experimental Psychology: Human Perception and Performance, 30, 378-96.
80. Lachs, L. & Pisoni, D. B. (In press). Crossmodal source identification in speech perception. Ecological Psychology.
81. Lachs, L., Pisoni D. B., & Kirk, K. I. (2001). Use of audiovisual information in speech perception by prelingually deaf children with cochlear implants: A first report. Ear and Hearing, 22, 236-51.
82. Lander, K., Christie, F., & Bruce, V. (1999). The role of movement in the recognition of famous faces. Memory & Cognition, 27, 974-85.
83. Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y. F., Field, A. S., & Stein, B. E. (2002). Deactivitation of sensory-specific cortex by cross-modal stimuli. Journal of Cognitive Neuroscience, 14, 420-9.
84. Lee, D. N. (1976). A theory of visual control of braking based on information about time-to-collision. Perception, 5, 437-59.
85. Liberman, A. M. & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1-36.
86. Liberman A. M. & Whalen D. H. (2000). On the relation of speech to language. Trends In Cognitive Science, 4, 187-96.
87. Macaluso, E., Frith, C. D., & Driver, J. (2000). Modulation of human visual cortex by crossmodal spatial attention. Science, 289, 1206-8.
88. MacLeod, A. & Summerfield, Q. (1990). A procedure for measuring auditory and audio-visual speech reception thresholds for sentences in noise: Rationale, evaluation, and recommendations for use. British Journal of Audiology, 24, 29-43.
89. MacNeilage, P. F. (1998). The frame/content theory of evolution of speech production. Behavioral & Brain Sciences, 21, 499-546.
90. MacSweeney, M., Amaro, E., Calvert, G. A., & Campbell, R. (2000). Silent speechreading in the absence of scanner noise: An event-related fMRI study. NeuroReport, 11, 1729-33.
91. MacSweeney, M., Calvert, G. A., Campbell, R., et al. (2002). Speechreading circuits in people born deaf. Neuropsychologia, 40, 801-7.
92. Marks, L. (1978). The Unity of the Senses. New York: Academic Press.
93. Massaro, D. W. (1987). Speech perception by ear and eye. In B. Dodd & R. Campbell (eds.), Hearing by Eye: The Psychology of Lip-Reading (pp. 53-83). Hillsdale, NJ: Lawrence Erlbaum.
94. Massaro, D. W. & Cohen, M. M. (1990). Perception of synthesized audible and visible speech. Psychological Science, 1, 55-63.
95. Massaro, D. W. & Cohen, M. M. (1996). Perceiving speech from inverted faces. Perception & Psychophysics, 58, 1047-65.
96. Massaro, D. W., Cohen, M. M., Gesi, A., Heredia, R., & Tsuzaki, M. (1993). Bimodal speech perception: An examination across languages. Journal of Phonetics, 21, 445-78.
97. McGurk, H. & MacDonald, J. W. (1976). Hearing lips and seeing voices. Nature, 264, 746-8.
98. Meltzoff, A. N. & Moore, M. K. (1977). Imitation of facial and manual gestures by human neonates. Science, 198, 75-8.
99. Meltzoff, A. N. & Moore, M. K. (1983). The origins of imitation in infancy: Paradigm, phenomena, and theories. Advances in Infancy Research, 2, 265-301.
100. Meredith, M. A. (2002). On the neuronal basis for multisensory convergence: a brief overview. Cognitive Brain Research, 14, 31-40.
101. Mergner, T. & Rosemeier, T. (1998). Interaction of vestibular, somatosensory and visual signals for postural control and motion perception under terrestrial and microgravity conditions-a conceptual model. Brain Research Reviews, 28, 118-35.
102. Mesulam, M. M. (1998). From sensation to cognition. Brain, 121, 1013-52.
103. Mills, A. E. (1987). The development of phonology in the blind child. In B. Dodd & R. Campbell (eds.), Hearing by Eye: The Psychology of Lip Reading (pp. 145-62). Hillsdale, NJ: Lawrence Erlbaum.
104. Montgomery, A. A. & Jackson, P. L. (1983). Physical characteristics of the lips underlying vowel lipreading performance. Journal of the Acoustical Society of America, 73, 2134-44.
105. Mottonen, R., Krause, C. M., Tiippana, K., & Sams, M. (2002). Processing of changes in visual speech in the human auditory cortex. Cognitive Brain Research, 13, 417-25.
106. Mullennix, J. W., Pisoni, D. B., & Martin, C. S. (1989). Some effects of talker variability on spoken word recognition. Journal of the Acoustical Society of America, 92, 1085-99.
107. Munhall, K. G. & Vatikiotis-Bateson, E. (1998). The moving face during speech communication. In R. Campbell, B. Dodd, & D. Burnham (eds.), Hearing by Eye II: The Psychology of Speechreading and Audiovisual Speech (pp. 123-39). Hove, UK: Psychology Press.
108. Nygaard, L. C. & Pisoni, D. B. (1998). Talker-specific learning in speech perception. Perception & Psychophysics, 60, 355-76.
109. Nygaard, L. C., Sommers, M. S., & Pisoni, D. B. (1994). Speech perception as a talker-contingent process. Psychological Science, 5, 42-6.
110. Ohala, J. J. (1984). An ethological perspective on common cross-language utilization of F0 of voice. Phonetica, 41, 1-16.
111. Ohala, J. J. (1996). Speech perception is hearing sounds, not tongues. Journal of the Acoustical Society of America, 99, 1718-25.
112. O'Leary, A. & Rhodes, G. (1984). Cross-modal effects on visual and auditory object perception. Perception & Psychophysics, 35, 565-9.
113. Palmeri, T. J., Goldinger, S. D., & Pisoni, D. B. (1993). Episodic encoding of voice attributes and recognition memory for spoken words. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 309-28.
114. Pisoni, D. B. (1997). Some thoughts on "normalization" in speech perception. In K. Johnson & J. Mullennix (eds.), Talker Variability in Speech Processing (pp. 9-32). San Diego, CA: Academic Press.
115. Radeau, M. & Bertelson, P. (1974). The after-effects of ventriloquism. Quarterly Journal of Experimental Psychology, 26, 63-71.
116. Rauschecker, J. P. & Korte, M. (1993). Auditory compensation for early blindness in cat cerebral cortex. Journal of Neuroscience, 13, 4538-48.
117. Reed, C. M., Delhorne, L. A., & Durlach, N. I. (1993). Historical overview in tactile aid research. In A. Risberg, S. Felicetti, & K.-E. Spens (eds.), Proceedings of the Second International Conference on Tactile Aids, Hearing Aids, and Cochlear Implants. Edsbruck, Sweden: Akademitryck AB.
118. Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd& R. Campbell (eds), Hearing by Eye: The Psychology of Lip Reading (pp. 97-113). Hillsdale, NJ: Lawrence Erlbaum.
119. Remez, R. E. (1986). Realism, language, and another barrier. Journal of Phonetics, 14, 89-97.
120. Remez, R. E. (1989). When the objects of perception are spoken. Ecological Psychology, 1, 161-80.
121. Remez, R. E., Fellowes, J. M., Pisoni, D. B., Goh, W. D., & Rubin, P. E. (1998). Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances. Speech Communication, 26, 65-73.
122. Remez, R. E., Fellowes, J. M., & Rubin, P. E. (1997). Talker identification based on phonetic information. Journal of Experimental Psychology: Human Perception & Performance, 23(3), 651-66.
123. Remez, R. E., Rubin, P. E., Nygaard, L. C., & Howell, W. A. (1987). Perceptual normalization of vowels produced by sinusoidal voices. Journal of Experimental Psychology: Human Perception & Performance, 13(1), 40-61.
124. Remez, R., Rubin, P., Pisoni, D., & Carrell, T. (1981). Speech perception without traditional speech cues. Science, 212, 947-50.
125. Rinne, T., Alho, K., Alku, P., Holi, M., Sinkkonen, J., Virtanen, J., Bertrand, O., & Naeaetaenen, R. (1999). Analysis of speech sounds is left-hemisphere predominant at 100-150 ms after sound onset. NeuroReport, 10; 1113-17.
126. Roberts, M. & Summerfield, Q. (1981). Audiovisual presentation demonstrates that selective adaptation in speech perception is purely auditory. Perception & Psychophysics, 30, 309-14.
127. Rosenblum, L. D. (2002). The perceptual basis of audiovisual speech integration. Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP), 1461-4.
128. Rosenblum, L. D. & Gordon, M. S. (2001). The generality of specificity: Some lessons from audiovisual speech. Behavioral & Brain Sciences, 24, 239-40.
129. Rosenblum, L. D. & Nichols, S. (2000). Recognizing speakers from visual speech information. Talk presented at the 41st meeting of the Psychonomics Society, New Orleans, LA, November 19.
130. Rosenblum, L. D. & Saldaňa, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318-31.
131. Rosenblum, L. D. & Saldaňa, H. M. (1998). Time-varying information for visual speech perception. In R. Campbell, B. Dodd, & D. Burnham (eds.), Hearing by Eye II: The Psychology of Speechreading and Audiovisual Speech (pp. 61-81). Hove, UK: Psychology Press.
132. Rosenblum, L. D. & Yakel, D. A. (2001). The McGurk effect from single and mixed speaker stimuli. Acoustic Research Letters Online, 2, 67-72.
133. Rosenblum, L. D., Johnson, J. A., & Saldaňa, H. M. (1996). Visual kinematic information for embellishing speech in noise. Journal of Speech & Hearing Research, 39, 1159-70.
134. Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997). The McGurk effect in infants. Perception & Psychophysics, 59, 347-57.
135. Rosenblum, L. D., Smith, N. M., Nichols, S., Lee, J., & Hale, S. (under review). Hearing a face: Cross-modal speaker matching using isolated visible speech. Perception & Psychophysics.
136. Rosenblum, L. D., Smith, N. M., & Niehus, R. P. (in preparation). Look who's talking: Identifying friends from visible articulation. Psychological Science.
137. Rosenblum, L. D., Yakel, D. A., Baseer, N., Panchal, A., Nordarse, B. C., & Niehus, R. P. (2002). Visual speech information for face recognition. Perception & Psychophysics, 64, 220-9.
138. Rosenblum, L. D., Yakel, D. A., & Green, K. G. (2000). Face and mouth inversion effects on visual and audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 26, 806-19.
139. Sadato, N., Pascual-Leone, A., Grafman, J., Ibanez, V., Deiber, M. P., Dold, G., & Hallett, M. (1996). Activation of the primary visual cortex by Braille reading in blind subjects. Nature, 380, 526-8.
140. Saldaňa, H. M. & Rosenblum, L. D. (1994). Voice information in auditory form-based priming. Journal of the Acoustical Society of America, 95, 2870.
141. Sams, M., Aulanko, R., Hamalainen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T., & Simola, J. (1991). Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neuroscience Letters, 127, 141-5.
142. Schwartz, J. L., Robert-Ribes, J., & Esculier, P. (1998). Ten years after Summerfield: A taxonomy of models for audiovisual fusion in speech perception. In R. Campbell, B. Dodd, & D. Burnham (eds.), Hearing by Eye II: The Psychology of Speechreading and Audiovisual Speech (pp. 85-108). Hove, UK: Psychology Press.
143. Schweinberger, S. R. & Soukup, G. R. (1998). Asymmetric relationships among perceptions of facial identity, emotion, and facial speech. Journal of Experimental Psychology: Human Perception and Performance, 24, 1748-65.
144. Sekiyama, K. & Tohkura, Y. (1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America, 90, 1797-1805.
145. Sekiyama, K. & Tokhura, Y. (1993). Inter-language differences in the influence of visual cues in speech perception. Journal of Phonetics, 21, 427-44.
146. Shams, L., Kamitani, Y., & Shimojo, S. (2000). Illusions: What you see is what you hear. Nature, 408, 788.
147. Shams, L., Kamitani, Y., Thompson, S., & Shimojo, S. (2001). Sound alters visual evoked potentials in humans. NeuroReport, 12, 3849-52.
148. Shankweiler, D. & Studdert-Kennedy, M. (1975). A continuum of lateralization for speech perception. Brain and Language, 2, 212-25.
149. Shannon, R. V., Zeng, F.-G., Kamath, V., Wygonski, J., & Ekelid, M. (1995). Speech recognition with primarily temporal cues. Science, 270, 303-4.
150. Sheffert, S. M. & Fowler, C. A. (1995). The effects of voice and visible speaker change on memory for spoken words. Journal of Memory and Language, 34, 665-85.
151. Shimojo, S. & Shams, L. (2001). Sensory modalities are not separate modalities: Plasticity and interactions. Current Opinion in Neurobiology, 11, 505-9.
152. Smeele, P. M. T. (1996). Psychology of human speechreading. In D. Stork (ed.), Speechreading by Man and Machine: Models, Systems, and Applications (pp. 3-15). 1995 NATO ASI workshop, Chateau de Bonas, France: Springer Verlag.
153. Smeele, P. M. T., Massaro, D. W., Cohen, M. M., & Sittig, A. C. (1998). Laterality in visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 24, 1232-42.
154. Smith, N. M. & Rosenblum, L. D. (2003). Idiolectic information for cross-modal speaker recognition. Abstracts of the Psychonomic Society, 8, 14.
155. Sommers, M. S., Nygaard, L. C., & Pisoni, D. B. (1994). Stimulus variability and spoken word recognition I: Effects of variability in speaking rate and overall amplitude. Journal of the Acoustical Society of America, 96, 1314-24.
156. Stein, B. E. & Meredith, M. A. (1993). The Merging of the Senses. Cambridge, MA: MIT Press.
157. Stein, B. E., London, N., Wilkinson, L. K., & Price, D. D. (1996). Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis. Journal of Cognitive Neuroscience, 8, 497-506.
158. Stevens, K. N. (1989). On the quantal nature of speech. Journal of Phonetics, 17, 3-45.
159. Stoffregen, T. A. & Bardy, B. G. (2001). On specification and the senses. Behavioral & Brain Sciences, 24, 195-261.
160. Strange, W., Jenkins, J. J., & Johnson, T. L. (1983). Dynamic specification of coarticulated vowels. Journal of the Acoustical Society of America, 74, 695-705.
161. Strange, W., Verbrugge, R. R., Shankweiler, D. P., & Edman, T. R. (1976). Consonant environment specifies vowel identity. Journal of the Acoustical Society of America, 60, 213-24.
162. Studdert-Kennedy, M. (1989). Cues to what? A comment on Diehl and Kluender "On the objects of speech perception." Ecological Psychology, 1, 181-93.
163. Sumby, W. H. & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-15.
164. Summerfield, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (eds.), Hearing by Eye: The Psychology of Lip Reading (pp. 53-83). Hillsdale, NJ: Lawrence Erlbaum.
165. Valentine, T. & Bruce, V. (1988). Mental rotation of faces. Memory & Cognition, 16, 556-66.
166. Vroomen, J. & de Gelder, B. (2000). Sound enhances visual perception: Cross-modal effects of auditory organization on vision. Journal of Experimental Psychology: Human Perception and Performance, 26, 1583-90.
167. Walden, B. E., Prosek, R. A., & Worthington, D. W. (1975). Auditory and audiovisual feature transmission in hearing-impaired adults. Journal of Speech & Hearing Research, 18, 272-80.
168. Walker, A. S. (1982). Intermodal perception of expression behaviors by human infants. Journal of Experimental Child Psychology, 33, 514-35.
169. Walker, J. T. & Scott, K. J. (1981). Auditory-visual conflicts in the perceived duration of lights, tones, and gaps. Journal of Experimental Psychology: Human Perception and Performance, 7, 1327-39.
170. Walker, S., Bruce, V., & O'Malley, C. (1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect. Perception & Psychophysics, 57, 1124-33.
171. Walker-Andrews, A. S. & Lennon, E. M. (1985). Auditory-visual perception of changing distance by human infants. Child Development, 56, 544-8.
172. Welch, R. B. & Warren, D. H. (1986). Intersensory interactions. In K. R. Boff, L. Kaufman, & J. P. Thomas (eds.), Handbook of Perception and Human Performance. Vol. 1: Sensory Processes and Perception (pp. 25.125.36). New York: John Wiley & Sons.
173. Williams, D. R., Verbrugge, R. R., & Studdert-Kennedy, M. (1983). Judging sine wave stimuli as speech and nonspeech. Journal of the Acoustical Society of America, 74, S66.
174. Yakel, D. A. (2000). Effects of time-varying information on vowel identification accuracy in visual speech perception. Doctoral dissertation, University of California, Riverside.
175. Yakel, D. A., Rosenblum, L. D., & Fortier, M. A. (2000). Effects of talker variability on speechreading. Perception & Psychophysics, 62, 1405-12.
176. Yakel, D. A., Rosenblum, L. D., Smith, N., & Burns, D. (in preparation). Salience of time varying information for visual speech perception. To be submitted to Perception & Psychophysics.
177. Yehia, H. C., Rubin, P. E., & Vatikiotis-Bateson, E. (1998). Quantitative association of vocal-tract and facial behavior. Speech Communication, 26, 23-43.