Visual influences on perception of speech and nonspeech vocal-tract events

Language and Speech

Volumn 49, Issue 1, 2006, Pages 21-53

  Brancazio, Lawrence ^a,b   Best, Catherine T ^b,c   Fowler, Carol A ^b,d,e  

^a SOUTHERN CONNECTICUT STATE UNIVERSITY   (United States)

^b HASKINS LABORATORIES   (United States)

^c UNIVERSITY OF WESTERN SYDNEY   (Australia)

^d UNIVERSITY OF CONNECTICUT   (United States)

^e YALE UNIVERSITY   (United States)

Author keywords

Audiovisual speech perception; Clicks; McGurk effect; Nonspeech

Indexed keywords

EID: 33747131028     PISSN: 00238309     EISSN: None     Source Type: Journal    
DOI: 10.1177/00238309060490010301     Document Type: Conference Paper

References (51)

1. BEST, C. T. (1984). Discovering messages in the medium: Speech and the prelinguistic infant. In H. E. Fitzgerald, B. Lester, & M. Yongman (Eds.), Advances in pediatric psychology, Vol.2 (pp.97-145). New York: Plenum.
2. BEST, C. T. (1994). The emergence of native-language phonological influences in infants: A perceptual assimilation model. In J. C. Goodman & H. C. Nusbaum (Eds.), The development of speech perception: The transition from speech sounds to spoken words (pp. 167-224). Cambridge: MIT Press.
3. BEST, C. T. (1995). A direct realist perspective on cross-language speech perception. In W. Strange (Ed.), Cross-language speech perception (pp. 171-204). Baltimore: York Press.
4. BEST, C. T., & AVERY, R. A. (1999). Left hemisphere advantage for click consonants is determined by linguistic significance. Psychological Science, 10, 65-70.
5. BEST, C. T., McROBERTS, G. W., & SITHOLE, N. M. (1988). Examination of perceptual reorganization for non-native speech contrasts: Zulu click discrimination by English-speaking adults and infants. Journal of Experimental Psychology: Human Perception and Performance, 14, 345-360.
6. CALVERT, G. A. (2001). Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex, 11, 1110-1123.
7. CALVERT, G. A., BULLMORE, E. T., BRAMMER, M. J., CAMPBELL, R., WILLIAMS, S. C. R., McGUIRE, P. K., WOODRUFF, P. W. R., IVERSEN, S. D., & DAVID, A. S. (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593-596.
8. CALVERT, G. A., & CAMPBELL, R. (2003). Reading speech from still and moving faces: The neural substrates of visible speech. Journal of Cognitive Neuroscience, 15, 57-70.
9. CALVERT, G. A., CAMPBELL, R., & BREMMER, M. J. (2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology, 10, 649-657.
10. CAMPBELL, R., MacSWEENEY, M., SURGULADZE, S., CALVERT, G., McGUIRE, P., SUCKLING, J., BRAMMER, M. J., & DAVID, A. S. (2001). Cortical substrates for the perception of face actions: An fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning), Cognitive Brain Research, 12, 233-243.
11. De GELDER, B., & VROOMEN, J. (2000). The perception of emotions by ear and eye. Cognition and Emotion, 14, 289-311.
12. DIEHL, R. L., & KLUENDER, K. R. (1989). On the objects of speech perception. Ecological Psychology, 1, 121-144.
13. DRIVER, J. (1996). Enhancement of selective listening of illusory mislocation of speech sounds due to lip-reading. Nature, 381, 66-68.
14. FANT, G. (1973). Speech sounds and features. Cambridge, MA: MIT Press.
15. FODOR, J. A. (1983). The modularity of mind: An essay on faculty psychology. Cambridge, MA: MIT Press.
16. FOWLER, C. A. (1986). An event approach to the study of speech perception from a direct-realist perspective. Journal of Phonetics, 14, 3-28.
17. FOWLER, C. A. (1994). Speech perception: Direct realist theory. In Encyclopedia of language and linguistics (pp.4199-4203). Oxford: Pergamon Press.
18. FOWLER, C. A., & DEKLE, D. J. (1991). Listening with eye and hand: Crossmodal contributions to speech perception. Journal of Experimental Psychology: Human Perception and Performance, 17, 816-828.
19. GAVER, W. W. (1993). What in the world do we hear? An ecological approach to auditory event perception. Ecological Psychology, 5, 1-29.
20. GIBSON, J. J. (1966). The senses considered as perceptual systems. Boston, MA: Houghton-Mifflin.
21. GIBSON, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.
22. GREEN, K. P., KUHL, P. K., MELTZOFF, A. N., & STEVENS. E. B. (1991). Integrating speech information across talkers, genders and sensory modality: Female faces and male voices in the McGurk effect. Perception and Psychophyxics, 50, 524-536.
23. GREEN, K. P., & NORRIX, L. W. (1997). Acoustic cues to place of articulation and the McGurk effect: The role of release bursts, aspiration, and formant transitions. Journal of Speech, Language, and Hearing Research, 40, 646-665.
24. GROSSMAN, E., DONNELLY, M., PRICE, R., PICKENS, D., MORGAN, V., NEIGHBOR, G., & BLAKE, R. (2000). Brain areas involved in perception of biological motion. Journal of Cognitive Neuroscience, 12, 711-720.
25. JONES, J. A., & CALLAN, D. E. (2003). Brain activity during audiovisual speech perception: An fMRI study of the McGurk effect. NeuroReport, 14, 1129-1133.
26. LADEFOGED, P., & MADDIESON, I. (1996). The sounds of the world's languages. Oxford, U.K.: Blackwell Publishers.
27. LADEFOGED, P., & TRAILL, A. (1994). Clicks and their accompaniments. Journal of Phonetics, 22, 33-64.
28. LIBERMAN, A. M., DELATTRE, P. C., COOPER, F. S., & GERSTMAN, L. J. (1954). The role of consonant-vowel transitions in the perception of the stop and nasal consonants. Psychological Monographs, 68, 1-13.
29. LIBERMAN, A. M., & MATTINGLY, I. G. (1985). The motor theory revised. Cognition, 21, 1-36.
30. MacDONALD, J., & McGURK., H. (1978). Visual influences on speech perception. Perception and Psychophysics, 24, 253-257.
31. MANUEL, S. Y., REPP, B. H., LIBERMAN, A. M., & STUDDERT-KENNEDY, M. (1983) Exploring the "McGurk effect." Talk presented at the 24th meeting of the Psychonomic Society, November, 1983.
32. MASSARO, D. W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Lawrence Erlbaum Associates.
33. MASSARO, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
34. MASSARO, D. W., & COHEN, M. M. (1993) Perceiving asynchronous bimodal speech in consonant-vowel and vowel syllables. Speech Communication, 13, 127-134.
35. MASSARO, D. W., & EGAN, P. B. (1996). Perceiving affect from the voice and face. Psychonomic Bulletin and Review, 3, 215-221.
36. McGURK., H., & MacDONALD, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
37. MÖTTÖNEN, R., KRAUSE, C. M., TIIPPANA, K., & SAMS, M. (2002). Processing of changes in visual speech in the human auditory cortex. Cognitive Brain Research, 13, 417-425.
38. OLSON, I. R., GATENBY, J. C., & GORE, J. C. (2002). A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Cognitive Brain Research, 14, 129-138.
39. PAULESU, E., PERANI, D., BLASI, V., SILANI, G., BORGHESE, N. A., de GIOVANNI, U., SENSOLO, S., & FAZIO, F. (2003). A functional-anatomical model for lipreading. Journal of Neurophysiology, 90, 2005-2013.
40. PUCE, A., ALLISON, T., BENTIN, S., GORE, J. C., & McCARTHY, G. (1998). Temporal cortext activation in humans viewing eye and mouth movements. Journal of Neuroscience, 18, 2188-2199.
41. RADEAU, M. (1994). Auditory-visual spatial interactions and modularity. Cahiers de Psychologie Cognitive, 13, 3-51.
42. RIZZOLATTI, G., FOGASSI, L., & GALLESE, V. (2002). Motor and cognitive functions of the ventral premotor cortex. Current Opinion in Neurobiology, 12, 149-154.
43. ROSENBLUM, L. D. (1987). Towards an ecological alternative to the motor theory. Perceiving-Acting Workshop, 2, 25-28.
44. ROSENBLUM, L. D. (1994). How special is audiovisual speech integration? Cahiers de Psychologie Cognitive, 13, 110-116.
45. ROSENBLUM, L. D., & FOWLER, C. A. (1991). An audio-visual investigation of the loudness/effort effect for speech and nonspeech events. Journal of Experimental Psychology: Human Perception and Performance, 17, 976-985.
46. ROSENBLUM, L. D., & SALDANA, H. M. (1992). Discrimination tests of visually influenced syllables. Perception and Psychophysics, 52, 461-473.
47. ROSENBLUM, L. D., & SALDANA, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318-331.
48. RUBIN, P. E. (1995). HADES: A case study on the development of a signal analysis system. In A. Syrdal, R. Bennett, & S. Greenspan (Ed.), Applied signal technology, (pp.501-520) Florida: CRC Press.
49. SALDANA, H. M., & ROSENBLUM, L. D. (1993). Visual influences on auditory pluck and bow judgments. Perception and Psychophysics, 54, 406-416.
50. SUMMERFIELD, Q., & McGRATH, M. (1984). Detection and resolution of audio-visual incompatibility in the perception of vowels. Quarterly Journal of Experimental Psychology, 36A, 51-74.
51. WELCH, R. B., & WARREN, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638-667.