Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds

Psychological Research

Volumn 71, Issue 1, 2007, Pages 4-12

  Navarra, Jordi ^a   Soto Faraco, Salvador ^a  

^a UNIVERSITY OF BARCELONA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; HUMAN; LEARNING; LIP; MOVEMENT (PHYSIOLOGY); PHYSIOLOGY; SPEECH AUDIOMETRY; SPEECH PERCEPTION; VISION;

HUMANS; LIP; MOVEMENT; SPEECH DISCRIMINATION TESTS; SPEECH PERCEPTION; VERBAL LEARNING; VISUAL PERCEPTION;

EID: 33846355606     PISSN: 03400727     EISSN: 14302772     Source Type: Journal    
DOI: 10.1007/s00426-005-0031-5     Document Type: Article

References (55)

1. Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15, 839-843.
2. Arnold, P., & Hill, F. (2001). Bisensory augmentation: a speech-reading advantage when speech is clearly audible and intact. British Journal of Psychology, 92, 339-355.
3. Auer, E. T., & Bernstein, L. (1997). Speechreading and the structure of the lexicon: computationally modeling the effects of reduced phonetic distinctiveness on lexical uniqueness. Journal of the Acoustical Society of America, 102, 3704-3710.
4. Best, C. (1995). A direct realist view of cross-language speech perception. In W. Strange (Ed.). Speech perception and linguistic experience. Issues in cross-language research(pp. 171-204) Baltimore, MD: York Press.
5. Bosch, L., Costa, A., & Sebastian-Gallés, N. (2000). First and second language vowel perception in early bilinguals. European Journal of Cognitive Psychology, 12, 189-222.
6. Burnham, D. K. (1998) Language specificity in the development of auditory-visual speech perception. In R. Campbell, B. Dodd& D.Burnham (Eds.) Hearing by eye II: advances in the psychology of speechreading and auditory-visual speech(pp. 27-60). Hove, UK: Psychology Press.
7. Callan, D. E., Jones, J. A., Munhall, K., Callan, A. M., Cross, C., & Vatikiotis-Bateson, E. (2003). Neural processes underlying perceptual enhancement by visual speech gestures. Neuroreport, 14, 2213-2218.
8. Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., & David, A. S. (1999). Response amplification in sensory-specific cortices during crossmodal binding. Neuroreport, 10, 2619-2623.
9. Calvert, G., Spence, C.,& Stein, B. E. (Eds.). (2004). The handbook of multisensory processes. Cambridge: MIT Press.
10. Campbell, R. (2004). Audiovisual speech processing. In K. Brown (Ed.). The Encyclopedia of language and linguistics, 2nd edition - Editor-in-chief Keith Brown, Oxford, UK: Elsevier.
11. Colin, C., Radeau, M., Soquet, A., Demolin, D., Colin, F., & Deltenre, P. (2002). Mismatch negativity evoked by the McGurk-MacDonald effect: a phonetic representation within short-term memory. Clinical Neurophysiology, 113, 495-506.
12. Davis, C., & Kim, J. (2004). Audio-visual interactions with intact clearly audible speech. Quarterly Journal of Experimental Psychology, 57, 1103-1121.
13. Ernst, M. O., & Banks, M. S. (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.
14. Ernst, M. O., & Bulthoff, H. H. (2004). Merging the senses into a robust percept TRENDS in Cognitive Sciences, 8, 162-169.
15. Fadiga, L., Craighero, L., Buccino, G., & Rizzolatti, G. (2002). Speech listening specifically modulates the excitability of tongue muscles: a TMS study. European Journal of Neuroscience, 15, 399-402.
16. Flege, J. (1995). Second language speech learning: Theory, findings and problems. In W. Strange (Ed.). Speech perception and linguistic experience. Issues in cross-language research (pp. 233-272), Baltimore, MD: York Press.
17. Forster, K. L., & Forster, J. C. (2003). DMDX: a windows display program with millisecond accuracy. Behavioural Research Methods, Instruments,& Computers, 35, 116-124.
18. Garner, W. R. (1978). Selective attention to attributes and to stimuli. Journal of Experimental Psychology: General, 107, 287-308.
19. Ghazanfar, A. A., & Logothetis, N. K. (2003). Facial expressions linked to monkey calls. Nature, 423, 937-938.
20. Hommel, B. (in press). Feature integration across perception and action: Event files affect response choice. Psychological Research, Psychologische Forschung.
21. Kerzel, D. (2002). Evidence for effects of phonological correspondence between visible speech and written syllables. Psychological Research, 66, 195-200.
22. Kerzel, D., & Bekkering, H. (2000). Motor activation from visible speech: evidence from stimulus response compatibility. Journal of Experimental Psychology. Human Perception and Performance, 26, 634-647.
23. Klucharev, V., Möttönen R., & Sams, M. (2003). Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. Cognitive Brain Research, 18, 65-75.
24. Kuhl, P. K. (1991). Human adults and human infants show a "perceptual magnet effect" for the prototypes of speech categories, monkeys do not. Perception and Psychophysics, 50, 93-107.
25. Kuhl, P. K., & Meltzoff, A. N. (1982). The bimodal perception of speech in infancy. Science, 218, 1138-1141.
26. Kuhl, P. K., Tsao, F. M., & Liu, H. M. (2003). Foreign-language experience in infancy: effects of short-term exposure and social interaction on phonetic learning. Proceedings of the National Academy of Sciences USA, 100, 9096-9101.
27. Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431-461.
28. Massaro, D. W. (1987). Speech perception by ear and eye. In B. Dodd& R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 53-83). London, England UK: Lawrence Erlbaum Associates, Inc.
29. Massaro, D. W. (1998). Perceiving talking faces. Cambridge, MA: MIT Press
30. Massaro, D. W., Tsuzaki, M., Cohen, M., Gesi, A., & Heredia, R. (1993). Bimodal speech perception: An examination across languages. Journal of Phonetics, 21:445-478
31. Mattys, S. L., Bernstein, L. E., & Auer, E. T. (2002). Stimulusbased lexical distinctiveness as a general word-recognition mechanism. Perception & Psychophysics, 64, 667-679.
32. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 265, 746-748.
33. Mehler, J., Sebastián-Gallés, N., & Nespor, M., (2004). Biological Foundation of Language Acquisition: Evidence from Bilingualism. In M. S. Gazzaniga (Ed.), Cognitive Neurosciences III (pp. 825-836). Cambridge, MA: MIT press.
34. Molholm, S., Ritter, W., Javitt, D. C., & Foxe, J. J. (2004). Multisensory visual-auditory object recognition in humans: A high-density electrical mapping study. Cerebral Cortex, 14, 452-465.
35. Navarra, J., Sebastián-Gallés, N., & Soto-Faraco, S. (2005). The perception of second language sounds in early bilinguals: New evidence from an implicit measure. Journal of Experimental Psychology: Human Perception and Performance, 31(5), 912.
36. Pallier, C., Colomé, A., & Sebastián-Gallés, N. (2001). The influence of native-language phonology on lexical access: Exemplar-based versus abstract lexical entries. Psychological Science, 12, 445-449.
37. Prinz, W. (1990). A common coding approach to perception and action. In O. Neumann & W. Prinz (Eds.), Relationships between perception and action: Current approaches (pp. 167-201). Berlin: Springer.
38. Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.). Hearing by eye: The psychology of lip-reading. Hillsdale: LEA
39. Sebastián-Gallés, N., & Baus, C. (2005). On the relationship between perception and production in L2 categories. In A. Cutler (Ed.) Twenty-First Century Psycholinguistics. Hillsdale: Erlbaum.
40. Sebastian-Gallés, N., & Soto-Faraco, S. (1999). Online processing of native and non-native phonemic contrasts in early bilinguals. Cognition, 72, 111-123.
41. Sekiyama K., & Tohkura Y. (1993) Inter-language differences in the influence of visual cues in speech perception. Journal of Phonetics, 21, 427-444.
42. Soto-Faraco, S., Navarra, J., & Alsius, A. (2004). Assessing automaticity in audiovisual speech integration: Evidence from the speeded classification task. Cognition, 92, B13-B23.
43. Soto-Faraco, S., Navarra, J., Voloumanos, A., Sebastián- Gallés, N., Weikum, & Werker, J. F. (submitted). Discriminating languages by speechreading. Perception and Psychophysics.
44. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: The MIT Press.
45. Strange, W. (1995). Cross-language studies of speech perception. A historical review. In W. Strange (Ed.) Speech perception and linguistic experience. Issues in cross-language research (pp. 3-45). Baltimore, MD: York Press.
46. Sumby, W., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
47. Summerfield, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd, R. Campbell (Eds.). Hearing by Eye: The Psychology of Lip-Reading (pp. 3-51). London, UK: LEA.
48. Tomiak, G. R., Mullennix, J. W., & Sawusch, J. R. (1987). Integral processing of phonemes: Evidence for a phonetic mode of perception. Journal of the Acoustical Society of America, 81, 755-764.
49. Van Wassenhove, V., Grant, K.W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the Natural Academy of Sciences USA, 102(4), 1181-1186.
50. Watkins, K. E., & Paus, T. (2004). Modulation of motor excitability during speech perception: The role of Broca's area. Journal of Cognitive Neuroscience, 16(6), 978-987.
51. Watkins, K. E., Strafella, A.P., & Paus, T. (2003). Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia, 47(8), 989-994.
52. Werker, J. F., & Tees, R. C. (1983). Developmental changes across childhood in the perception of non-native speech sounds. Canadian Journal of Psychology, 37(2), 278-286.
53. Werker, J. F., Gilbert, J. H. V., Humphrey, K., & Tees, R.C. (1981). Developmental aspects of cross-language speech perception. Child Development, 52, 349-355.
54. Wilson, S. M., Pinar Saygin, A., Sereno, M. I., & Iacobini, M. (2004). Listening to speech activates motor areas involved in speech production. Nature, 7(7), 701-702.
55. Wright, T. M., Pelphrey, K. A., Allison, T., McKeown, M. J., & McCarthy, G. (2003). Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cerebral Cortex, 13, 1034-1043.