Linguistic experience and audio-visual perception of non-native fricatives

Journal of the Acoustical Society of America

Volumn 124, Issue 3, 2008, Pages 1716-1726

  Wang, Yue ^a   Behne, Dawn M ^b   Jiang, Haisheng ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

^b NORWEGIAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Norway)

Author keywords

[No Author keywords available]

Indexed keywords

AUDIO ACOUSTICS; LINGUISTICS; SPEECH PROCESSING; VISUAL COMMUNICATION;

AUDIO-VISUAL; CANADIAN ENGLISH; L2 SPEECH; LEARNING THEORIES; MANDARIN CHINESE; OVERALL PERFORMANCE; POSITIVE EFFECTS; VISUAL INFORMATIONS; VISUAL SPEECH;

SPEECH;

ADULT; ARTICLE; AUDIOVISUAL EQUIPMENT; AUDITORY STIMULATION; CANADA; CHINESE; FEMALE; HUMAN; HUMAN EXPERIMENT; LANGUAGE; MALE; NOISE; PRIORITY JOURNAL; SOUND; SPEECH; SPEECH ARTICULATION; SPEECH PERCEPTION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDIOMETRY, SPEECH; AUDITORY PERCEPTION; CANADA; CHINA; CUES; FACIAL EXPRESSION; FEMALE; HUMANS; LEARNING; LIPREADING; MALE; MULTILINGUALISM; NOISE; PERCEPTUAL MASKING; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 52449108294     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.2956483     Document Type: Article

References (60)

1. Bernstein, L. E., Auer, E., and Takayanagi, S. (2004). " Auditory speech detection in noise enhanced by lipreading.," Speech Commun. 44, 5-18.
2. Best, C. T. (1995). " A direct realist view of cross-language speech perception.," in Speech Perception and Linguistic Experience: Issues in Cross-Language Research, edited by, W. Strange, (Baltimore, York), pp. 171-204.
3. Best, C. T., McRoberts, G. W., and Sithole, N. M. (1988). " Examination of perceptual reorganization for nonnative speech contrasts: Zulu click discrimination by English-speaking adults and infants.," J. Exp. Psychol. Hum. Percept. Perform. 14, 45-60.
4. Brooke, N. M., and Summerfield, Q. (1983). " Analysis, synthesis, and perception of visible articulatory movements.," J. Phonetics 11, 63-76.
5. Burnham, D., and Dodd, B. (1998). " Familiarity and novelty in infant cross-language studies: Factors, problems, and a possible solution.," in Advances in Infancy Research, edited by, C. Rovee-Collier, pp. 170-187.
6. Burnham, D., and Dodd, B. (2004). " Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk Effect.," Dev. Psychobiol. 45, 204-220.
7. Burnham, D., Lau, S., Tam, H., and Schoknecht, C. (2001). " Visual discrimination of Cantonese tone by tonal but non-Cantonese speakers, and by non-tonal language speakers.," in Proceedings of the International Conference on Auditory-Visual Speech Processing, edited by, D. Massaro, J. Light, and, K. Geraci, pp. 155-160.
8. Chen, Y., and Hazan, V. (2007a). " Developmental Factor in Auditory-Visual Speech Perception-The McGurk Effect in Mandarin-Chinese and English Speakers.," in Proceedings of the International Conference on Auditory-Visual Speech Processing, edited by, J. Vroomen, M. Swerts, and, E. Krahmer,.
9. Chen, Y., and Hazan, V. (2007b). " Language effects on the degree of visual influence in audiovisual speech perception.," in Proceedings of the 16th International Congress of Phonetic Sciences, Saarbrueken, Germany, pp. 2177-2180.
10. Chen, T., and Massaro, D. W. (2004). " Mandarin speech perception by ear and eye follows a universal principle.," Percept. Psychophys. 66, 820-836.
11. Cienkowski, K. M., and Carney, A. E. (2002). " Auditory-visual speech perception and aging.," Ear Hear. 23, 439-449.
12. Davis, C., and Kim, J. (2004). " Audio-visual interactions with intact clearly audible speech.," Q. J. Exp. Psychol. 57A, 1103-1121.
13. de Gelder, B., and Vroomen, J. (1992). " Auditory and visual speech perception in alphabetic and non-alphabetic Chinese/Dutch bilinguals.," in Cognitive Processing in Bilinguals, edited by, R. J. Harris, (Elsevier, Amsterdam), pp. 413-426.
14. Desjardins, R., and Werker, J. F. (2004). " Is the integration of heard and seen speech mandatoryfor young infants?," Dev. Psychobiol. 45, 187-203.
15. Dodd, B. (1977). " The role of vision in the perception of speech.," Perception 6, 31-40.
16. Erber, N. P., (1969). " Interaction of audition and vision in the recognition of oral speech stimuli.," J. Speech Hear. Res. 12, 423-425.
17. Erdener, V., and Burnham, D. (2005). " The role of audiovisual speech and orthographic information in nonnative speech production.," Lang. Learn. 55, 191-228.
18. Flege, J. E. (1987). " A critical period for learning to pronounce foreign languages?," Appl. Linguist. 8, 162-177.
19. Flege, J. E. (1995). " Second language speech learning: Theory, findings, and problems.," in Speech Perception and Linguistic Experience, edited by, W. Strange, (Baltimore, York), pp. 233-273.
20. Flege, J. E. (1998). " The role of subject and phonetic variables in L2 speech acquisition.," in Papers from the 34th Annual Meeting of the Chicago Linguistics Society, Volume II, The Panels, edited by, M. Gruber, D. Higgins, K. Olsen, and, T. Wysocki, (Chicago Linguistic Society, Chicago), pp. 213-232.
21. Flege, J. E. (2007). " Language contact in bilingualism: Phonetic system interactions.," in Laboratory Phonology 9, edited by, J. Cole, and, J. I. Hualde, (Mouton de Gruyter, Berlin).
22. Flege, J. E., Munro, M. J., and MacKay, I. R. A. (1995). " Factors affecting strength of perceived foreign accent in a second language.," J. Acoust. Soc. Am. 97, 3125-3134.
23. Flege, J. E., Yeni-Komshian, G., and Liu, S. (1999). " Age constraints on second language learning.," J. Mem. Lang. 41, 78-104.
24. Hardison, D. M. (1999). " Second-language spoken word identification: Effects of perceptual training, visual cues, and phonetic environment.," Lang. Learn. 49, 213-283.
25. Hardison, D. M. (2003). " Acquisition of second-language speech: Effects of visual cues, context, and talker variability.," Appl. Psycholinguist. 24, 495-522.
26. Hardison, D. M. (2005a). " Second-language spoken word identification: Effects of perceptual training, visual cues, and phonetic environment.," Appl. Psycholinguist. 26, 579-596.
27. Hardison, D. M. (2005b). " Variability in bimodal spoken language processing by native and nonnative speakers of English: A closer look at effects of speech style.," Speech Commun. 46, 73-93.
28. Hattori, T. (1987). " A study of nonverbal intercultural communication between Japanese and Americans-Focusing on the use of the eyes.," Japan Assoc. Lang. Teachers 8, 109-118.
29. Hazan, V., Sennema, A., Faulkner, A., and Ortega-Llebaria, M. (2006). " The use of visual cues in the perception of non-native consonant contrasts.," J. Acoust. Soc. Am. 119, 1740-1751.
30. Hazan, A., Sennema, A., Iba, M., and Faulkner, A. (2005). " Effect of audiovisual perceptual training on the perception and production of consonants by Japanese learners of English.," Speech Commun. 47, 360-378.
31. Jongman, A., Wang, Y., and Kim, B. (2003). " Contributions of semantic and facial information to perception of nonsibilant fricatives.," J. Speech Lang. Hear. Res. 46, 1367-1377.
32. Kuhl, P. K., and Meltzoff, A. N. (1982). " The bimodal perception of speech in infancy.," Science 218, 1138-1141.
33. Lehiste, I. (1970). Suprasegmentals (MIT, Cambridge, MA).
34. Massaro, D. W. (1987). Speech Perception by Ear and Eye: A Paradigm for Psychological Inquiry (Erlbaum, Hillsdale, NJ).
35. Massaro, D. W. (1998). Perceiving Talking Faces: From Speech Perception to a Behavioral Principle (MIT, Cambridge, MA).
36. Massaro, D. W., Tsuzaki, M., Cohen, M. M., Gesi, A., and Heredia, R. (1993). " Bimodal speech perception: An examination across language.," J. Phonetics 21, 445-478.
37. Mayo, L. H., and Florentine, M. (1997). " Age of second-language acquisition and perception of speech in noise.," J. Speech Lang. Hear. Res. 40, 686-693.
38. McAllister, R., Flege, J., and Piske, T. (2002). " The influence of L1 on the acquisition of Swedish quantity by native speakers of Spanish, English, and Estonian.," J. Phonetics 30, 229-258.
39. McGurk, H., and MacDonald, J. (1976). " Hearing lips and seeing voices.," Nature (London) 264, 746-748.
40. Nábelek, A. K., and Robinson, P. K. (1982). " Monaural and binaural speech perception in reverberation for listeners of various ages.," J. Acoust. Soc. Am. 71, 1242-1248.
41. Navarra, J., and Soto-Faraco, S. (2007). " Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds.," Psychol. Res. 71, 4-12.
42. Ortega-Llebaria, M., Faulkner, A., and Hazan, V. (2001) " Auditory-visual L2 speech perception: Effects of visual cues and acoustic-phonetic context for Spanish learners of English.," in Proceedings of the International Conference on Auditory-Visual Speech Processing, edited by, D. Massaro, J. Light, and, K. Geraci, pp. 149-154.
43. Patterson, M. L., and Werker, J. F. (1999). " Matching phonetic information in lips and voice is robust in 4.5-month -old infants.," Infant Behav. Dev. 22, 237-247.
44. Patterson, M. L., and Werker, J. F. (2003). " Two-month-old infants match phonetic information in lips and voice.," Dev. Sci. 6, 191-196.
45. Reisberg, D., McLean, J., and Goldfield, A. (1987). " Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli.," in Hearing by Eye: The Psychology of Lip-reading, edited by, B. Dodd, and, R. Campbell, (Erlbaum, London), pp. 97-113.
46. Riney, T. J., and Flege, J. E. (1998). " Changes over time in global foreign accent and liquid identifiability and accuracy.," Stud. Second Lang. Acquis. 20, 213-244.
47. Rosenblum, L. D., Schmuckler, M. A., and Johnson, J. A. (1997). " The McGurk effect in infants.," Percept. Psychophys. 59, 347-357.
48. Sams, M., Manninen, P., Surakka, V., Helin, P., and Katto, R. (1998). " McGurk effect in Finnish syllables, isolated words, and words in sentences: effects of word meaning and sentence context.," Speech Commun. 26, 75-87.
49. Sekiyama, K. (1997). " Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects.," Percept. Psychophys. 59, 73-80.
50. Sekiyama, K., Kanno, I., Miura, S., and Sugita, Y. (2003). " Auditory-visual speech perception examined by fMRI and PET.," Neurosci. Res. (N Y) 47, 277-287.
51. Sekiyama, K., and Tohkura, Y. (1991). " McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility.," J. Acoust. Soc. Am. 90, 1797-1805.
52. Sekiyama, K., and Tohkura, Y. (1993). " Inter-language differences in the influence of visual cues in speech perception.," J. Agric. Sci. 21, 427-444.
53. Sekiyama, K., Tohkura, Y., and Umeda, M. (1996). " A few factors which affect the degree of incorporating lip-read information into speech perception.," in Proceedings of the 4th International Conference on Spoken Language Processing, Philadelphia, PA, pp. 1481-1484.
54. Soto-Faraco, S., Navarra, J., Voloumanos, A., Sebastián- Galĺs, N., Weikum, W., and Werker, J. F. (2007). " Discriminating languages by speechreading.," Percept. Psychophys. 69, 218-231.
55. Sumby, W., and Pollack, I. (1954) " Visual contribution to speech intelligibility in noise.," J. Acoust. Soc. Am. 26, 212-215.
56. Summerfield, Q. (1979). " Use of visual information for phonetic perception.," Phonetica 36, 314-331.
57. Takata, Y., and Nábelek, A. K. (1990). " English consonant recognition in noise and in reverberation by Japanese and American listeners.," J. Acoust. Soc. Am. 88, 663-666.
58. Walden, B. E., Prosek, R. A., Montgomery, A. A., Scherr, C. K., and Jones, C. J. (1977). " Effects of training on the visual recognition of consonants.," J. Speech Hear. Res. 20, 130-145.
59. Weikum, W., Vouloumanos, A., Navarra, J., Soto-Faraco, S., Sebastián-Galĺs, N., and Werker, J. F. (2007). " Visual language discrimination in infancy.," Science 316 (5828), 1159.
60. Werker, J. F., Frost, P. E., and McGurk, H. (1992). " La langue et les levres: Cross-language influences on bimodal speech perception.," Can. J. Psychol. 46, 551-568.