Audiovisual perception in adverse conditions: Language, speaker and listener effects

Speech Communication

Volumn 52, Issue 11-12, 2010, Pages 996-1009

  Hazan, Valerie ^a   Kim, Jeesun ^b   Chen, Yuchun ^c  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF WESTERN SYDNEY   (Australia)

^c NATIONAL TAIWAN NORMAL UNIVERSITY   (Taiwan)

Author keywords

Audiovisual; L2 perception; Speaker listener variability

Indexed keywords

AUDIOVISUAL; AUDIOVISUAL PERCEPTION; L2 PERCEPTION; MANDARIN CHINESE; MULTIPLE ITERATIONS; NON-NATIVE SPEAKERS; RELATIVE CONTRIBUTION; SINGLE-CHANNEL; SPEAKER CHARACTERISTICS; SPEAKER/LISTENER VARIABILITY; SPEECH PERCEPTION; VISUAL CUES; VISUAL INFORMATION;

WEIGHING;

DEGRADATION;

EID: 78649319048     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2010.05.003     Document Type: Article

References (50)

1. C.T. Best, and M.D. Tyler Nonnative and second-language speech perception: commonalities and complementarities O.-S. Bohn, M.J. Munro, Language Experience in Second Language Speech Learning: In Honor of James Emil Flege Bohn 2007 John Benjamins Publishing Company Philadelphia 13 34
2. C.T. Best A direct realist view of cross-language speech perception W. Strange, Speech Perception and Linguistic Experience: Issues in Cross-Language Research 1995 York Press Timonium 171 203
3. C.A. Binnie, A.A. Montgomery, and P.L. Jackson Auditory and visual contribution to the perception of consonants J. Speech Hear. Res. 17 1974 619 630
4. W.T. Brandy Reliability of voice tests of speech discrimination J. Speech Hear. Res. 9 1966 461 465
5. Chen, Y., Hazan, V., 2007. Language effects on the degree of visual influence in audiovisual speech perception. In: Proc. 16th Internat. Cong. Phonetic Sciences. pp. 2177-2180.
6. Y. Chen, and V. Hazan Developmental factor and the nonnative speaker effect in auditory-visual speech perception J. Acoust. Soc. Am. 126 2009 858 865
7. T.H. Chen, and D.W. Massaro Mandarin speech perception by ear and eye follows a universal principle Percept. Psychophys. 66 2004 820 836
8. A. Cutler, A. Weber, R. Smits, and N. Cooper Patterns of English phoneme confusions by native and non-native listeners J. Acoust. Soc. Am. 116 2004 3668 3678
9. A. Cutler, M.L. Garcia Lecumberri, and M.P. Cooke Consonant identification in noise by native and non-native listeners: effects of local context J. Acoust. Soc. Am. 124 2008 1264 1268
10. C. Davis, and J. Kim Repeating and remembering foreign language words: implications for language teaching system Artif. Intell. Rev. 16 2001 37 47
11. C. Davis, and J. Kim Audio-visual interactions with intact clearly audible speech Q. J. Exp. Psychol. 57A 2004 1103 1121
12. B. de Gelder, and J. Vroomen Auditory and visual speech perception in alphabetic and non-alphabetic Chinese-Dutch bilinguals J. Alegria, D. Holender, J. Morais, M. Radeau, Cognitive Processing in Bilinguals 1992 Elsevier Amsterdam 413 426
13. de Gelder, B., Bertelson, P., Vroomen, J., Chen, H.C., 1995. Interlanguage differences in the McGurk effect for Dutch and Cantonese listeners. In: Proc. 4th European Conf. on Speech Communication and Technology. pp. 1699-1702.
14. B. Dodd The role of vision in the perception of speech Perception 6 1977 31 40
15. N. Erber Interaction of audition and vision in the recognition of oral speech stimuli J. Speech Hear. Res. 12 1969 423 425
16. Fixmer, E., Hawkins, S., 1998. The influence of quality of information on the McGurk effect. In: Proc. AVSP. pp. 27-32.
17. J. Flege Second-language speech learning: theory, findings, and problems W. Strange, Speech Perception and Linguistic Experience: Issues in Cross-Language Research 1995 York Press Timonium, MD 229 273
18. K.I. Forster, and J.C. Forster DMDX: a windows display program with millisecond accuracy Behav. Res. Meth. 35 2003 116 124
19. A. Fuster-Duran Perception of conflicting audio-visual speech: an examination across Spanish and German D.G. Stork, M.E. Hennecke, Speechreading by Humans and Machines: Models Systems and Applications 1996 Springer-Verlag New York 135 143
20. J.-P. Gagné, V. Masterson, K.G. Munhall, N. Bilida, and C. Querengesser Across talker variability in auditory, visual, and audiovisual speech intelligibility for conversational and clear speech J. Acad. Rehab. Audiol. 27 1994 135 158
21. J.-P. Gagné, A.J. Rochette, and M. Charest Auditory, visual and audiovisual clear speech Speech Commun. 37 2002 213 230
22. M.L. Garcia Lecumberri, and M. Cooke Effect of masker type on native and non-native consonant perception in noise J. Acoust. Soc. Am. 119 2006 2445 2454
23. K.W. Grant, B.E. Walden, and P.F. Seitz Auditory-visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory visual integration J. Acoust. Soc. Am. 103 1998 2677 2690
24. K.W. Grant, and P.F. Seitz The use of visible speech cues for improving auditory detection of spoken sentences J. Acoust. Soc. Am. 108 2000 1197 1208
25. D.M. Hardison Bimodal speech perception by native and nonnative speakers of English: factors influencing the McGurk effect Lang. Learn. 49 1999 213 283
26. Hayashi, Y., Sekiyama, K., 1998. Native-foreign language effect in the McGurk effect: a test with Chinese and Japanese. In: Proc. AVSP. pp. 61-66.
27. Hazan, V., Li, E., 2008. The effect of auditory and visual degradation on audiovisual perception of native and non-native speakers. In: Proc. Interspeech. pp. 1191-1194.
28. V. Hazan, A. Sennema, A. Faulkner, M. Ortega-Llebaria, M. Iba, and H. Chung The use of visual cues in the perception of nonnative consonant contrasts J. Acoust. Soc. Am. 119 2006 1740 1751
29. Kuhl, P.K., Tsuzaki, M., Tohkura, Y., Meltzoff, A.N., 1994. Human processing of auditory-visual information in speech perception: potential for multimodal human-machine interfaces. In: Proc. Internat. Conf. on Spoken Language Proceedings, Tokyo. pp. 539-542.
30. J. MacDonald, S. Andersen, and T. Bachmann Hearing by eye: how much spatial degradation can be tolerated? Perception 29 2000 1155 1168
31. D.W. Massaro, M.M. Cohen, and P.M. Smeele Cross-linguistic comparisons in the integration of visual and auditory speech Mem. Cognition 23 1995 113 131
32. D.W. Massaro Perceiving Talking Faces: From Speech Perception to a Behavioral Principle 1998 MIT Press Cambridge, MA
33. L.H. Mayo, M. Florentine, and S. Buus Age of second-language acquisition and perception of speech in noise J. Speech Lang. Hear. Res. 40 1997 686 693
34. H. McGurk, and J.W. MacDonald Hearing lips and seeing voices Nature 264 1976 746 748
35. J. Navarra, and S. Soto-Faraco Hearing lips in a second language: visual articulatory information enables the perception of second language sounds Psychol. Res. 71 2007 4 12
36. Nielsen, K., 2004. Segmental differences in the visual contribution to speech intelligibility. In: Proc. Interspeech. pp. 2533-2536.
37. Ortega-Llebaria, M., Faulkner, A., Hazan, V., 2001. Auditory-visual L2 speech perception: effects of visual cues and acoustic-phonetic context for Spanish learners of English. In: Proc. AVSP. pp. 149-154.
38. C.L. Rogers, J.J. Lister, D.M. Febo, J.M. Besing, and H.B. Abrams Effects of bilingualism, noise, and reverberation on speech perception by listeners with normal hearing App. Psycholinguist. 27 2006 465 485
39. L.A. Ross, D. Saint-Amour, V.M. Leavitt, D.C. Javitt, and J.J. Foxe Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments Cereb. Cortex 17 2007 1147 1153
40. K. Sekiyama, D. Burnham, H. Tam, and D. Erdener Auditory-visual speech perception development in Japanese and English speakers Proc. AVSP 2003 61 66
41. K. Sekiyama, and D. Burnham Impact of language on development of auditory-visual speech perception Dev. Sci. 11 2008 303 317
42. K. Sekiyama, and Y. Tohkura McGurk effect in non-English listeners: few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility J. Acoust. Soc. Am. 90 1991 1805 1979
43. K. Sekiyama, and Y. Tohkura Inter-language differences in the influence of visual cues in speech perception J. Phonetics 21 1993 427 444
44. Sekiyama, K., Braida, L.D., Nishino, K., Hayashi, M., Tsuyo, M., 1995. The McGurk effect in Japanese and American perceivers. In: Proc. 13th Internat. Cong. Phonetic Sciences, Vol. 3. pp. 214-217.
45. K. Sekiyama Cultural and linguistic factors in audiovisual speech processing: the McGurk effect in Chinese subjects Percept. Psychophys. 59 1997 73 80
46. W. Sumby, and I. Pollack Visual contribution to speech intelligibility in noise J. Acoust. Soc. Am. 26 1954 212 215
47. Y. Takata, and A.K. Nabelek English consonant recognition in noise and in reverberation by Japanese and American listeners J. Acoust. Soc. Am. 88 1990 663 666
48. S.M. Thomas, and T.R. Jordan Determining the influence of Gaussian blurring on inversion effects with talking faces Percept. Psychophys. 64 2002 932 944
49. Y. Wang, D. Behne, and H. Jiang Linguistic experience and audio-visual perception of nonnative fricatives J. Acoust. Soc. Am. 124 2008 1716 1726
50. Y. Wang, D. Behne, and H. Jiang Influence of native language phonetic system on audio-visual speech perception J. Phonetics 37 2009 344 356