Faciliation of Mandarin tone perception by visual speech in clear and degraded audio: Implications for cochlear implants

Journal of the Acoustical Society of America

Volumn 131, Issue 2, 2012, Pages 1480-1489

  Smith, Damien ^a   Burnham, Denis ^a  

^a UNIVERSITY OF WESTERN SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

AUDITORY-VISUAL; LEXICAL TONES; SPEECH INFORMATION; TONE LANGUAGES; VISUAL SPEECH;

AUDITION; COCHLEAR IMPLANTS;

VISUAL LANGUAGES;

ADOLESCENT; ADULT; ARTICLE; ASSOCIATION; AUSTRALIA; CHINA; COCHLEA PROSTHESIS; FEMALE; HEARING; HUMAN; LANGUAGE; MALE; PHYSIOLOGY; SPEECH PERCEPTION; UNITED KINGDOM; VISION;

ADOLESCENT; ADULT; AUSTRALIA; CHINA; COCHLEAR IMPLANTS; CUES; ENGLAND; FEMALE; HUMANS; LANGUAGE; MALE; MULTILINGUALISM; PITCH DISCRIMINATION; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84863121501     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.3672703     Document Type: Article

References (42)

1. Arnold, P., and Hill, F. (2001). Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact., Br. J. Psychol. 98, 339-355. 10.1348/000712601162220
2. Au, D. K. K. (2003). Effects of stimuation rates on Cantonese lexical tone perception by cochlear implant users in Hong Kong., Clin. Otolaryngol. 28, 533-538. 10.1046/j.1365-2273.2003.00747.x
3. Barry, J. G., Blamey, P. J., Martin, L. F. A., Lee, K. Y., Tang, T., Ming, Y. Y., and Hasselt, C. A. V. (2002). Tone discrimination in Cantonese-speaking children using a cochlear implant., Clin. Linguist. Phonet. 16, 79-99. 10.1080/02699200110109802
4. Blicher, D. L., Diehl, R. L., and Cohen, L. B. (1990). Effects of syllable duration on the perception of the Mandarin Tone 2/Tone 3 distinction: evidence of auditory enhancement., J. Phonetics 18, 37-49.
5. Boersma, P., and Weenink, D. (2010), Praat: Doing phonetics by computer (version 5.1.39) [computer program]., http://www.praat.org (Last viewed November 28, 2011).
6. Burkholder, R., Pisoni, D., and Svirsky, M. (2004). Perceptual learning and nonword repetition using a cochlear implant simulation., Int. Congress Ser. 1273, 208-211. 10.1016/j.ics.2004.08.023
7. Burnham, D., Ciocca, V., and Stokes, S. (2001). Auditory-visual perception of lexical tone., in Eurospeech Conference 2001, edited by, P. Dalsgaard, B. Lindberg, H. Benner, pp. 395-398.
8. Burnham, D., Lau, S., Tam, H., and Schoknecht, C. (2001). Visual discrimination of cantonese tone by tonal but non-cantonese speakers, and by non-tonal language speakers., in Auditory-Visual Speech Perception Conference 2001, edited by, D. Massaro, J. Light, and, K. Geraci, pp. 155-160.
9. Burnham, D., Reynolds, J., Vatikiotis-Bateson, E. Yehia, H., Ciocca, V., Haszard Morris, R., Hill, H., Vignali, G., Bollwerk, S., Tam, H., Jones, C. (2006) The perception and production of phones and tones: The role of rigid and non-rigid face and head motion., in Proceedings of ISSP 2006, 7th International Seminar on Speech Production, edited by, H. C. Yehia, D. Demolin, and, R. Laboissiere, pp. 185-192.
10. Cavé, C., Guatella, I., Bertrand, R., Santi, S., Harlay, F., and Espesser, R. (1996). About the relationship between eyebrow movements and F0 variations., in Proceedings of the 4th International Conference on Spoken Language Processing 4, edited by, T. Bunnell, and, W. Idsardi, pp. 2175-2178.
11. Chao, Y. R. (1968). A Grammar of Spoken Chinese (University of California Press, Berkeley, CA), 847 pp.
12. Chen, T. H., and Massaro, D. W. (2008). Seeing pitch: Visual information for lexical tones of Mandarin-Chinese., J. Acoust. Soc. Am. 123, 2356-2366. 10.1121/1.2839004
13. Ciocca, V., Francis, A. L., Aisha, R., and Wong, L. (2002). The perception of Cantonese lexical tone by early-deafened cochlear implantees., J. Acoust. Soc. Am. 111, 2250-2256. 10.1121/1.1471897
14. Cooper, N., Cutler, A., and Wales, R. (2002). Constraints of lexical stress on lexical access in English: Evidence from native and non-native listeners., Lang. Speech 45, 207-228. 10.1177/00238309020450030101
15. Cvejic, E., Kim, J., and Davis, C. (2010). Prosody off the top of the head: Prosodic contrasts can be discriminated by head motion., Speech Commun. 52, 555-564. 10.1016/j.specom.2010.02.006
16. Dorman, M., Loizou, P., Kemp, L., Iler Kirk, K. (2000). Word recognition by children listening to speech processed into a small number of channels: Data from normal-hearing listeners and children with cochlear implants., Ear Hear. 21, 590-596. 10.1097/00003446-200012000-00006
17. Forster, K. I., and Forster, J. C. (2003). DMDX: A windows display program with millisecond accuracy., Behav. Res. Methods Instrum. Comput. 35, 116-124. 10.3758/BF03195503
18. Friesen, L., Shannon, R., Baskent, D., and Wang, X. (2001). Speech recognition in noise as a function of the number of channels: Comparison of acoustic hearing and cochlear implants., J. Acoust. Soc. Am. 110, 1150-1163. 10.1121/1.1381538
19. Fromkin, V. A. (1978). Tone: A Linguistic Survey (Academic, New York), 299 pp.
20. Gandour, J. T. (1978). The perception of tone., in Tone: A Linguistic Survey, edited by, V. A. Fromkin, (Academic, New York), pp. 41-76.
21. Hayashi, Y., and Sekiyama, K. (1998). Native-foreign langauge effect in the McGurk effect: A test with Chinese and Japanese., in AVSP-1998, pp. 61-66.
22. Honda, K. (2004). Physiological factors causing tonal characteristics of speech: From global to local prosody., in SP-2004, 739-744.
23. Huang, C., Yang, H., Sher, Y., Lin, Y., and Wu, J. (2005). Speech intelligibility of Mandarin-speaking deaf children with cochlear implants., Int. J. Pediatr. Otorhinolaryngol. 69, 505-511. 10.1016/j.ijporl.2004.10.017
24. Huang, T., Wang, N., and Liu, S. (1995). Tone perception of Mandarin speaking postlingually deaf implantees using the Nucleus 22-channel cochlear implant mini system., Ann. Otol. Rhinol. Laryngol. Suppl. 166, 294-298.
25. Lassaletta, L., Castro, A., Bastarrica, M., de Sarria, M. J., and Gavilan, J. (2006). Quality of life in postlingually deaf patients following cochlear implantation., Eur. Arch. Otorhinolaryngol. 263, 267-270. 10.1007/s00405-005-0987-1
26. Looi, V., McDermott, H., McKay, C., and Hickson, L. (2004). Pitch discrimination and melody recognition by cochlear implant users., Int. Congr. Ser. 1273, 197-200. 10.1016/j.ics.2004.08.038
27. McDermott, H., and Looi, V. (2004). Perception of complex signals, including musical sounds, with cochlear implants., Int. Congr. Ser. 1273, 201-204. 10.1016/j.ics.2004.08.034
28. McGurk, H., and MacDonald, J. (1976). Hearing lips and seeing voices., Nature 264, 746-748. 10.1038/264746a0
29. Mixdorff, H., Hu, Y., and Burnham, D. (2005). Visual cues in Mandarin tone perception., in Proceedings of Eurospeech 2005, pp. 405-408.
30. Rönnberg, J., Andersson, J., Samuelsson, S., Söderfeldt, B., Lyxell, B., and Risberg, J. (1999). A speechreading expert: The case of MM., J. Speech Lang. Hear. Res. 42, 5-20.
31. Sekiyama, K., and Burnham, D. (2008). Impact of language on development of auditory-visual speech perception., Dev. Sci. 11, 303-317. 10.1111/j.1467-7687.2008.00677.x
32. Sekiyama, K., and Tohkura, Y. (1993). Inter-language differences in the influence of visual cues in speech perception., J. Phonet. 21, 427-444.
33. th International Congress on Acoustics, Sydney, 23-27 August 2010.
34. Smith, D., and Croot, K. (2006). Visual speech contributions to learning an unfamiliar language contrast., in 33rd Australian Experimental Psychology Conference, Brisbane, April 21-23, 2006. Aust. J. Psychol, 58 (suppl. 1), 90.
35. Sumby, W. H., and Pollack, I. (1954). Visual contribution to speech intelligibility in noise., J. Acoust. Soc. Am. 26, 212-215. 10.1121/1.1907309
36. Swanson, B., and Mauch, H. (2006). Nucleus MATLAB Toolbox (version 4.20) [Computer program]., (Cochlear Corporation, Sydney, Australia).
37. Vandali, A. E., Whitford, L. A., Plant, K. L., and Clark, G. M. (2000). Speech perception as a function of electrical stimulation rate: Using the nucleus 24 cochlear implant system., Ear Hear. 21, 608-624. 10.1097/00003446- 200012000-00008
38. Venables, W. N., and Ripley, B. D. (2002). Modern Applied Statistics with S. (New York, Springer-Verlag), 495 pages.
39. Wang, Y., Spence, M. M., Jongman, A., and Sereno, J. A. (1999). Training American listeners to perceive Mandarin tones., J. Acoust. Soc. Am. 106, 3649-3658. 10.1121/1.428217
40. Whalen, D. H., and Xu, Y. (1992). Information for Mandarin tones in the amplitude contour and in brief segments., Phonetica 49, 25-47. 10.1159/000261901
41. Yehia, H., Kuratate, T., and Vatikiotis-Bateson, E. (2002). Linking facial animation, head motion and speech acoustics., J. Phonet. 30, 555-568. 10.1006/jpho.2002.0165
42. Yip, M. J. W. (2002). Tone (Cambridge University Press, Cambridge, UK), 341 pp.