Seeing pitch: Visual information for lexical tones of Mandarin-Chinese

Journal of the Acoustical Society of America

Volumn 123, Issue 4, 2008, Pages 2356-2366

  Chen, Trevor H ^a   Massaro, Dominic W ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BIOLOGICAL ORGANS; INFORMATION RETRIEVAL; SPEECH RECOGNITION;

LEXICAL TONES; SEEING PITCH; VISIBLE MOVEMENTS; VISUAL LEXICAL TONES;

VISUAL COMMUNICATION;

ADULT; ARTICLE; CHINESE; CONTROLLED STUDY; FEMALE; HEAD MOVEMENT; HUMAN; HUMAN EXPERIMENT; LANGUAGE; MALE; MOUTH; NECK; PITCH; PRIORITY JOURNAL; SPEECH; VISUAL INFORMATION;

ADULT; ASIAN CONTINENTAL ANCESTRY GROUP; FEMALE; HUMANS; MALE; PHONETICS; PITCH PERCEPTION; SPEECH PERCEPTION; VISUAL PERCEPTION; VOCABULARY;

EID: 41849089354     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.2839004     Document Type: Article

References (48)

1. Blicher, D. L., Diehl, R. L., and Cohen, L. B. (1990). " Effects of syllable duration on the perception of the Mandarin tone 2/tone 3 distinction: Evidence of auditory enhancement.," J. Phonetics JPHNB9 0095-4470 18, 37-49.
2. Burnham, D., Ciocca, V., Lauw, C., Lau, S., and Stokes, S. (2000). " Perception of visual information for Cantonese tones.," in Proceedings of the Eighth Australian International Conference on Speech Science and Technology, edited by, M. Barlow, and, P. Rose, (Australian Speech Science and Technology Association, Canberra) pp. 86-91.
3. Burnham, D., Lau, S., Tam, H., and Schoknecht, C. (2001). " Visual discrimination of Cantonese tone by tonal but non-Cantonese speakers, and by non-tonal language speakers.," Proceedings of the International Conference on Auditory-Visual Speech Processing, edited by, D. Massaro, J. Light, and, K. Geraci, (Aalborg, Denmark, September 7-9, 2001), pp. 155-160.
4. Burnham, D., Reynolds, J., Vatikiotis-Bateson, E., Yehia, H., Ciocca, V., Morris, R. H., Hill, H., Vignali, G., Bollwerk, S., Tam, H., and Jones, C. (2006). " The perception and production of phones and tones: The role of rigid and non-rigid face and head motion.," in Proceedings of the Seventh International Seminar on Speech Production, edited by, H. Yehia, D. Demolin, and, R. Laboissiere, (Ubatuba, Sao Paolo, Brazil), pp. 185-192.
5. Chao, Y. R. (1968). A Grammar of Spoken Chinese (University of California Press, Berkeley).
6. Chen, J.-Y. (1999). " The representation and processing of tone in Mandarin Chinese: Evidence from slips of the tongue.," Appl. Psycholinguist. APPSDZ 0142-7164 20, 289-301.
7. Chen, T. H., and Massaro, D. W. (2004). " Mandarin speech perception by ear and eye follows a universal principle.," Percept. Psychophys. PEPSBJ 0031-5117 66, 820-836.
8. Connell, B. (2000). " The perception of lexical tone in Mambila.," Lang Speech ZZZZZZ 0023-8309 43, 163-182.
9. Connell, B. A., Hogan, J. T., and Rozsypal, A. J. (1983). " Experimental evidence of interaction between tone and intonation in Mandarin Chinese.," J. Phonetics JPHNB9 0095-4470 11, 337-351.
10. Garding, E., Kratochvil, P., Svantesson, J.-O., and Zhang, J. (1986). " Tone 4 and tone 3 discrimination in modern standard Chinese.," Lang Speech ZZZZZZ 0023-8309 29, 281-293.
11. Gesi, A. T., Massaro, D. W., and Cohen, M. M. (1992). " Discovery and expository methods in teaching visual consonants and word identification.," J. Speech Hear. Res. JSPHAH 0022-4685 35, 1180-1188.
12. Green, K. P., and Miller, J. L. (1985). " On the role of visual rate information in phonetic perception.," Percept. Psychophys. PEPSBJ 0031-5117 38, 269-276.
13. Hayashi, Y., and Sekiyama, K. (1998). " Native-foreign language effect in the McGurk effect: A test with Chinese and Japanese.," Proceedings of Auditory-Visual Speech Processing 1998 International Conference, Terrigal-Sydney, Australia, 4-7 December, pp. 61-66.
14. Jiang, X., Wang, Y., and Zhang, F. (1999). " Visual speech analysis and synthesis with application to Mandarin speech training.," Virtual reality software and technology (VRST 1999), pp. 111-115.
15. Jongman, A., Wang, Y., Moore, C. B., and Sereno, J. A. (2005). " Perception and production of Mandarin Chinese tones.," http://www.ku.edu/ ~kuppl/sereno/Handbookjong%20in%20press.pdf.
16. Kim, J., and Davis, C. (2001). " Visible speech cues and auditory detection of spoken sentences: An effect of degree of correlation between acoustic and visual properties.," Proceedings of the International Conference on Auditory-Visual Speech Processing, edited by, D. Massaro, J. Light, and, K. Geraci, (Aalborg, Denmark, September 7-9, 2001), pp. 127-131.
17. Ladefoged, P. (2001). A Course in Phonetics, 4th ed. (Thompson, Learning, Boston).
18. Lass, N. J. (1976). Contemporary Issues in Experimental Phonetics (Academic, New York).
19. Lee-Schoenfeld, V. (2002). " Third tone prominence.," http://people.ucsc.edu/~vls/Papers/T3_Prominence.pdf.
20. Lehiste, I. (1970). Suprasegmentals (MIT, Cambridge, MA).
21. Lieberman, P. (1967). Intonation, Perception, and Language (MIT, Cambridge, MA).
22. Liu, S., and Samuel, A. G. (2004). " Perception of Mandarin lexical tones when F0 information is neutralized.," Lang Speech ZZZZZZ 0023-8309 47, 109-138.
23. Massaro, D. W. (1998). Perceiving Talking Faces: From Speech Perception to a Behavioral Principle (MIT, Cambridge, MA).
24. Massaro, D. W., Cohen, M. M., Campbell, C. S., and Rodriguez, T. (2001). " Bayes factor of model selection validates FLMP.," Psychonomic Bull. Rev. 8, 1-17.
25. Massaro, D. W., Cohen, M. M., and Gesi, A. T. (1993). " Long-term training, transfer, and retention in learning to lipread.," Percept. Psychophys. PEPSBJ 0031-5117 53, 549-562.
26. Massaro, D. W., Cohen, M. M., and Smeele, P. M. T. (1995). " Cross-linguistic comparisons in the integration of visual and auditory speech.," Mem. Cognit. MYCGAO 0090-502X 23, 113-131.
27. Massaro, D. W., Cohen, M. M., and Tseng, C. Y. (1985). " The evaluation and integration of pitch height and pitch contour in lexical tone perception in Mandarin Chinese.," J. Chin. Linguist. ZZZZZZ 0091-3723 13, 267-289.
28. Massaro, D. W., and Light, J. (2003). " Read my tongue movements: Bimodal learning to perceive and produce non-native speech /r/ and /l/.," Proceedings of Eurospeech (Interspeech), Eight European Conference on Speech Communication and Technology, Geneva, Switzerland,.
29. Massaro, D. W., and Light, J. (2004). " Improving the vocabulary of children with hearing loss.," Volta Rev. VOLRAT 0042-8639 104, 141-174.
30. Massaro, D. W., Tseng, C. Y., and Cohen, M. M. (1983). " Vowel and lexical tone perception in Mandarin Chinese: Psycholinguistic and psychoacoustic contributions.," J. Quant. Linguist. ZZZZZZ 0929-6174 19, 76-102.
31. Massaro, D. W., Tsuzaki, M., Cohen, M. M., Gesi, A., and Heredia, R. (1993). " Bimodal speech perception: An examination across languages.," J. Phonetics JPHNB9 0095-4470 21, 445-478.
32. McGurk, H., and MacDonald, J. (1976). " Hearing lips and seeing voices.," Nature NATUAS 0028-0836 10.1038/264746a0 264, 746-748.
33. Mixdorff, H., Charnvivit, P., and Burnham, D. (2005a). " Auditory-visual perception of syllabic tones in Thai.," Proceedings of Auditory-Visual Speech Processing Workshop 2005, edited by, E. Vatikiotis-Babeson, (British Columbia, Canada, July 24-27, 2005) pp. 3-8.
34. Mixdorff, H., Hu, Y., and Burnham, D. (2005b). " Visual cues in Mandarin tone perception.," Proceedings of Eurospeech 2005 (InterSpeech-2005), Lisbon, Portugal, pp. 405-408.
35. Mixdorff, H., Luong, M. C., Nguyen, D. T., and Burnham, D. (2006). " Syllabic tone perception in Vietnamese.," Proceedings of International Symposium on Tonal Aspects of Languages 2006, La Rochelle, France, pp. 137-142.
36. Mullin, W. J., Gerace, W. J., Mestre, J. P., and Velleman, S. L. (2003). Fundamentals of Sound with Applications to Speech and Hearing (Pearson Education, Boston, MA).
37. Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., and Vatikiotis-Bateson, E. (2004). " Visual prosody and speech intelligibility: Head movement improves auditory speech perception.," Psychol. Sci. ZZZZZZ 0956-7976 15, 133-137.
38. Rosenblum, L. D., and Fowler, C. A. (1991). " Audiovisual investigation of the loudness-effort effect for speech and nonspeech events.," J. Exp. Psychol. Hum. Percept. Perform. JPHPDH 0096-1523 10.1037/0096-1523.17.4.976 17, 976-985.
39. Schwartz, J.-L., and Savariaux, C. (2001). " Is it easier to lipread one's own speech gestures than those of somebody else? It seems not!," Proceedings of the International Conference on Auditory-Visual Speech Processing, pp. 18-23.
40. Sekiyama, K. (1997). " Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects.," Percept. Psychophys. PEPSBJ 0031-5117 59, 73-80.
41. Sekiyama, K., and Tohkura, Y. (1991). " McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility.," J. Acoust. Soc. Am. JASMAN 0001-4966 10.1121/1.401660 90, 1797-1805.
42. Sekiyama, K., and Tohkura, Y. (1993). " Inter-language differences in the influence of visual cues in speech perception.," J. Phonetics JPHNB9 0095-4470 21, 427-444.
43. Srinivasan, R. J., and Massaro, D. W. (2003). " Perceiving prosody from the face and voice: Distinguishing statements from echoic questions in English.," Lang Speech ZZZZZZ 0023-8309 46, 1-22.
44. Tseng, C. (1981). " An acoustic phonetic study on tones in Mandarin Chinese.," Ph.D. dissertation, Brown University, Providence, RI.
45. Tseng, C. Y., Massaro, D. W., and Cohen, M. M. (1986). " Lexical tone perception in Mandarin Chinese: Evaluation and integration of acoustic features.," in Linguistics, Psychology, and the Chinese Language, edited by, H. S. R. Kao, and, R. Hoosain, (Centre of Asian Studies, University of Hong Kong), pp. 91-104.
46. Wang, Y., Spence, M., Jongman, A., and Sereno, J. (1999). " Training American listeners to perceive Mandarin tones.," J. Acoust. Soc. Am. JASMAN 0001-4966 10.1121/1.428217 106, 3649-3658.
47. Whalen, D. H., and Xu, Y. (1992). " Information for Mandarin tones in the amplitude contour and in brief segments.," Phonetica PHNTAW 0031-8388 49, 25-47.
48. Yehia, H. C., Kuratate, T., and Vatikiotis-Bateson, E. (2002). " Linking facial animation, head motion and speech acoustics.," J. Phonetics JPHNB9 0095-4470 10.1006/jpho.2002.0165 30, 555-568.