Audiovisual representation of prosody in expressive speech communication

Speech Communication

Volumn 46, Issue 3-4, 2005, Pages 473-484

  Granström, Björn ^a   House, David ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Animation; Audiovisual prosody; Expressive speech; Multimodal communication; Talking heads

Indexed keywords

ANIMATION; AUDIO ACOUSTICS; COMPUTER SUPPORTED COOPERATIVE WORK; MATHEMATICAL MODELS; NEUROPHYSIOLOGY; SPEECH ANALYSIS;

ACOUSTIC SPEECH; AUDIOVISUAL PROSODY; EXPRESSIVE SPEECH; INTONATION TRAINING; MULTIMODAL COMMUNICATIONS; TALKING HEADS;

SPEECH COMMUNICATION;

EID: 21844443583     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2005.02.017     Document Type: Conference Paper

References (32)

1. Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K.-E., Öhman, T., 1999. Synthetic visual speech driven from auditory speech. In: Proc AVSP 99, pp. 123-127.
2. Bell, L., Gustafson, J., 1999. Interaction with an animated agent in a spoken dialogue system. In: Proc Eurospeech'99, Budapest, pp. 1143-1146.
3. Beskow, J., 1997. Animation of talking agents. In: Proc. ESCA Workshop on Audio-visual Speech Processing, Rhodes, Greece, pp. 149-152.
4. Beskow, J., 2003. Talking heads-models and applications for multimodal speech synthesis. PhD thesis, TMH/KTH.
5. Beskow, J., Granström, B., House, D., Lundeberg, M., 2000. Experiments with verbal and visual conversational signals for an automatic language tutor. In: Proc. InSTIL 2000.
6. Beskow, J., Engwall, O., Granström, B., 2003. Resynthesis of facial and intraoral articulation from simultaneous measurements. In: Proc. ICPhS 2003, Barcelona, Spain.
7. Brennan, S.E., 1990. Seeking and providing evidence for mutual understanding. Doctoral dissertation, Stanford University, Stanford, CA, Unpublished.
8. R. Carlson, and B. Granström Speech synthesis W. Hardcastle J. Laver The handbook of phonetic sciences 1997 Blackwell Publishers Ltd Oxford 768 788
9. Cavé, C., GuaÏtella, I., Bertrand, R., Santi, S., Harlay, F., Espesser, R., 1996. About the relationship between eyebrow movements and F0 variations. In: Bunnell, H.T., Idsardi, W. (Eds.), Proc. ICSLP 96, Philadelphia, PA, USA, pp. 2175-2178.
10. H.H. Clark, and E.F. Schaeffer Contributing to discourse Cognitive Sci. 13 1989 259 294
11. Edlund, J., Beskow, J., Nordstrand, M., 2002. GESOM-a model for describing and generating multi-modal output. In: Proc ISCA Workshop Multi-Modal Dialogue in Mobile Environments.
12. I. Fonâgy La mimique buccale Phonetica 33 1976 31 44
13. Gill, S.P., Kawamori, M., Katagiri, Y., Shimojima, A., 1999. Pragmatics of body moves. In: Proc. 3rd Internat. Cognitive Technology Conf., pp. 345-358.
14. Granström, B., House, D., Lundeberg, M., 1999. Prosodic cues in multimodal speech perception. In: Proc. Internat. Congress of Phonetic Sciences (ICPhS99), San Francisco, pp. 655-658.
15. Granström, B., House, D., Swerts, M., 2002. Multimodal feedback cues in human-machine interactions. In: Bel, B., Marlien, I. (Eds.), Proc. Speech Prosody 2002 Conf., Laboratoire Parole et Langage, Aix-en-Provence, pp. 347-350.
16. Gustafson, J., 2002. Developing multimodal spoken dialogue systems. Empirical studies of spoken human-computer interaction. Doctoral thesis, Department of Speech, Music and Hearing, KTH, Stockholm.
17. Hirschberg, J., Litman, D., Swerts, M., 2001. Identifying user corrections automatically in spoken dialogue systems. In: Proc. NAACL 2001, Pittsburg.
18. D. House Focal accent in Swedish: Perception of rise properties for accent 1 W. van Dommelen T. Fretheim Nordic Prosody Vol. 8 2001 Peter Lang. Frankfurt 127 135
19. House, D., 2002. Intonational and visual cues in the perception of interrogative mode in Swedish. In: Proc. ICSLP 2002, Colorado, Denver, pp. 1957-1960.
20. House, D., Beskow, J., Granström, B., 2001. Timing and interaction of visual cues for prominence in audiovisual speech perception. In: Proc. Eurospeech 2001, pp. 387-390.
21. E. Krahmer, M. Swerts, M. Theune, and M. Weegels The dual of denial: Two uses of disconfirmations in dialogue and their prosodic correlates Speech Comm. 36 1-2 2002 133 145
22. D.W. Massaro Perceiving Talking Faces: From Speech Perception to a Behavioural Principle 1998 The MIT Press Cambridge, MA
23. D.W. Massaro, M.M. Cohen, and P.M.T. Smeele Perception of asynchronous and conflicting visual and auditory speech J. Acoust. Soc. Amer. 100 1996 1777 1786
24. Nordenberg, M., 2003. Modelling and rendering dynamic wrinkles in a virtual face. TMH/KTH MSc thesis (available at http://www.speech.kth.se/qpsr/masterproj/ ).
25. Nordstrand, M., Svanfeldt, G., Granström, B., House, D., 2003. Measurements of articulatory variation and communicative signals in expressive speech. In: Proc AVSP'03, pp. 233-238.
26. M. Nordstrand, G. Svanfeldt, B. Granström, and D. House Measurements of articulatory variation in expressive speech for a set of Swedish vowels Speech Comm. 44 2004 187 196
27. F.I. Parke Parameterized models for facial animation IEEE Comput. Graphics 2 9 1982 61 68
28. C. Pelachaud, N. Badler, and M. Steedman Generating facial expressions for speech Cognitive Sci. 20 1996 1 46
29. A. Shimojima, Y. Katagiri, H. Koiso, and M. Swerts Informational and dialogue-coordinating functions of prosodic features of Japanese echoic responses Speech Comm. 36 1-2 2002 113 132
30. Siciliano, C., Williams, G., Beskow, J., Faulkner, A., 2003. Evaluation of a multilingual synthetic talking face. as a communication aid for the hearing impaired. In: Proc. 15th Internat. Congress of Phonetic Sciences.
31. R.J. Srinivasan, and D.W. Massaro Perceiving prosody from the face and voice: Distinguishing statements from echoic questions in English Lang. Speech 46 1 2003 1 22
32. Traum, D.R., 1994. A computational theory of grounding in natural language conversation. PhD thesis, Rochester.