Learning a model of speaker head nods using gesture corpora

Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS

Volumn 1, Issue , 2009, Pages 212-219

  Lee, Jina ^a   Marsella, Stacy ^a  

^a Institute for Creative Technologies University of Southern California ^*  (United States)

Author keywords

Embodied conversational agents; Head nods; Machine learning; Nonver bal behaviors; Virtual agents

Indexed keywords

INTELLIGENT AGENTS; LEARNING SYSTEMS; MULTI AGENT SYSTEMS;

COMMUNICATIVE FUNCTIONS; DISTRIBUTED ARTIFICIAL INTELLIGENCE; EMBODIED CONVERSATIONAL AGENT; HEAD NODS; LINGUISTIC FEATURES; NONVER BAL BEHAVIORS; PRECISION AND RECALL; VIRTUAL AGENT;

AUTONOMOUS AGENTS;

EID: 84899785169     PISSN: 15488403     EISSN: 15582914     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. W. Breitfuss, H. Prendinger, and M. Ishizuka. Automated generation of non-verbal behavior for virtual embodied characters. In ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces, pages 319-322, New York, NY, USA, 2007. ACM.
2. C. Busso, Z. Deng, M. Grimm, U. Neumann, and S. Narayanan. Rigid head motion in expressive speech animation: Analysis and synthesis. IEEE Transactions on Audio, Speech and Language Processing, 15(3):1075 -1086, 2007.
3. J. Carletta. Unleashing the killer corpus: experiences in creating the multi-everything AMI meeting corpus. Language Resources and Evaluation Journal, 41(2):181-190, 2007.
4. J. Cassell. More than just another pretty face: Embodied conversational interface agents. Communications of the ACM, 43:70-78, 2000.
5. J. Cassell, H. H. Vilhjalmsson, and T. Bickmore. BEAT: The behavior expression animation toolkit. In SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 477-486, New York, NY, USA, 2001. ACM.
6. N. Cathcart, J. Carletta, and E. Klein. A shallow model of backchannel continuers in spoken dialogue. In In EACL qf03: Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics, pages 51-58, 2003.
7. E. Charniak. Statistical Language Learning. MIT Press, Cambridge, Massachusetts, 1993.
8. E. Charniak. A maximum-entropy-inspired parser. In Proceedings of the first conference on North American chapter of the Association for Computational Linguistics, pages 132-139, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc.
9. D. Decarlo, C. Revilla, M. Stone, and J. J. Venditti. Making discourse visible: Coding and animating conversational facial displays. In In Proc. Computer Animation 2002, pages 11-16, 2002.
10. M. E. Foster and J. Oberlander. Corpus-based generation of head and eyebrow motion for an embodied conversational agent. Language Resources and Evaluation, 41:305-324(3), 2007.
11. M. E. Foster, M. White, A. Setzer, and R. Catizone. Multimodal generation in the COMIC dialogue system. In ACL '05: Proceedings of the ACL 2005 on Interactive poster and demonstration sessions, pages 45-48, Morristown, NJ, USA, 2005. Association for Computational Linguistics.
12. U. Hadar, T. J. Steiner, E. C. Grant, and F. C. Rose. Kinematics of head movements accompanying speech during conversation. Human Movement Science, 2:35-46, 1983.
13. U. Hadar, T. J. Steiner, and F. C. Rose. Head movement during listening turns in conversation. Journal of Nonverbal Behavior, 9(4):214-228, 1985.
14. D. Heylen. Challenges ahead: Head movements and other social acts in conversations. In AISB 2005, Social Presence Cues Symposium, 2005.
15. R. W. Hill, J. Belanich, H. C. Lane, M. G. Core, M. Dixon, E. Forbell, J. Kim, and J. Hart. Pedagogically structured game-based training: Development of the elect bilat simulation. In Proceedings of the 25th Army Science Conference (ASC 2006). Association for Computational Linguistics, Noverber, 2006.
16. B. H. Hwang and L. R. Rabiner. Hidden markov models for speech recognition, August 1991.
17. A. Kendon. Some uses of the head shake. Gesture, 2:147-182(36), 2002.
18. P. G. Kenny, T. D. Parsons, J. Gratch, A. Leuski, and A. A. Rizzo. Virtual patients for clinical therapist skills training. In C. Pelachaud, J.-C. Martin, E. Andre', G. Chollet, K. Karpouzis, and D. Pele', editors, IVA, volume 4722 of Lecture Notes in Computer Science, pages 197-210. Springer, 2007.
19. M. Kipp, M. Neff, K. H. Kipp, and I. Albrccht. Towards natural gesture synthesis: Evaluating gesture units in a data-driven approach to gesture synthesis. In IVA, pages 15-28, 2007.
20. F. L. A. Knoppel, A. S. Tigelaar, D. O. Bos, T. Alofs, and Z. Ruttkay. Trackside DEIRA: A dynamic engaging intelligent reporter agent. In AAMAS '08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, pages 112-119, Richland, SC, 2008. International Foundation for Autonomous Agents and Multiagent Systems.
21. J. Lee and S. Marsella. Nonverbal behavior generator for embodied conversational agents. Proceedings of the 6th International Conference on Intelligent Virtual Agents, Marina del Rey, CA, pages 243-255. Springer, 2006.
22. R. M. Maatman, J. Gratch, and S. Marsella. Natural behavior of a listening agent. In IVA, pages 25-36, 2005.
23. M. Mancini, R. Bresin, and C. Pelachaud. A virtual head driven by music expressivity. IEEE Transactions on Audio, Speech and Language Processing, 15 (6): 1833-1841, 2007.
24. E. Z. McClave. Linguistic functions of head movements in the context of speech. Journal of Pragmatics, 32:855-878(24), June 2000.
25. G. A. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. J. Miller. Introduction to Word Net: An on-line lexical database. Int J Lexicography, 3(4):235-244, January 1990.
26. L.-P. Morency, I. de Kok, and J. Gratch. Predicting listener backchannels: A probabilistic multimodal approach. In IVA, pages 176-190, 2008.
27. K. G. Munhall, J. A. Jones, D. E. Callan, T. Kuratate, and E. Vatikiotis-Bateson. Visual prosody and speech intelligibility: Head movement improves auditory speech perception. Psychological Science, 15:133-137(5), February 2004.
28. R. Nishimura, N. Kitaoka, and S. Nakagawa. A spoken dialog system for chat-like conversations considering response timing. In TSD, pages 599-606, 2007.
29. L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286, 1989.
30. M. Strauss and M. Kipp. Eric: A generic rule-based framework for an affective embodied commentary agent. In AAMAS '08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, pages 97-104, Richland, SC, 2008. International Foundation for Autonomous Agents and Multiagent Systems.
31. W. Swartout, J. Gratch, R. W. Hill, E. Hovy, S. Marsella, J. Rickel, and D. Traum. Toward virtual humans. AI Mag., 27(2):96-108, 2006.
32. G. Tom, P. Pettersen, T. Lau, T. Burton, and J. Cook. The role of overt head movement in the formation of affect. Basic and Applied Social Psychology, 12(3):281-289, 1991.
33. D. Traum, A. Roque, A. L. P. Georgiou, J. Gerten, B. M. S. Narayanan, S. Robinson, and A. Vaswani. Hassan: A virtual human for tactical questioning. In Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, pages 71-74, Antwerp, September 2007. Association for Computational Linguistics.
34. T. Ward and W. Tsukahara. Visual prosody and speech intelligibility in english and japanese. Pragmatics, 23:1177-1207, 2004.
35. A. D. Wilson and A. F. Bobick. Parametric hidden markov models for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(9):884-900, 1999.