Modeling the semantic coordination of speech and gesture under cognitive and linguistic constraints

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 8108 LNAI, Issue , 2013, Pages 203-216

  Bergmann, Kirsten ^a   Kahl, Sebastian ^a   Kopp, Stefan ^a  

^a BIELEFELD UNIVERSITY   (Germany)

Author keywords

cognitive modeling; generation; gesture; semantic coordination; Speech

Indexed keywords

COGNITIVE MODEL; COGNITIVE PROCESS; EMPIRICAL DATA; GENERATION; GESTURE; LINGUISTIC CONSTRAINTS; NATURAL SEMANTICS; VIRTUAL AGENT;

BEHAVIORAL RESEARCH; INTELLIGENT VIRTUAL AGENTS; LINGUISTICS; SPEECH;

DATA REDUCTION;

EID: 84883424058     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-40415-3_18     Document Type: Conference Paper

References (29)

1. Anderson, J., Bothell, D., Byrne, M., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychological Review 111(4), 1036-1060 (2004)
2. Bavelas, J., Kenwood, C., Johnson, T., Philips, B.: An experimental study of when and how speakers use gestures to communicate. Gesture 2(1), 1-17 (2002)
3. Bergmann, K., Eyssel, F., Kopp, S.: A second chance to make a first impression? How appearance and nonverbal behavior affect perceived warmth and competence of virtual agents over time. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 126-138. Springer, Heidelberg (2012)
4. Bergmann, K., Kopp, S.: Verbal or visual: How information is distributed across speech and gesture in spatial dialog. In: Proceedings of SemDial 2006, pp. 90-97 (2006)
5. Bergmann, K., Kopp, S.: GNetIc - Using Bayesian decision networks for iconic gesture generation. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 76-89. Springer, Heidelberg (2009)
6. Bergmann, K., Kopp, S.: Gestural alignment in natural dialogue. In: Proceedings of the 34th Annual Conference of the Cognitive Science Society (CogSci 2013), pp. 1326-1331. Cognitive Science Society, Austin (2012)
7. Bergmann, K., Kopp, S., Eyssel, F.: Individualized gesturing outperforms average gesturing - evaluating gesture production in virtual humans. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS, vol. 6356, pp. 104-117. Springer, Heidelberg (2010)
8. Breslow, L., Harrison, A., Trafton, J.: Linguistic spatial gestures. In: Proceedings of Cognitive Modeling 2010, pp. 13-18 (2010)
9. Cassell, J., Stone, M., Yan, H.: Coordination and context-dependence in the generation of embodied conversation. In: Proceedings of the First International Conference on Natural Language Generation (2000)
10. Cassell, J., Vilhjálmsson, H., Bickmore, T.: BEAT: The behavior expression animation toolkit. In: Proceedings of SIGGRAPH 2001, New York, NY, pp. 477-486 (2001)
11. Collins, A.M., Loftus, E.F.: A spreading-activation theory of semantic processing. Psychological Review 82(6), 407-428 (1975)
12. Endrass, B., Damian, I., Huber, P., Rehm, M., André, E.: Generating culture-specific gestures for virtual agent dialogs. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS, vol. 6356, pp. 329-335. Springer, Heidelberg (2010)
13. Hostetter, A., Alibali, M.: Raise your hand if you're spatial - relations between verbal and spatial skills and gesture production. Gesture 7, 73-95 (2007)
14. Hostetter, A., Alibali, M.: Cognitive skills and gesture-speech redundancy. Gesture 11(1), 40-60 (2011)
15. Kita, S., Özyürek, A.: What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48, 16-32 (2003)
16. Kita, S., Özyürek, A., Allen, S., Brown, A., Furman, R., Ishizuka, T.: Relations between syntactic encoding and co-speech gestures: Implications for a model of speech and gesture production. Language and Cognitive Processes 22, 1212-1236 (2007)
17. Kita, S., Davies, T.S.: Competing conceptual representations trigger co-speech representational gestures. Language and Cognitive Processes 24(5), 761-775 (2009)
18. Kopp, S., Bergmann, K.: Automatic and strategic alignment of co-verbal gestures in dialogue. In: Wachsmuth, I., de Ruiter, J., Jaecks, P., Kopp, S. (eds.) Alignment in Communication: Towards a New Theory of Communication, ch. 6. John Benjamins, Amsterdam (in press)
19. Kopp, S., Bergmann, K., Kahl, S.: A spreading-activation model of the semantic coordination of speech and gesture. In: Proceedings of the 35th Annual Conference of the Cognitive Science Society (CogSci 2013). Cognitive Science Society, Austin (in press, 2013)
20. Kopp, S., Tepper, P., Ferriman, K., Striegnitz, K., Cassell, J.: Trading spaces: How humans and humanoids use speech and gesture to give directions. In: Nishida, T. (ed.) Conversational Informatics, pp. 133-160. John Wiley, New York (2007)
21. Kraemer, N., Bente, G.: Personalizing e-learning. The social effects of pedagogical agents. Educational Psycholoy Review 22, 71-87 (2010)
22. Lee, J., Marsella, S.: Nonverbal behavior generator for embodied conversational agents. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 243-255. Springer, Heidelberg (2006) (Pubitemid 44532618)
23. Levelt, W.J.M.: Speaking: From intention to articulation. MIT Press (1989)
24. McNeill, D., Duncan, S.: Growth points in thinking-for-speaking. In: Language and Gesture, pp. 141-161. Cambridge University Press, Cambridge (2000)
25. Melinger, A., Kita, S.: Conceptualisation load triggers gesture production. Language and Cognitive Processes 22(4), 473-500 (2007)
26. Neff, M., Kipp, M., Albrecht, I., Seidel, H.P.: Gesture modeling and animation based on a probabilistic re-creation of speaker style. ACM Transactions on Graphics 27(1), 1-24 (2008)
27. Özyürek, A.: Speech-gesture relationship across languages and in second language learners: Implications for spatial thinking and speaking. In: Proceedings of the 26th Boston University Conference on Language Development, pp. 500-509 (2002)
28. Sowa, T., Kopp, S.: A cognitive model for the representation and processing of shape-related gestures. In: Proc. European Cognitive Science Conference (2003)
29. Swets, B., Jacovina, M.E., Gerrig, R.J.: Effects of conversational pressures on speech planning. Discourse Processes 50(1), 23-51 (2013)