Embodied communicative activity in cooperative conversational interactions - studies in visual interaction management

Acta Polytechnica Hungarica

Volumn 9, Issue 1, 2012, Pages 19-40

  Jokinen, Kristiina ^a   Scherer, Stefan ^b  

^a UNIVERSITY OF HELSINKI   (Finland)

^b TRINITY COLLEGE DUBLIN   (Ireland)

Author keywords

Embodied communicative activity; Interaction management; Naturalistic interaction

Indexed keywords

EID: 84859644889     PISSN: 17858860     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (40)

1. J. Allwood. Bodily communication - dimensions of expression and content. In B. Granström, D. House, and I. Karlsson, editors, Multimodality in Language and Speech Systems, pages 7-26. Kluwer, 2002.
2. J. Allwood, L. Cerrato, K. Jokinen, C. Navarretta, and P. Paggio. The mumin coding scheme for the annotation of feedback, turn management and sequencing phenomena. Multimodal corpora for modelling human multimodal behaviour. Special issue of the International Journal of Language Resources and Evaluation, 41(3-4):273-287, 2007.
3. E. André, T. Rist, and J. Müller. Employing ai methods to control the behavior of animated interface agents. Applied Artificial Intelligence, 13:415-448, 1999.
4. M. Argyle and M. Cook. Gaze and mutual gaze. Cambridge University Press, 1976.
5. J. Bavelas and J. Gerwing. Linguistic influences on gesture's form. Gesture, 4(2):157-195, 2004.
6. M. Bennewitz, F. Faber, D. Joho, and S. Behnke. Fritz - a humanoid communication robot. In Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2007.
7. N. Campbell and K Jokinen. Non-verbal information sources for constructive dialogue management. In In Proc. of Language Resources and Evaluation Conference (LREC). ELRA, 2008.
8. N. Campbell and S. Scherer. Comparing measures of synchrony and alignment in dialogue speech timing with respect to turn-taking activity. In Proceedings of Interspeech. ISCA, 2010.
9. J. Cassell, Y. Nakano, T. W. Bickmore, C. Sidner, and C. Rich. Non-verbal cues for discourse structure. In Proceedings of the 41st Annual Meeting of the ACL, pages 106-115, 2001.
10. J. Cassell, H. H. Vilhjálmsson, and T. W. Bickmore. Beat: the behavior expression animation toolkit. In Proceedings of SIGGRAPH, pages 477-486. ACM Press, 2001.
11. H. Clark and E. F. Schaefer. Contributing to discourse. Cognitive Science, 13:259-294, 1989.
12. M. C. Corballis. From Hand to Mouth. The origins of language. Princeton University Press, 2002.
13. C. E. Douglas, N. Campbell, R. Cowie, and P. Roach. Emotional speech: Towards a new generation of databases. Speech Communication, 40:33-60, 2003.
14. R. S. Feldman and B. Rim. Fundamentals of Nonverbal Behavior. Cambridge University Press, 1991.
15. M. Gullberg. A helping hand? gestures, l2 learners, and grammar. In S. McCafferty and G. Stam, editors, Gesture, Second Language acquisition and Classroom research. Applied Linguistics Series, pages 185-210. Routledge, 2008.
16. D. Heylen, M. Ghijsen, A. Nijholt, and R. op den Akker. Facial signs of affect during tutoring sessions. In J. Tao, T. Tan, and R. Picard, editors, Proceedings of Affective Computing and Intelligent Interaction (ACII 2005), pages 24-31. Springer, 2005.
17. E. Horvitz, C. Kadie, T. Paek, and D. Hovel. Models of attention in computing and communication: from principles to applications. Communications of ACM, 43(3):52-59, 2003.
18. H. Jaeger. Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the echo state network approach. Technical Report 159, Fraunhofer-Gesellschaft, St. Augustin Germany, 2002.
19. H. Jaeger and H. Haas. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304:78-80, 2004.
20. K. Jokinen. Constructive Dialogue Management - Speech Interaction and Rational Agents. Wiley, 2009.
21. K. Jokinen. Gestures and synchronous communication management. In A. Esposito, N. Campbell, C. Vogel, A. Hussain, and A. Nijholt, editors, Development of Multimodal Interfaces: Active Listening and Synchrony, volume 5967, pages 33-49. Springer, 2010.
22. K. Jokinen, P. Paggio, and C. Navarretta. Distinguishing the communicative functions of gestures. an experiment with annotated gesture data. machinelearning for multimodal interaction. In Proceedings of 5th International Workshop, MLMI 2008, pages 38-49. Springer, 2008.
23. K. Jokinen and M. Vanhasalo. Stand-up gestures - annotation for communication management. In Proceedings of the Multimodal Workshop at Nodalida Conference, 2009.
24. Y. Katagiri. Interactional alignment in collaborative problem solving dialogues. In Proceedings of 9th International Pragmatics Conference, 2005.
25. A. Kendon. Gesture: Visible action as utterance. Cambridge, 2004.
26. M. Kipp. Gesture Generation by Imitation: From Human Behavior to Computer Character Animation. Dissertation.com, Boca Raton, Florida, 2005.
27. H. Koiso, Y. Horiuchi, S. Tutiya, A. Ichikawa, and Y. Den. An analysis of turn taking and backchannels based on prosodic and syntactic features in japanese map task dialogs. Language and Speech, 41(3-4):295-321, 1998.
28. A. F. Krause, B. Blaesing, V. Duerr, and T. Schack. Direct control of an active tactile sensor using echo state networks. In H. Ritter, G. Sagerer, R. Dillmann, and M. Buss, editors, Proceedings of 3rd International Workshop on HumanCentered Robotic Systems (HCRS'09), Cognitive Systems Monographs, pages 11-21. Springer, 2009.
29. S. Levinson. Activity types and language. In Drew P. and Heritage J., editors, Talk at work, pages 67-107. Cambridge University Press., 1992.
30. J. C. Martin, R. Niewiadomski, L. Devillers, S. Buisine, and C. Pelachaud. Multimodal complex emotions: gesture expressivity and blended facial expressions. International Journal of Humanoid Robotics. special issue on Achieving Human-Like Qualities in Interactive Virtual and Physical Humanoids, 3(3):269-291, 2006.
31. Y. Nakano and T. Nishida. Attentional behaviours as nonverbal communicative signals in situated interactions with conversational agents. In T. Nishida, editor, Conversational Informatics, chapter 5, pages 85-102. Wiley, 2007.
32. E. Padilha and J. Carletta. Nonverbal behaviours improving a simulation of small group discussion. In Proceedings of the 1st Nordic Symposium on Multimodal Communication, pages 93-105, 2003.
33. C. Pelachaud, V. Carofiglio, B. D. Carolis, F. de Rosis, and I. Poggi. Embodied contextual agent in information delivering application. In Proceedings of the First international joint conference on Autonomous agents and multiagent systems (AAMAS'02), pages 758-765. ACM Press, 2002.
34. M. Pickering and S. Garrod. Towards a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27:169-226, 2004.
35. M. Rehm, Y. Nakano, E. André, T. Nishida, N. Bee, B. Endrass, M. Wissner, A. Lipi, and H.-H. Huang. From observation to simulation: generating culturespecific behavior for interactive systems. AI Soc., 24(3):267-280, 2009.
36. T. Rist, E. André, and S. Baldes. A flexible platform for building applications with life-like characters. In Proceedings of the 8th international conference on Intelligent user interfaces (IUI'03), pages 158-168. ACM Press, 2003.
37. S. Scherer, M. Oubbati, F. Schwenker, and G. Palm. Real-time emotion recognition from speech using echo state networks. In Proceedings of the 3rd IAPR workshop on Artificial Neural Networks in Pattern Recognition (AN- NPR'08), pages 205-216, Berlin, Heidelberg, 2008. Springer.
38. S. Scherer, F. Schwenker, W. N. Campbell, and G. Palm. Multimodal laughter detection in natural discourses. In H. Ritter, G. Sagerer, R. Dillmann, and M. Buss, editors, Proceedings of 3rd International Workshop on Human- Centered Robotic Systems (HCRS'09), Cognitive Systems Monographs, pages 111-121. Springer, 2009.
39. M. H. Tong, A. D. Bickett, E. M. Christiansen, and G. W. Cottrell. Learning grammatical structure with echo state networks. Neural Networks, 20:424-432, 2007.
40. N. Ward and W. Tsukahara. Prosodic features which cue back-channel responses in english and japanese. Journal of Pragmatics, 23:1177-1207, 2000.