Gesture and speech in interaction: An overview

Speech Communication

Volumn 57, Issue , 2014, Pages 209-232

  Wagner, Petra ^a   Malisz, Zofia ^a   Kopp, Stefan ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

SPEECH;

FUNCTIONAL INTERACTION; GESTURE ANALYSIS; GESTURE INTERACTION; HEAD GESTURES; LANGUAGE PRODUCTION; TECHNICAL SYSTEMS;

SPEECH COMMUNICATION;

EID: 84888307476     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2013.09.008     Document Type: Editorial

References (227)

1. David Abercrombie Gesture ELT Journal 9 1 1954 3 12
2. Hatice Çinar Akakin, and Bülent Sankur Robust classification of face and head gestures in video Image Vision Computing 29 7 2011 470 483
3. S. al Moubayed, J. Beskow, and B. Granström Auditory visual prominence: from intelligence to behavior Journal on Multimodal User Interfaces 3 4 2010 299 309
4. Martha Wagner Alibali, and Alyssa A. DiRusso The function of gesture in learning to count: more than keeping track Cognitive Development 14 1 1999 37 56
5. M.W. Alibali, D.C. Heath, and H.J. Myers Effects of visibility between speaker and listener on gesture production: some gestures are (Pubitemid 33379929)
6. M.W. Alibali, S. Kita, and A.J. Young Gesture and the process of speech production: we think, therefore we gesture Language and Cognitive Processes 15 2000 593 613 (Pubitemid 32002346)
7. Jens Allwood, Cerrato Loredana, Kristiina Jokinen, Constanza Navarretta, and Patrizia Paggio The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena Language Resources and Evaluation 41 2007 273 287
8. Jens Allwood, and Loredana Cerrato A study of gestural feedback expressions First Nordic Symposium on Multimodal Communication 2003 Copenhagen Denmark 7 22
9. Andreas Altorfer, Stefan Jossen, Othmar Würmle, Marie-Louise Käsermann, Klaus Foppa, and Heinrich Zimmermann Measurement and meaning of head movements in everyday face-to-face communicative interaction Behaviour Research Methods Instruments and Computers 32 1 2000 17 32
10. Janet B. Bavelas Gestures as part of speech: methodological implications Research on Language and Social Interaction 27 1994 201 221
11. J. Bavelas, J. Gerwing, C. Sutton, and D. Prevost Gesturing on the telephone: independent effects of dialogue and visibility Journal of Memory and Language 58 2008 495 520
12. Janet B. Bavelas, Linda Coates, and Trudy Johnson Listener responses as a collaborative process: the role of gaze Journal of Communication 52 3 2006 566 580 (Pubitemid 37164239)
13. Becker, Raymond, Cienki, Alan, Bennett, Austin, Cudina, Christina, Debras, Camille, Fleischer, Zuzanna, Haaheim, Michael, Müller, Torsten, Stec, Kashmiri, Zarcone, Alessandra. 2011. Aktionsarten, speech and gesture. In Proceedings of GESPIN2011: Gesture and Speech in Interaction, Bielefeld, Germany.
14. Mary Beckman, and Janet Pierrehumbert Intonational structure in Japanese and English Phonology Yearbook 3 1986 255 309
15. Kirsten Bergmann, and Stefan Kopp Verbal or visual? how information is distributed across speech and gesture in spatial dialogue David Schlangen, R. Fernandez, Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue 2006 Universitätsverlag Potsdam, Germany 90 97
16. Kirsten Bergmann, and Stefan Kopp Increasing expressiveness for virtual agents - autonomous generation of speech and gesture for spatial description tasks K. Decker, J. Sichman, G. Sierra, C. Castelfranchi, Proceedings of the Eighth International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009) 2009 Ann Arbor MI 361 368 IFAAMAS
17. Kirsten Bergmann, and Stefan Kopp Systematicity and idiosyncrasy in iconic gesture use: empirical analysis and computational modeling Gesture in Embodied Communication and Human-Computer Interaction 2010 Springer Berlin 182 194
18. Bergmann, Kirsten, Macedonia, and Manuela A virtual agent as vocabulary trainer: Iconic gestures help to improve learners' memory performance R. Aylett, B. Krenn, C. Pelachaud, H. Shimodaira, Intelligent Virtual Agents Lecture Notes in Artificial Intelligence 2013 Springer Berlin, Heidelberg 139 148
19. Kirsten Bergmann, Stefan Kopp, and Friederike Eyssel Individualized gesturing outperforms average gesturing - evaluating gesture production in virtual humans J. Allbeck, N. Badler, T. Bickmore, C. Pelachaud, A. Safonova, Proceedings of the 10th International Conference on Intelligent Virtual Agents (LNCS 6356) 2010 Springer Berlin, Heidelberg 104 117
20. Bergmann, Kirsten, Aksu, Volkan, Kopp, Stefan. 2011. The relation of speech and gestures: temporal synchrony follows semantic synchrony. In: Proceedings of GESPIN2011: Gesture and Speech in Interaction, Bielefeld, Germany.
21. Bergmann, Kirsten, Kahl, Sebastian, Kopp, Stefan. 2013. Modeling the semantic coordination of speech and gesture under cognitive and linguistic constraints. In: Proceedings of the International Conference on Intelligent Virtual Agents (IVA 2013).
22. Beskow, J., Granström, B., House, D. 2006. Focal accent and facial movements in expressive speech Fonetic 2006, Working Papers, pp. 52:9-52:12.
23. J. Beskow, B. Granström, and D. House Analysis and synthesis of multimodal verbal and non-verbal interaction for animated interface agents A. Esposito, M. Faundez-Zanuy, E. Keller, M. Marinaro, Verbal and Nonverbal Communication Behaviours 2007 Springer Berlin 250 263
24. Bevacqua, Elisabetta. 2009. Computational Model of Listener Behavior for Embodied Conversational Agents. Ph.D Thesis, Université Paris 8, Paris, France.
25. Ray L. Birdwhistell Introduction to Kinesics: An Annotation System for Analysis of Body Motion and Gesture 1952 University of Michigan Press
26. Ray L. Birdwhistell Kinesics and Context Essays on Body Motion Communication 1970 University of Pennsylvania Press Philadelphia, PA
27. Boersma, Paul, Weenink, David. 2008. Praat: Doing phonetics by computer (version 5.0.13). , March 18.
28. Dwight Bolinger Contrastive accent and contrastive stress Language 37 1961 87 96
29. Dwight Bolinger Intonation in nondeclaratives Chicago Linguistics Society Parasession on Nondeclaratives 1982 Chicago Linguistics Society Chicago 1 15
30. Dwight Bolinger Intonation and gesture American Speech 58 1983 156 174
31. Dwight Bolinger Intonation and Its Parts: Melody in Spoken English 1986 Stanford University Press
32. Konstantinos Bousmalis, Marc Mehu, and Maja Pantic Towards the automatic detection of spontaneous agreement and disagreement based on nonverbal behaviour: a survey of related cues databases and tools Image and Vision Computing 31 2 2012 203 221
33. Jana Bressem, and Silva Ladewig Rethinking gesture phases: articulatory features of gestural movement Semiotica 184 1/4 2011 53 91
34. Catherine Browman, and Louis. Goldstein Towards an articulatory phonology Phonology Yearbook 3 1986 219 252
35. Brugman, Hennie, Russel, Albert. 2004. Annotating multimedia/multi-modal resources with elan. In: Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC), Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen, The Netherlands, pp. 2065-2068. .
36. Brugman, Hennie, Wittenburg, Peter, Levinson, Stephen C., Kita, Sotaro. 2002. Multimodal annotations in gesture and sign language studies. In: Third International Conference on Language Resources and Evaluation, pp. 176-182.
37. Peter Bull, and Gerry Connelly Body movement and emphasis in speech Journal of Nonverbal Behavior 9 3 1985 169 187
38. Samuel R. Buss 3D Computer Graphics: A Mathematical Introduction with Open GL 2003 Cambridge University Press
39. B. Butterworth, and G. Beattie Gesture and silence as indicators of planning in speech R.N. Campbell, P.T. Smith, Recent Advances in the Psychology of Language: Formal and Experimental Approaches 1978 Plenum New York
40. B. Butterworth, and U. Hadar Gesture, speech, and computational stages: A reply to mcneill Psychological Review 96 1989 168 174
41. Cafaro, Angelo., Vilhjálmsson, Högni Hannes, Bickmore, Timothy W., Heylen, Dirk, Johannsdottir, Kamilla R., Valgardsson, Gunnar Steinn. 2012. First impressions: Users' judgments of virtual agents' personality and interpersonal attitude in first encounters. In: IVA, pp. 67-80.
42. Caldognetto, Emanuela M., Poggi, Isabella, Cosi, Piero, Cavicchio, Federica, Merola, G. 2004. Multimodal score: an ANVIL based annotation scheme for multimodal audio-video analysis. In: Martin, J.-C., Os, E.D., Kühnlein, P., Boves, L., Paggio, P., Catizone, R. (Eds.), Proceedings of the LREC-Workshop on Multimodal Corpora, Lisbon, Portugal, pp. 29-33.
43. Caldognetto, Magno, Poggi, Isabella. 2001. Dall'analisi della multimodalità quotidiana alla costruzione di agenti animati con facce parlanti ed espressive. In: Cosi, P., Caldognetto, M. (Eds.), Multimodalità e multimedialità della communicatione, Atti delle XI Giornate di Studio del G.F.S. Unipress, Padova.
44. Justine Cassell A framework for gesture generation and interpretation R. Cipolla, A. Pentland, Computer Vision in Human-Machine Interaction 1996 Cambridge University Press
45. Justine Cassell, Joseph Sullivan, Scott Prevost, and Elizabeth F. Churchill Embodied Conversational Agents 2000 MIT Press
46. Cerrato, Loredana. 2007. Investigating Communicative Feedback Phenomena across Languages and Modalities. Ph.D Thesis, KTH Computer Science and Communication, Department of Speech, Music and Hearing, Stockholm, Sweden.
47. Chiu, Chung-Cheng, Marsella, Stacy, 2011. How to train your avatar a data driven approach to gesture generation. In: The 11th International Conference on Intelligent Virtual Agents (IVA), Taipei, Taiwan, May 2-6.
48. Nicholas Christenfeld, Stanley Schachter, and Frances Bilous Filled pauses and gestures: it's not coincidence Journal of Psycholinguistic Research 20 1 1991 1 10
49. K. Chui Temporal patterning of speech and iconic gestures in conversational discourse Journal of Pragmatics 37 2005 871 887 (Pubitemid 40251014)
50. Alan Cienki, and Cornelia Müller Metaphor, gesture, and thought R.W. Gibbs Jr. The Cambridge Handbook of Metaphor and Thought 2008 Cambridge University Press 483 502
51. Herbert H. Clark, and Deanna Wilkes-Gibbs Referring as a collaborative process Cognition 22 1986 1 39
52. Herbert H. Clark, and Susan Brennan Grounding in communication G. Lockhead, J. Pomerantz, Perspectives on Socially Shared Cognition 1991 APA Books Washington 127 149
53. W.S. Condon, and W.D. Ogston Speech and body motion synchrony of the speaker-hearer D.L. Horton, J.J. Jenkins, Perception of Language 1971 Merrill Columbus, Ohio
54. A. Corradini Real-time gesture recognition by means of hybrid recognizers Ipke Wachsmuth, Timo Sowa, Gesture and Sign Language in Human-Computer Interaction 2002 Springer Berlin, Heidelberg, New York 34 46
55. J.P. de Ruiter, A. Bangerter, and P. Dings Interplay between gesture and speech in the production of referring expressions: investigating the tradeoff hypothesis Topics in Cognitive Science 4 2 2012 232 248
56. J.P. de Ruiter The production of gesture and speech David McNeill, Language and Gesture 2000 Cambridge University Press 248 311
57. J.P. de Ruiter The production of gesture and speech David McNeill, Language and Gesture 2000 Cambridge University Press Cambridge, UK 284 311
58. J.P. de Ruiter, M.L. Noordzij, S. Newman-Norlund, R. Newman-Norlund, P. Hagoort, S.C. Levinson, and I. Toni Exploring the cognitive infrastructure of communication Interaction Studies 11 1 2010 51 77
59. D. DeSteno, C. Breazeal, R.H. Frank, D. Pizarro, J. Baumann, L. Dickens, and J. Lee Detecting the trustworthiness of novel partners in economic exchange Psychological Science 23 2012 1549 1556
60. A.T. Dittmann, and L.G. Llewellyn Relationship between vocalizations and head nods as listener responses Journal of Personality and Social Psychology 9 1968 79
61. A.T. Dittmann, and L.G. Llewelyn Body movement and speech rhythm in social conversation Journal of Personality and Social Psychology 23 1969 283 292
62. S.M. Dobrogaev Ucnenie o reflekse v problemakh iasykovedeniia (observations on reflexes and issues in language study) Iazykovedenie i Materializm 1929 105 173
63. Starkey Duncan Some signals and rules for taking speaking turns in conversations Journal of Personality and Social Psychology 23 1972 283 292
64. Eickeler, Stefan, Kosmala, Andreas, Rigoll, Gerhard. 1998. Hidden Markov model based continuous online gesture recognition. In: Proceedings of the International Conference on Pattern Recognition (ICPR), Brisbane, Australia, August 1998, pp. 1206-1208.
65. P. Ekman About brows: emotional and conversational signals M. von Cranach, K. Foppa, W. Lepenies, D. Ploog, Human Ethology: Claims and Limits of a New Discipline 1979 Cambridge University Press Cambridge 169 202
66. P. Ekman, and W.V. Friesen Hand movements Journal of Communication 22 1972 353 374
67. Eriksson, A., Thunberg, G., Traunmüller, H. 2001. Syllable prominence: a matter of vocal effort, phonetic distinctness and top-down processing. In: Proceedings of EUROSPEECH, Aalborg, Denmark, pp. 399-402.
68. Esteve-Gibert, Núria, Prieto, Pilar. Infants temporally coordinate gesture-speech combinations before they produce their first words. Speech Communication, this issue.
69. Fernández-Baena, Adso, Montaño, Raúl, Antonijoan, Marc, Roversi, Arturo, Miralles, David, Alías, Francesc. Gesture synthesis adapted to speech emphasis. Speech Communication, this issue.
70. Ferré, G. 2010. Timing relationships between speech and co-verbal gestures in spontaneous French. In: Language Resources and Evaluation Conference (LREC). Workshop on Multimodal Corpora, Malta.
71. Ferré, Gaëlle. A multimodal approach to markedness in spoken French. Speech Communication, this issue.
72. P. Feyereisen Gestures and speech, interactions and separations: a reply to McNeill Psychological Review 94 1987 168 174
73. Maurizio Gentilucci, and Riccardo Dalla Volta The motor system and the relationship between speech and gesture Gesture 7 2 2007 159 177
74. S. Goldin-Meadow, H. Nusbaum, S.D. Kelly, and S. Wagner Explaining math: gesture lightens the load Psychological Science 12 6 2001 516 522 (Pubitemid 33653571)
75. Susan Goldin-Meadow The role of gesture in communication and thinking Trends in Cognitive Sciences 3 11 1999 419 429 (Pubitemid 29499810)
76. Susan Goldin-Meadow Beyond words: the importance of gesture to researchers and learners Child Development 71 1 2003 231 239
77. Susan Goldin-Meadow, M.W. Alibali, and S.W. Church Transitions in concept acquisition: using the hand to read the mind Psychological Review 100 1993 279 297
78. John Goldsmith Autosegmental and Metrical Phonology 1990 Blackwell Oxford
79. Charles Goodwin Conversational organization: interaction between speakers and hearers 1981 Academic Press New York
80. Granström, Björn, House, David. 2007. Inside out - acoustic and visual aspects of verbal and non-verbal communication. In Proceedings of the XVIth International Congress of the Phonetic Sciences, Saarbrücken, Germany, pp. 11-18.
81. Agustin Gravano, and Julia Hirschberg Turn-taking cues in task-oriented dialogue Computer, Speech and Language 25 3 2011 601 634
82. Carlos Gussenhoven Discreteness and gradience in intonational contrasts Language and Speech 42 2-3 1999 283 305
83. Ulrike Gut, and Jan Thorsten Milde Annotation and analysis of conversational gestures in the TASX environment KI 17 2003 34 49
84. B. Habets, S. Kita, Z. Shao, A. Özyürek, and P. Hagoort The role of synchrony and ambiguity in speech-gesture integration during comprehension Journal of Cognitive Neuroscience 23 8 2011 1845 1854
85. U. Hadar, T.J. Steiner, E.C. Grant, and F. Clifford Rose Kinematics of head movements accompanying speech during conversation Human Movement Science 2 1983 35 46
86. Uri Hadar, T.J. Steiner, E.C. Grant, and F. Clifford Rose The timing of shifts of head postures during conversation Human Movement Science 3 1984 237 245 (Pubitemid 14008247)
87. Uri Hadar, T.J. Steiner, and Clifford F. Rose Head movement during listening turns in conversation Journal of Nonverbal Behavior 9 1985 214 228
88. P. Harling, and A. Edwards Hand tension as a gesture segmentation cue P. Harling, A. Edwards, Progress in Gestural Interaction: Proceedings of the Gesture Workshop 1996 1997 Springer Berlin, Heidelberg, New York 75 87
89. Harrison, S. 2013. The temporal coordination of negation gestures in relation to speech. In: Proceedings of TiGeR 2013, Tilburg, NL.
90. Björn Hartmann, Maurizio Mancini, and Catherine Pelachaud Implementing expressive gesture synthesis for embodied conversational agents Sylvie Gibet, Nicolas Courty, Jean-François Kamp, Gesture in Human-Computer Interaction and Simulation 2006 Springer Berlin, Heidelberg 188 199 (Pubitemid 43980007)
91. Mattias Heldner, Anna Hjalmarsson, and Jens Edlund Continuer relevance spaces Eva-Liina Asu, Pärtel Lippus, Nordic Prosody. Proceedings of the XIth Conference, Tartu 2012 2013 Peter Lang Frankfurt 137 146
92. Heylen, Dirk. 2005. Challenges ahead: head movements and other social acts in conversations. In: Proceedings of AISB 2005, pp. 45-52.
93. Dirk Heylen Head gestures, gaze and the principles of conversational structure International Journal of Humanoid Robotics 3 03 2006 241 267 (Pubitemid 44604950)
94. Heylen, and Dirk Listening heads Ipke Wachsmuth, Guenther Knoblich, Modeling Communication with Robots and Virtual Humans 2008 Springer Berlin 241 259 (Pubitemid 351641674)
95. Dirk Heylen, Elisabetta Bevacqua, Catherine Pelachaud, Isabella Poggi, Jonathan Gratch, and Marc Schröder Generating listening behaviour Paolo Petta, Catherine Pelachaud, Roddy Cowie, Emotion-Oriented Systems: The Humaine Handbook 2011 Springer Berlin 321 348
96. Hoetjes, Marieke, Krahmer, Emiel, Swerts, Marc. Does our speech change when we cannot gesture? Speech Communication, this issue.
97. J. Holler, and R. Stevens The effect of common ground on how speakers use gesture Journal of Language and Social Psychology 26 2009 4 27
98. Judith Holler, and Geoffrey Beattie Pragmatic aspects of representational gestures Gesture 3 2 2003 127 154
99. Judith Holler, and Rachel Stevens The effect of common ground on how speakers use gesture and speech to represent size information Journal of Language and Social Psychology 26 4 2007 4 27
100. A.B. Hostetter, and M.W. Alibali Raise your hand if you're spatial: relations between verbal and spatial skills and gesture production Gesture 7 1 2007 73 95
101. A.B. Hostetter, and A.L. Potthoff Effects of personality and social situation on representational gesture production Gesture 12 1 2012 62 83
102. Autumn B. Hostetter, and Martha Wagner Alibali Visible embodiment: gestures as simulated action Psychonomic Bulletin and Review 15 2008 495 514
103. Autumn B. Hostetter, Martha Wagner Alibali, and Sotaro Kita I see it in my hand's eye: representational gestures reflect conceptual demands Language and Cognitive Processes 22 3 2007 313 336 (Pubitemid 47039387)
104. Ishi, Carlos Toshinori, Ishiguro, Hiroshi, Hagita, Norihiro, Analysis of relationship between head motion events and speech in dialogue conversation. Speech Communication, this issue.
105. Ryo Ishii, and Yikiko I. Nakano Estimating user's conversational engagement based on gaze behaviors Intelligent Virtual Agents Lecture Notes in Computer Science vol. 5208 2008 Springer 200 207
106. J.M. Iverson, and E. Thelen Hand, mouth and brain - the dynamic emergence of speech and gesture Journal of Consciousness Studies 6 11-12 1999 19 40
107. J.M. Iverson, and S. Goldin-Meadow Why do people gesture as they speak? Nature 396 1998 228 (Pubitemid 28541379)
108. Roman Jakobson Motor signs for 'yes' and 'no' Language and Society 1 1972 91 96
109. S. Jannedy, and N. Mendoza-Denton Structuring information through gesture and intonation Interdisciplinary Studies on Information Structure 3 2005 199 244
110. Jokinen, Kristiina, Nishida, Masfumi, Yamamoto, Seiichi. 2010. On eye-gaze and turn-taking. In: Proceedings of EGIHMI'10, New York, USA, pp. 118-123.
111. Sun-Ah Jun Prosodic typology Sun-Ah Jun, Prosodic Typology: The Phonology of Intonation and Phrasing 2007 Oxford University Press Oxford, New York 430 462 chapter 16
112. Maciej Karpiński, Ewa Jarmołowicz-Nowikow, and Zofia Malisz Aspects of gestural and prosodic structure of multimodal utterances in Polish task-oriented dialogues Speech and Language Technology 11 2009 113 122
113. J.A.S. Kelso, B. Tuller, and K.S. Harris A dynamic pattern perspective on the control and coordination of movement P.F. McNeilage, The Production of Speech 1983 Springer-Verlag New York 138 173
114. Adam Kendon Some relationships between body motion and speech A. Siegman, B. Pope, Studies in Dyadic Communication 1972 Pergamon New York 177 210
115. Adam Kendon Gesture and speech: two aspects of the process of utterance Mary R. Key, Nonverbal Communication and Language 1980 Mouton The Hague 207 227
116. Adam Kendon Some uses of the head shake Gesture 2 2003 147 182
117. Adam Kendon Gesture - Visible Action as Utterance 2004 Cambridge University Press
118. Kim, Jeesun, Cvejic, Erin, Davis, Christopher, Tracking eyebrows and head gestures associated with spoken prosody. Speech Communication, this issue.
119. Kipp, Michael. 2001. From human gesture to synthetic action. In: Proceedings of the Workshop on "Multimodal Communication and Context in Embodied Agents" held in conjunction with the Fifth International Conference on Autonomous Agents (AGENTS), Montreal, Canada, May.
120. Kipp, Michael. 2004. Gesture generation by imitation: From human behavior to computer character animation. Ph.D Thesis, Saarland University.
121. Kipp, Michael, 2012. Multimodal annotation, querying and analysis in ANVIL. In: Maybury, Mark T. (Ed.), Multimedia information extraction, John Wiley and Sons Inc, Hoboken, NJ, pp. 531-368.
122. Michael Kipp, and J.-C. Martin Gesture and emotion: can basic gestural form features discriminate emotions? Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII-09) 2009 IEEE Press
123. Michael Kipp, Jean-Claude Martin, Patrizia Paggio, and Dirk Heylen Multimodal Corpora: From Models of Natural Interaction to Systems and Applications Lecture Notes in Computer Science vol. 5509 2009 Springer Berlin, Heidelberg
124. Michael Kipp, Michael Neff, and Irene Albrecht An annotation scheme for conversational gestures: how to economically capture timing and form Language Resources and Evaluation 41 2007 325 339
125. Kirchhof, Carolin. 2011. So what's your affiliation with gesture? In: Proceedings of GESPIN2011: Gesture and Speech in Interaction, Bielefeld, Germany.
126. Kirchhof, Carolin, de Ruiter, J.P. 2012. On the audiovisual integration of speech and gesture. In: Fifth Conference of the International Society for Gesture Studies, Lund, Sweden.
127. S. Kita, and T.S. Davies Competing conceptual representations trigger co-speech representational gestures Language and Cognitive Processes 24 5 2009 795 804
128. Sotaro Kita How representational gestures help speaking David McNeill, Language and Gesture 2000 Cambridge University Press 162 185
129. Sotaro Kita, and Asli Özyürek What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: evidence for an interface representation of spatial thinking and speaking Journal of Memory and Language 48 1 2003 16 32 (Pubitemid 36177475)
130. Sotaro Kita, Ingeborg van Gijn, and Harry van der Hulst Movement phases in signs and co-speech gestures* and their transcription by human coders Ipke Wachsmuth, Martin Fröhlich, Gesture and Sign Language in Human-Computer Interaction Lecture Notes in Artificial Intelligence vol. 1371 1998 Springer 23 35
131. Dawn Knight The future of multimodal corpora RBLA 2 2011 391 415
132. Stefan Kopp, Jens Allwood, Karl Grammar, Elisabeth Ahlsén, and Thorsten Stocksmeier Modeling embodied feedback with virtual humans Ipke Wachsmuth, Günther Knoblich, Modeling Communication with Robots and Virtual Humans 2008 Springer-Verlag Berlin 18 37 (Pubitemid 351641664)
133. Kopp, Stefan, Bergmann, Kirsten, Kahl, Sebastian. 2013. A spreading-activation model of the semantic coordination of speech and gesture. In: Proceedings of the 35th Annual Meeting of the Cognitive Science Society (COGSCI 2013).
134. Stefan Kopp, B. Krenn, S. Marsella, A. Marshall, C. Pelachaud, H. Pirker, K. Thorisson, and H. Vilhjálmsson Towards a common framework for multimodal generation in ecas: the behavior markup language Proceedings of Intelligent Virtual Agents (IVA) J. Gratch, LNAI vol. 4133 2006 Springer-Verlag 205 217 (Pubitemid 44532615)
135. Kopp, Stefan, Tuller, B., Cassell, Justine. 2004. Towards integrated microplanning of language and iconic gesture for multimodal output. In: Sharma, R., Darrell, T. (Eds.), Proceedings of the International Conference on Multimodal Interfaces (ICMI'04), pp. 97-104.
136. Kousidis, Spyros, Malisz, Zofia, Wagner, Petra, Schlangen, David. 2013. Exploring annotation of head gesture forms in spontaneous human interaction. In: TiGeR 2013, Tilburg Gesture Research Meeting.
137. Kousidis, Spyros, Pfeiffer, Thies, Malisz, Zofia, Wagner, Petra, Schlangen, David. 2012. Evaluating a minimally invasive laboratory architecture for recording multimodal conversational data. In: Proceedings of the Interdisciplinary Workshop on Feedback Behaviours in Dialogue, Skamania Lodge, Stevenson, WA.
138. Emiel Krahmer, and Marc Swerts The effects of visual beats on prosodic prominence: acoustic analyses, auditory perception and visual perception Journal of Memory and Language 57 3 2007 396 414
139. R.M. Krauss, and U. Hadar The role of speech-related arm/hand gestures in word retrieval R. Campbell, L. Messing, Gesture, Speech, and Sign 1999 Oxford University Press 93 116
140. Krenn, Brigitte, Pirker, Hannes. 2004. Defining the gesticon: language and gesture coordination for interacting embodied agents. In: Proceedings of the AISB-2004 Symposium on Language, Speech and Gesture for Expressive Characters, Leeds, UK, March 29-April 1, pp. 107-115.
141. Mikhail Kunin, Yasuhiro Osaki, Bernard Cohen, and Theodore Raphan Rotation axes of the head during positioning, head shaking and locomotion Journal of Neurophysiology 98 5 2007 3095 3108 (Pubitemid 350127965)
142. John Lakoff, and Mark Johnson Methaphors We Live by 1980 The University of Chicago Press Chicago
143. Hedda Lausberg, and Han Sloetjes Coding gestural behaviour with the NEUROGE-ELAN system Behaviour Research Methods, Instruments, and Computers 41 3 2009 841 949
144. Lee, Jina, Marsella, Stacy. 2006. Nonverbal behavior generator for embodied conversational agents. In: Proceedings of Sixth International Conference on Intelligent Virtual Agents (IVA), Marine del Rey, USA, August 21-23, pp. 243-255.
145. Leonard, Thomas, Cummins, Fred. 2009. Temporal alignment of gesture and speech. In: Jarmołowicz-Nowikow, Ewa, Juszczyk, Konrad, Malisz, Zofia, Szczyszek, Michał (Eds.), Proceedings of GESPIN2009: Gesture and Speech in Interaction, Poznań, Poland, pp. 1-6.
146. Thomas Leonard, and Fred Cummins The temporal relation between beat gestures and speech Language and Cognitive Processes 26 10 2010 1457 1471
147. W.J.M. Levelt, G. Richardson, and W.L. Hey Pointing and voicing in deictic expressions Journal of Memory and Language 24 1985 133 164
148. Loehr, D. 2004. Gesture and Intonation. Ph.D Thesis, Georgetown University, Washington, D.C.
149. Daniel Loehr Temporal, structural, and pragmatic synchrony between intonation and gesture. Laboratory Phonology Journal of the Association for Laboratory Phonology 3 2012 71 89
150. H. Loevenbruck, M. Dohen, and C. Vialin Pointing is 'special' S. Fuchs, H. Loevenbruck, D. Pape, P. Perrier, Some Aspects of Speech and the Brain 2009 Peter Lang New York 211 258
151. Max M. Louwerse, Rick Dale, Ellen G. Bard, and Patrick Jeuniaux Behaviour matching in multimodal communication is synchronized Cognitive Science 36 8 2012 1404 1426
152. Andy Lücking, Kirsten Bergmann, Florian Hahn, Stefan Kopp, and Hannes Rieser Data-based analysis of speech and gesture: the Bielefeld speech and gesture alignment corpus (SaGA) and its applications Journal of Multimodal User Interfaces 7 1-2 2013 5 18
153. Martell, C., 2002. Form: An extensible, kinematically based gesture annotation scheme. In: Proceedings of ICSLP'02, pp. 353-356.
154. Craig. H. Martell Form - an extensible, kinematically based gesture annotation scheme J.C.J. van Kuppevelt, Advances in Natural Multimodal Dialogue Systems 2005 Springer 79 96
155. R.E. Mayer, and C.S. DaPra An embodiment effect in computer-based learning with animated pedagogical agents Journal of Experimental Psychology: Applied 18 2012 239 252
156. McClave, E. 1991. Intonation and Gesture. Ph.D Thesis, Georgetown University.
157. Evelyn McClave Gestural beats: the rhythm hypothesis Journal of Psycholinguistic Research 23 1 1994 45 66
158. Evelyn McClave Linguistic functions of head movements in the context of speech Journal of Pragmatics 32 2000 855 878
159. David McNeill So you think gestures are nonverbal? Psychological Review 92 1985 350 371
160. David McNeill So you do think gestures are nonverbal? reply to Feyereisen (1987) Psychological Review 92 1987 350 371
161. David McNeill A straight path to where? reply to Butterworth and Hadar Psychological Review 94 1989 499 504
162. David McNeill Hand and Mind: What Gestures Reveal about Thought 1992 University of Chicago Press Chicago
163. David McNeill Gesture and Thought 2005 University of Chicago Press
164. Inge Mertins, Roger Moore, and Dafydd Gibbon Handbook of Multimodal and Spoken Dialogue Systems: Resources and Product Evaluation 2000 Kluwer Academic Publishers Norwell
165. Morency, Louis-Philippe, Sidner, Candace, Lee, Christopher, Darrell, Trevor. 2005. Contextual recognition of head gestures. In: Proceedings of the Seventh International Conference on Multimodal Interfaces, pp. 18-24.
166. Palmer Morrel-Samuels, and Robert M. Krauss Word familiarity predicts temporal asynchrony of hand gestures and speech Journal of Experimental Psychology: Human Learning and Memory 18 3 1992 615 622
167. Kevin G. Munhall, Jeffery A. Jones, Daniel E. Callan, Takaaki Kuratate, and Eric Vatikiotis-Bateson Visual prosody and speech intelligibility Psychological Science 15 2 2004 133 137
168. Michael Neff, Michael Kipp, Irene Albrecht, and Hans-Peter Seidel Gesture modeling and animation based on a probabilistic re-creation of speaker style ACM Transactions on Graphics 27 1 2008 5.1 5.2
169. Shuichi Nobe Where to most spontaneous representational gestures actually occur with respect to speech? David McNeill, Language and Gesture 2000 Cambridge University Press 186 198
170. Catharine Oertel, Fred Cummins, Jens Edlund, Petra Wagner, and Nick Campbell D64 - a corpus of richly recorded conversational interaction Journal of Multimodal User Interfaces 7 1-2 2013 19 28
171. John Ohala An ethological perspective on common cross-language utilization of f0 of voice Phonetica 41 1984 1 16 (Pubitemid 14120527)
172. A. Özyürek, R.M. Willems, S. Kita, and P. Hagoort On-line integration of semantic information from speech and gesture: insights from event-related brain potentials Journal of Cognitive Neuroscience 19 4 2007 605 616 (Pubitemid 46495292)
173. Parrell, B., Goldstein, L., Lee, S., Byrd, D. 2011. Temporal coupling between speech and manual motor actions. In: Proceedings of the Ninth International Seminar on Speech Production.
174. Poggi, I., D'Errico, F., Vincze, L. 2010. Types of nods. The polysemy of a social signal. In: Proceedings of the Seventh International Conference on Language Resources and Evaluation, pp. 17-23.
175. Poggi, Isabella. 2001. The lexicon and the alphabet of gesture, gaze, and touch. In: Proceedings of Intelligent Virtual Agents - Third International Workshop (IVA 2001), pp. 235-236, Madrid, Spain, September 10-11.
176. Isabella Poggi, Francesca D'Errico, and Laura Vincze Comments by words, face and body Journal of Multimodal User Interfaces 7 2013 67 78
177. Priesters, M.A., Mittelberg, I. 2013. Individual differences in speakers' gesture spaces: multi-angle views from a motion-capture study. In: Proceedings of the Tilburg Gesture Research Meeting (TiGeR), June 19-21.
178. Toni Rietveld, and Carlos Gussenhoven On the relation between pitch excursion size and prominence Journal of Phonetics 13 1985 299 308
179. A. Rochet-Capellan, Rafael Laboissière, Arturo Galván, and Jean-Luc Schwartz The speech focus position effect on jaw-finger coordination in a pointing task Journal of Speech, Language and Hearing Research 51 6 2008 1507 1521
180. H.M. Rosenfeld, and M. Hancks The nonverbal context of verbal listener responses Mary Ritchie Key, The Relationship of Verbal and Nonverbal Communication 1980 Mouton Publishers The Hague, The Netherlands 193 206
181. Wolff-Michael Roth Gestures: their role in teaching and learning Review of Educational Research 71 3 2001 365 392 (Pubitemid 33402448)
182. Roustan, Benjamin, Dohen, Marion. 2010. Co-production of contrastive focus and manual gestures: temporal coordination and effects on the acoustic and articulatory correlates of focus. In: Speech Prosody 2010, Chicago, IL.
183. Rowbotham, Samantha J., Holler, Judith, Lloyd, Donna, Wearden, Alison. The semantic interplay of speech and co-speech gestures in the description of pain sensations. Speech Communication, this issue.
184. Rusiewicz, H.L., Shaiman, S., Iverson, J., Szuminsky, N. Effects of perturbation and prosody on the coordination of speech and gesture. Speech Communication, this issue.
185. Zsófia Ruttkay Presenting in style by virtual humans COST 2102 Workshop (Vietri) Anna Esposito, Marcos Faúndez-Zanuy, Eric Keller, Maria Marinaro, Lecture Notes in Computer Science vol. 4775 2007 Springer 978-3-540-76441-0 23 36
186. Maha Salem, Stefan Kopp, Ipke Wachsmuth, Katharina Rohlfing, and Frank Joublin Generation and evaluation of communicative robot gesture International Journal of Social Robotics 1875-4805 2012 201 217 Special Issue on Expectations, Intentions, and Actions
187. Sargin, Mehmet, Aran, Oya, Karpov, Alexey, Ofli, Ferda, Yasinnik, Yelena, Wilson, Stephen, Erzin, Engin, Yemez, Yucel, Tekalp, Murat A. 2006. Combined gesture-speech analysis and speech driven gesture synthesis. In: Proceedings of the IEEE International Conference on Multimedia and Expo, Toronto, Canada, pp. 893-896.
188. Mehmet E Sargin, Yucel Yemez, Engin Erzin, and Ahmet M Tekalp Analysis of head gesture and prosody patterns for prosody-driven head-gesture animation IEEE Transactions on Pattern Analysis and Machine Intelligence 30 8 2008 1330 1345
189. E.A. Schegloff On some gestures' relation to talk J.M. Atkinson, J. Heritage, Structures of Social Action 1984 Cambridge University Press Cambridge 266 298
190. Schmidt, T. 2004. Transcribing and annotating spoken language with EXMARaLDA. In: Proceedings of the LREC-Workshop on XML-based Richly Annotated Corpora, Lisbon. European Language Resources Association (ELRA).
191. Schröder, M., Cowie, R., Douglas-Cowie, E., Westerdijk, M., Gielen, S. 2001. Acoustic correlates of emotion dimensions in view of speech synthesis. In: Proceedings of EUROSPEECH, vol. 1, pp. 87-90.
192. Margret Selting, Peter Auer, Birgit Barden, Jörg Bergmann, Slizabeth Couper-Kuhlen, Susanne Günthner, Uta Quasthoff, Christoph Meier, Peter Schlobinski, and Susanne Uhmann Gesprächsanalytisches Transkriptionssystem (GAT) Linguistische Berichte 173 1998 91 122
193. Margret Selting, Peter Auer, Jörg Bergmann, Pia Bergmann, Karin Birkner, Elizabeth Couper-Kuhlen, Arnulf Deppermann, Peter Gilles, Susanne Günthner, Martin Hartung, Friederike Kern, Christine Mertzlufft, Christian Meyer, Miriam Morek, Frank Oberzaucher, Jörg Peters, Uta Quasthoff, Wilfried Schütte, Anja Stukenbrock, and Susanne Uhmann Gesprä chsanalytisches Transkriptionssystem 2 (GAT2) Gesprächsforschung - Online-Zeitschrift zur verbalen Interaktion 10 2009 353 402
194. Stefanie Shattuck-Hufnagel, Yelena Yasinnik, Nanette Veilleux, and Margaret Renwick A method for studying the time alignment of gestures and prosody in American English: 'hits' and pitch accents in academic-lecture-style speech Anna Esposito, Maja Bratanić, Eric Keller, Maria Marinaro, Fundamentals of Verbal and Nonverbal Communication and the Biometric Issue NATO Security Through Science series E: Human and Societal Dynamics vol. 18 2007 IOS Press Washington, DC
195. Dan I. Slobin From "thought and language" to thinking for speaking J.J. Gumperz, S.C. Levinson, Rethinking Linguistic Relativity 1996 Cambridge University Press Cambridge 70 96
196. W.C. So, S. Kita, and S. Goldin-Meadow Using the hands to identify who does what to whom: gesture and speech go hand-in-hand Cognitive Science 33 2009 115 125
197. D. Spoons, C. Sparrell, and K. Thorisson Integrating simultaneous input from speech, gaze and hand gestures M. Maybury, Intelligent Multimedia Interfaces 1993 AAAI Press/MIT Press Cambridge 257 276
198. Raymond Herbert Stetson Motor Phonetics 1951 North-Holland Amsterdam
199. Matthew Stone, Doug DeCarlo, Insuk Oh, Christian Rodriguez, Adrian Stere, Alyssa Lees, and Chris Bregler Speaking with hands: creating animated conversational characters from recordings of human performance ACM Transactions on Graphics 23 3 2004 506 513
200. Jürgen Streeck Depicting by gesture Gesture 8 3 2008 285 301
201. M. Swerts, and R. Geluykens Prosody as a marker of information flow in spoken discourse Language and Speech 37 1994 21 43
202. Marc Swerts, and Emiel Krahmer Facial expressions and prosodic prominence: comparing modalities and facial areas Journal of Phonetics 36 2 2008 219 238
203. Tamburini, Fabio, Wagner, Petra. 2007. On automatic prominence detection for German. In: Proceedings of Interspeech 2007, Antwerp, Belgium, pp. 1809-1812.
204. Jacques Terken Fundamental frequency and perceived prominence Journal of the Acoustical Society of America 89 4 1991 1768 1776
205. Mariët Theune, and Chris Brandhorst To beat or not to beat: beat gestures in direction giving Stefan Kopp, Ipke Wachsmuth, Gesture in Embodied Communication and Human-Computer Interaction Lecture Notes in Artificial Intelligence vol. 5934 2010 Springer 195 206
206. R.D. Tomlison, R. Cheung, and A. Blakeman Naso-occipital vestibulo-ocular reflex responses in normal subjects IEEE Engineering in Medicine and Biology Magazine 19 2 2000 43 47
207. Paul Treffner, Mira Peter, and Mark Kleidon Gestures and phases: the dynamics of speech-hand communication Ecological Psychology 20 2008 32 64
208. Trippel, Thorsten, Gibbon, Dafydd, Thies, Alexandra, Milde, Jan-Torsten, Looks, Karin, Hell, Benjamin, Gut, Ulrike. 2004. Cogest: a formal transcription system for conversational gesture. In: Proceedings of LREC 2004, Lisbon, Portugal.
209. Truong, Khiet P., Poppe, Ronald, de Kok, Iwan, Heylen, Dirk. 2011. A multimodal analysis of vocal and visual backchannels in spontaneous dialogs. In: Proceedings of INTERSPEECH 2011, Florence, Italy. International Speech Communication Association, pp. 2973-2976.
210. Kevin Tuite The production of gesture Semiotica 1/2 1993 83 105
211. Urban, Christian. 2011. Temporales Alignment von Sprache und Gestik, B.Sc. Thesis, Bielefeld University.
212. Zeno Vendler Linguistics in Philosophy 1967 Cornell University Press Ithaca, NY
213. H. Vilhjálmsson, N. Cantelmo, J. Cassell, N.E. Chafai, M. Kipp, and Stefan Kopp The behavior markup language: recent developments and challenges Proceedings of Intelligent Virtual Agents (IVA) Lecture Notes in Artificial Intelligence vol. 4722 2007 Springer 99 111 (Pubitemid 350271478)
214. Wagner, Petra, Inden, Benjamin, Malisz, Zofia, Wachsmuth, Ipke, in press. Interaction phonology. In: Wachsmuth, Ipke, Jaecks, Petra, de Ruiter, J.P. (Eds.), Towards a New Theory of Communication. Benjamins.
215. Alan Wexelblat An approach to natural gesture in virtual environments ACM Transactions on Computer-Human Interaction (TOCHI) 2 3 1995 179 200
216. Andrew D. Wilson, Aaron F. Bobick, and Justine Cassell Recovering the temporal structure of natural gesture International Conference on Automatic Face and Gesture Recognition 1996 IEEComputer Society Press Los Alamitos, CA
217. Włodarczak, Marcin, Buschmeier, Hendrik, Malisz, Zofia, Kopp, Stefan, Wagner, Petra. 2012. Listener head gestures and verbal feedback expressions in a distraction task. In: Proceedings of the Interdisciplinary Workshop on Feedback Behaviours in Dialogue, Skamania Lodge, Stevenson, WA.
218. Ying Wu, and Thomas S. Huang Vision-based gesture recognition: a review Annelies Braffort, Rachid Gherbi, Sylvie Gibet, Daniel Teil, James Richardson, Gesture-Based Communication in Human-Computer Interaction Lecture Notes in Computer Science Vol. 1739 1999 Springer Berlin 103 115
219. Y. Yassinik, M. Renwick, and S. Shattuck-Hufnagel The timing of speech-accompanying gesture with respect to prosody Proceedings of the International Conference: From Sound to Sense 2004 MIT Cambridge C97 C102 June 10-13
220. C. Yehia, Takaaki Kuratate, and Eric Vatikiotis-Bateson Linking facial animation, head motion and speech acoustics Journal of Phonetics 30 3 2002 555 568
221. Narayan Yoganandan, Frank A Pintar, Jiangyue Zhang, and Jamie L Baisden Physical properties of the human head: mass, center of gravity and moment of inertia Journal of Biomechanics 42 9 2009 1177 1192
222. Loredana Cerrato, and Gunilla Svanfeldt A method for the detection of communicative head nods in expressive speech J. Allwood, B. Dorriots, S. Nicholson, Gothenburg Papers in Theoretical Linguistics 92: Proceedings from The Second Nordic Conference on Multimodal Communication 2005 Göteborg University Sweden 153 165
223. Ugur Murat Erdem, and Stan Sclaroff Automatic detection of relevant head gestures in American Sign Language communication Proceedings of the International Conference on Pattern Recognition 2002 IEEE Computer Society Press 10460 10463
224. Ashish Kapoor, and Rosalind W. Picard A real time nod and shake detector Proceedings of the 2001 Workshop on Perceptive User Interfaces 2001 ACM 1 5
225. Li, Renxiang, Taskiran, Cüneyt M., Danielsen, Mike. 2007. Head pose tracking and gesture detection using block motion vectors on mobile devices. In: Mobility '07: Proceedings of the Fourth International Conference on Mobile Technology, Applications and Systems and the First International Symposium on Computer Human Interaction in Mobile Technology, pp. 572-575.
226. Peng Lu, Xiangsheng Huang, Xinshan Zhu, and Yangsheng Wang Head gesture recognition based on Bayesian network Proceedings of the Second Iberian Conference on Pattern Recognition and Image Analysis Lecture Notes in Computer Science 2005 Springer-Verlag 495 499
227. Laurent Nguyen, Jean-Marc Odobez, and Daniel Gatica-Perez Using self-context for multimodal detection of head nods in face-to-face interactions Proceedings of the International Conference on Multimodal Interaction 2012 ACM 289 292