A review of verbal and non-verbal human-robot interactive communication

Robotics and Autonomous Systems

Volumn 63, Issue P1, 2015, Pages 22-35

  Mavridis, Nikolaos ^a  

^a INSTITUTE OF NANOSCIENCE AND NANOTECHNOLOGY   (Greece)

Author keywords

Human robot communication; Human robot interaction; Non verbal; Survey; Verbal

Indexed keywords

HUMAN COMPUTER INTERACTION; SURVEYING;

FORWARD LOOKING; HUMAN ROBOTS; HUMAN-ROBOT COMMUNICATION; INTERACTIVE COMMUNICATIONS; NON-VERBAL; NON-VERBAL HUMAN; VERBAL;

HUMAN ROBOT INTERACTION;

EID: 84915822231     PISSN: 09218890     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.robot.2014.09.031     Document Type: Article

References (235)

1. L. Ballard Robotics'founding father George C. Devol-serial entrepreneur and inventor Robot-Congers 31 2011 58
2. G.C. Devol, Encoding apparatus, January 24 1984, US Patent 4,427,970.
3. D.-L. Gera Ancient Greek Ideas on Speech, Language, and Civilization 2003 Oxford University Press
4. R. Lattimore, and R. Martin The Iliad of Homer 2011 University of Chicago Press
5. C. Huffman Archytas of Tarentum: Pythagorean, Philosopher and Mathematician King 2005 Cambridge University Press
6. J. Needham Science and Civilisation in China: Volume 2 1959 Cambridge University Press
7. N. Sharkey, The programmable robot of ancient Greece, New Scientist, July 2007, pp. 32-35.
8. M.E. Rosheim Robot Evolution: The Development of Anthrobotics first ed. 1994 John Wiley & Sons, Inc. New York, NY, USA
9. N. Hockstein, C. Gourin, R. Faust, and D. Terris A history of robots: from science fiction to surgical robotics J. Robot. Surg. 1 2 2007 113 118
10. D.H. Klatt Review of text-to-speech conversion for English J. Acoust. Soc. Am. 82 3 1987 737 793
11. G. Antoniol, R. Cattoni, M. Cettolo, M. Federico, Robust speech understanding for robot telecontrol, in: Proceedings of the 6th International Conference on Advanced Robotics, 1993, pp. 205-209.
12. W. Burgard, A.B. Cremers, D. Fox, D. Hähnel, G. Lakemeyer, D. Schulz, W. Steiner, S. Thrun, The interactive museum tour-guide robot, in: Proc. of the Fifteenth National Conference on Artificial Intelligence, AAAI-98, 1998.
13. L. Versweyveld, Voice-controlled surgical robot ready to assist in minimally invasive heart surgery, Virtual Medicine World Monthly, March 1998.
14. I. Horswill Polly: a vision-based artificial agent Proceedings of the Eleventh National Conference on Artificial Intelligence AAAI-93 1993 Press 824 829
15. I. Horswill The design of the Polly system, Tech. Rep. 1996 The Institute for the Learning Sciences, Northwestern University September
16. M. Torrance Natural communication with mobile robots (Master's thesis) 1994 MIT Department of Electrical Engineering and Computer Science January
17. G. Antoniol, B. Caprile, A. Cimatti, R. Fiutem, Experiencing real-life interactions with the experimental platform of MAIA, in: Proceedings of the 1st European Workshop on Human Comfort and Security, 1994.
18. I. Androutsopoulos A principled framework for constructing natural language interfaces to temporal databases (Ph.D. dissertation) 1996 Department of Artificial Intelligence, University of Edinburgh
19. H. Asoh, T. Matsui, J. Fry, F. Asano, and S. Hayamizu A spoken dialog system for a mobile office robot Proceedings of the European Conference on Speech Communication and Technology, EUROSPEECH 1999 ISCA
20. J. Fry, H. Asoh, T. Matsui, Natural dialogue with the Jijo-2 office robot, in: 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems, Proceedings, vol. 2, 1998, pp. 1278-1283.
21. T. Matsui, H. Asoh, J. Fry, Y. Motomura, F. Asano, T. Kurita, I. Hara, and N. Otsu Integrated natural spoken dialogue system of Jijo-2 mobile robot for office services Proceedings of the Sixteenth National Conference on Artificial Intelligence and the Eleventh Innovative Applications of Artificial Intelligence Conference Innovative Applications of Artificial Intelligence ser. AAAI'99/IAAI'99 1999 American Association for Artificial Intelligence Menlo Park, CA, USA 621 627
22. C. Crangle, and P. Suppes Language and Learning for Robots Ser. CSLI Lecture Notes 1994 Center for the Study of Language and Information [Online]. Available: http://books.google.gr/books?id=MlMQ11Pqz10C
23. J.R. Searle Speech Acts: An Essay in the Philosophy of Language 1969 Cambridge University Press Cambridge
24. K. Vogeley, and G. Bente "Artificial humans": psychology and neuroscience perspectives on embodiment and nonverbal communication Neural Netw. 23 8 2010 1077 1090
25. S. Harnad The symbol grounding problem Physica D 42 1 1990 335 346
26. G. Schreiber, A. Stemmer, R. Bischoff, The fast research interface for the KUKA lightweight robot, in: IEEE Conference on Robotics and Automation, ICRA, 2010.
27. S. Wrede, C. Emmerich, R. Grünberg, A. Nordmann, A. Swadzba, and J. Steil A user study on kinesthetic teaching of redundant robots in task and configuration space J. Hum.-Robot Interact. 2 1 2013 56 81
28. B.D. Argall, S. Chernova, M. Veloso, and B. Browning A survey of robot learning from demonstration Robot. Auton. Syst. 57 5 2009 469 483
29. C.L. Nehaniv, and K. Dautenhahn Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions 2007 Cambridge University Press
30. N. Mavridis, D. Roy, Grounded situation models for robots: where words and percepts meet, in: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006, pp. 4690-4697.
31. T. van der Zant, T. Wisspeintner, RoboCup@ home: creating and benchmarking tomorrows service robot applications, Robotic Soccer, 2007, pp. 521-528.
32. M.E. Foster, T. By, M. Rickert, and A. Knoll Human-robot dialogue for joint construction tasks Proceedings of the 8th International Conference on Multimodal Interfaces ser. ICMI'06 2006 ACM New York, NY, USA 68 71
33. M. Giuliani, and A. Knoll Evaluating supportive and instructive robot roles in human-robot interaction Social Robotics 2011 Springer 193 203
34. K. Wada, and T. Shibata Living with seal robots - its sociopsychological and physiological influences on the elderly at a care house IEEE Trans. Robot. 23 5 2007 972 980
35. K. Kamei, K. Shinozawa, T. Ikeda, A. Utsumi, T. Miyashita, and N. Hagita Recommendation from robots in a real-world retail shop International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction 2010 ACM 19
36. V. Chidambaram, Y.-H. Chiang, and B. Mutlu Designing persuasive robots: how robots might persuade people using vocal and nonverbal cues Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'12 2012 ACM New York, NY, USA 293 300 [Online]. Available: http://doi.acm.org/10.1145/2157689.2157798
37. M. Makatchev, I. Fanaswala, A. Abdulsalam, B. Browning, W. Ghazzawi, M. Sakr, R. Simmons, Dialogue patterns of an arabic robot receptionist, in: 2010 5th ACM/IEEE International Conference on Human-Robot Interaction, HRI, 2010, pp. 167-168.
38. T. Kanda, M. Shiomi, Z. Miyashita, H. Ishiguro, and N. Hagita An affective guide robot in a shopping mall Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction ser. HRI'09 2009 ACM New York, NY, USA 173 180 [Online]. Available: http://doi.acm.org/10.1145/1514095.1514127
39. M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita Interactive humanoid robots for a science museum Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction ser. HRI'06 2006 ACM New York, NY, USA 305 312 [Online]. Available: http://doi.acm.org/10.1145/1121241.1121293
40. A. Yamazaki, K. Yamazaki, T. Ohyama, Y. Kobayashi, and Y. Kuno A techno-sociological solution for designing a museum guide robot: regarding choosing an appropriate visitor Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'12 2012 ACM New York, NY, USA 309 316 [Online]. Available: http://doi.acm.org/10.1145/2157689.2157800
41. V. Evers, N. Menezes, L. Merino, D. Gavrila, F. Nabais, M. Pantic, P. Alvito, and D. Karreman The development and real-world deployment of frog, the fun robotic outdoor guide Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'14 2014 ACM New York, NY, USA 100 [Online]. Available: http://doi.acm.org/10.1145/2559636.2559649
42. D.E. Karreman, G.U. Sepúlveda Bradford, B.E. van Dijk, M. Lohse, and V. Evers What happens when a robot favors someone? How a tour guide robot uses gaze behavior to address multiple persons while storytelling about art Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'13 2013 IEEE Press Piscataway, NJ, USA 157 158 [Online]. Available: http://dl.acm.org/citation.cfm?id=2447556.2447615
43. V. Evers, R. de Vries, and P. Alvito Designing interruptive behaviors of a public environmental monitoring robot Proceedings of the 6th International Conference on Human-Robot Interaction ser. HRI'11 2011 ACM New York, NY, USA 131 132 [Online]. Available: http://doi.acm.org/10.1145/1957656.1957696
44. S. Tellex, and D. Roy Spatial routines for a simulated speech-controlled vehicle Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction 2006 ACM 156 163
45. Y. Kobayashi, Y. Kinpara, E. Takano, Y. Kuno, K. Yamazaki, and A. Yamazaki A wheelchair which can automatically move alongside a caregiver Proceedings of the 6th International Conference on Human-Robot Interaction ser. HRI'11 2011 ACM New York, NY, USA 407 408 [Online]. Available: http://doi.acm.org/10.1145/1957656.1957805
46. K. Dautenhahn, M. Walters, S. Woods, K.L. Koay, C.L. Nehaniv, A. Sisbot, R. Alami, and T. Siméon How may i serve you? A robot companion approaching a seated person in a helping context Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction 2006 ACM 172 179
47. Y. Kobayashi, M. Gyoda, T. Tabata, Y. Kuno, K. Yamazaki, M. Shibuya, and Y. Seki Assisted-care robot dealing with multiple requests in multi-party settings Proceedings of the 6th International Conference on Human-Robot Interaction ser. HRI'11 2011 ACM New York, NY, USA 167 168 [Online]. Available: http://doi.acm.org/10.1145/1957656.1957714
48. N. Mavridis, and D. Hanson The IbnSina Center: an augmented reality theater with intelligent robotic and virtual characters The 18th IEEE International Symposium on Robot and Human Interactive Communication RO-MAN 2009 2009 IEEE 681 686
49. J. Zlotowski, T. Bleeker, C. Bartneck, and R. Reynolds I sing the body electric: an experimental theatre play with robots Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'13 2013 IEEE Press Piscataway, NJ, USA 427 428 [Online]. Available: http://dl.acm.org/citation.cfm?id=2447556.2447702
50. K. Petersen, J. Solis, and A. Takanishi Musical-based interaction system for the waseda flutist robot Auton. Robots 28 4 2010 471 488
51. K. Kosuge, T. Hayashi, Y. Hirata, and R. Tobiyama Dance partner robot-MS Dancer 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2003, Proceedings, vol. 4 2003 IEEE 3459 3464
52. V.A. Kulyukin On natural language dialogue with assistive robots Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction 2006 ACM 164 171
53. N. Mavridis Grounded situation models for situated conversational assistants (Ph.D. dissertation) 2007 Massachusetts Institute of Technology
54. T. Winograd A procedural model of language understanding Readings in Natural Language Processing 1986 Morgan Kaufmann Publishers Inc. 249 266
55. T. Winograd Understanding natural language Cogn. Psychol. 3 1 1972 1 191
56. J. Dzifcak, M. Scheutz, C. Baral, P. Schermerhorn, What to do and how to do it: translating natural language directives into temporal and dynamic logic representation for goal management and action execution, in: Proceedings of the 2009 IEEE International Conference on Robotics and Automation, ICRA'09, Kobe, Japan, May 2009.
57. J. Austin, How to Do Things with Words. Oxford, 1962.
58. J. Searle A taxonomy of illocutionary acts K. Gunderson, Language, Mind and Knowledge 1975 University of Minnesota Press 344 369
59. J.F. Allen, D.K. Byron, M. Dzikovska, G. Ferguson, L. Galescu, and A. Stent Towards conversational human-computer interaction AI Mag. 22 2001 27 37
60. P.N. Johnson-Laird Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness, Vol. 6 1983 Harvard University Press
61. R.A. Zwaan, and G.A. Radvansky Situation models in language comprehension and memory Psychol. Bull. 123 2 1998 162
62. H.P. Grice, Logic and conversation, 1975, pp. 41-58.
63. S. Wilske, and G.-J. Kruijff Service robots dealing with indirect speech acts 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems 2006 IEEE 4698 4703
64. F. Krsmanovic, C. Spencer, D. Jurafsky, A.Y. Ng, Have we met? MDP based speaker ID for robot dialogue. in: INTERSPEECH, 2006.
65. C.T. Ishi, H. Ishiguro, N. Hagita, Analysis of prosodic and linguistic cues of phrase finals for turn-taking and dialog acts. in: INTERSPEECH, 2006.
66. N. Mavridis, M. Petychakis, A. Tsamakos, P. Toulis, S. Emami, W. Kazmi, C. Datta, C. BenAbdelkader, and A. Tanoto FaceBots: steps towards enhanced long-term human-robot interaction by utilizing and publishing online social information Paladyn 1 3 2010 169 178
67. N. Mavridis, C. Datta, S. Emami, A. Tanoto, C. BenAbdelkader, and T. Rabie FaceBots: robots utilizing and publishing social information in facebook 2009 4th ACM/IEEE International Conference on Human-Robot Interaction, HRI 2009 IEEE 273 274
68. B. Wrede, S. Buschkaemper, C. Muhl, K.J. Rohlfing, Analyses of feedback in HRI, in: How People Talk to Computers, Robots, and Other Artificial Communication Partners, 2006, p. 38.
69. R. Stiefelhagen, H.K. Ekenel, C. Fugen, P. Gieselmann, H. Holzapfel, F. Kraft, K. Nickel, M. Voit, and A. Waibel Enabling multimodal human-robot interaction for the Karlsruhe humanoid robot IEEE Trans. Robot. 23 5 2007 840 851
70. D. Ertl, A. Green, H. Hüttenrauch, F. Lerasle, Improving human-robot communication with mixed-initiative and context-awareness co-located with Ro-Man 2009.
71. M. Ralph, and M.A. Moussa Toward a natural language interface for transferring grasping skills to robots IEEE Trans. Robot. 24 2 2008 468 475
72. J. Weizenbaum Eliza - a computer program for the study of natural language communication between man and machine Commun. ACM 9 1 1966 36 45
73. M.L. Mauldin, Chatterbots, tinymuds, and the Turing test: entering the Loebner prize competition, in: AAAI, vol. 94, 1994, pp. 16-21.
74. D. Roy, A computational model of word learning from multimodal sensory input, in: Proceedings of the International Conference of Cognitive Modeling, ICCM2000, Groningen, Netherlands. Citeseer, 2000.
75. R. Baillargeon, E.S. Spelke, and S. Wasserman Object permanence in five-month-old infants Cognition 20 3 1985 191 208
76. D. Roy, K.-Y. Hsiao, and N. Mavridis Mental imagery for a conversational robot IEEE Trans. Syst. Man Cybern. B 34 3 2004 1374 1383
77. E. Tulving Elements of Episodic Memory 1983 Clarendon Press Oxford
78. N. Mavridis, M. Petychakis, Human-like memory systems for interactive robots: desiderata and two case studies utilizing groundedsituation models and online social networking.
79. D. Gentner, and K.D. Forbus Computational models of analogy Wiley Interdiscip. Rev.: Cogn. Sci. 2 3 2011 266 276
80. C.S. Pierce Logic as semiotic: the theory of signs The Philosophical Writings of Pierce 1955 98 119
81. C.S. Peirce Collected Papers of Charles Sanders Peirce, Vol. 3 1974 Harvard University Press
82. T. Spexard, S. Li, B. Wrede, J. Fritsch, G. Sagerer, O. Booij, Z. Zivkovic, B. Terwijn, and B. Krose BIRON, where are you? Enabling a robot to learn new places in a real home environment by integrating spoken dialog and visual localization 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems 2006 IEEE 934 940
83. L. Steels Evolving grounded communication for robots Trends Cogn. Sci. 7 7 2003 308 312
84. S.D. Larson Intrinsic Representation: Bootstrapping Symbols from Experience 2004 Springer
85. P.J. Gorniak The affordance-based concept (Ph.D. dissertation) 2005 Massachusetts Institute of Technology
86. C. Yu, L.B. Smith, A.F. Pereira, Grounding word learning in multimodal sensorimotor interaction, in: Proceedings of the 30th Annual Conference of the Cognitive Science Society, 2008, pp. 1017-1022.
87. T. Regier, and L.A. Carlson Grounding spatial language in perception: an empirical and computational investigation J. Exp. Psychol. [Gen.] 130 2 2001 273
88. D.K. Roy Learning visually grounded words and syntax for a scene description task Comput. Speech Lang. 16 3 2002 353 385
89. K.R. Coventry, and S.C. Garrod Saying, Seeing and Acting: The Psychological Semantics of Spatial Prepositions 2004 Psychology Press
90. J.M. Siskind, Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic, 2011. ArXiv Preprint arXiv:1106.0256.
91. J. Zlatev Spatial semantics Handbook of Cognitive Linguistics 2007 318 350
92. M. Skubic, D. Perzanowski, S. Blisard, A. Schultz, W. Adams, M. Bugajska, and D. Brock Spatial language for human-robot dialogs IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 34 2 2004 154 167
93. H. Zender, O. Martínez Mozos, P. Jensfelt, G.-J. Kruijff, and W. Burgard Conceptual spatial representations for indoor mobile robots Robot. Auton. Syst. 56 6 2008 493 502
94. S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A.G. Banerjee, S.J. Teller, N. Roy, Understanding natural language commands for robotic navigation and mobile manipulation. in: AAAI, 2011.
95. S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A.G. Banerjee, S. Teller, and N. Roy Approaching the symbol grounding problem with probabilistic graphical models AI Mag. 32 4 2011 64 76
96. S. Tellex, P. Thaker, J. Joseph, and N. Roy Learning perceptually grounded word meanings from unaligned parallel data Mach. Learn. 2013 1 17
97. K. Gold, B. Scassellati, Grounded pronoun learning and pronoun reversal, in: Proceedings of the 5th International Conference on Development and Learning, 2006.
98. L. Steels The symbol grounding problem has been solved. so what's next Symbols and Embodiment: Debates on Meaning and Cognition 2008 223 244
99. L. Steels Semiotic dynamics for embodied agents IEEE Intell. Syst. 21 3 2006 32 38
100. C. Hudelot, N. Maillot, and M. Thonnat Symbol grounding for semantic image interpretation: from image data to semantics Tenth IEEE International Conference on Computer Vision Workshops ICCVW'05 2005 IEEE
101. A.M. Cregan Symbol grounding for the semantic Web The Semantic Web: Research and Applications 2007 Springer 429 442
102. C. Wallraven, M. Schultze, B. Mohler, A. Vatakis, and K. Pastra The poeticon enacted scenario corpus - a tool for human and computational experiments on action understanding 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops FG 2011 2011 IEEE 484 491
103. K. Pastra, C. Wallraven, M. Schultze, A. Vataki, K. Kaulard, The poeticon corpus: capturing language use and sensorimotor experience in everyday interaction. in: LREC, Citeseer, 2010.
104. P. Gärdenfors Conceptual Spaces: The Geometry of Throught 2004 MIT press
105. J. Orkin, and D. Roy The restaurant game: learning social behavior and language from thousands of players online J. Game Dev. 3 1 2007 39 60
106. S. Chernova, J. Orkin, C. Breazeal, Crowdsourcing HRI through online multiplayer games, in: Proc. Dialog with Robots: AAAI Fall Symposium, 2010.
107. N. DePalma, S. Chernova, C. Breazeal, Leveraging online virtual agents to crowdsource human-robot interaction, in: Proceedings of CHI Workshop on Crowdsourcing and Human Computation, 2011.
108. L. Von Ahn, R. Liu, and M. Blum Peekaboom: a game for locating objects in images Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2006 ACM 55 64
109. E.R. Hilgard The trilogy of mind: cognition, affection, and conation J. Hist. Behav. Sci. 16 2 1980 107 117
110. R.W. Picard Affective computing: challenges Int. J. Hum.-Comput. Stud. 59 1 2003 55 64
111. R. Picard, S. Papert, W. Bender, B. Blumberg, C. Breazeal, D. Cavallo, T. Machover, M. Resnick, D. Roy, and C. Strohecker Affective learning - a manifesto BT Technol. J. 22 4 2004 253 269
112. G. Haddock, G.R. Maio, K. Arnold, and T. Huskinson Should persuasion be affective or cognitive? The moderating effects of need for affect and need for cognition Pers. Soc. Psychol. Bull. 34 6 2008 769 778
113. C. Breazeal Emotion and sociable humanoid robots Int. J. Hum.-Comput. Stud. 59 1 2003 119 155
114. J. Cassell Embodied conversational interface agents Commun. ACM 43 4 2000 70 78
115. W.L. Johnson, J.W. Rickel, and J.C. Lester Animated pedagogical agents: face-to-face interaction in interactive learning environments Int. J. Artif. Intell. Educ. 11 1 2000 47 78
116. F.d. Rosis, C. Pelachaud, I. Poggi, V. Carofiglio, and B.D. Carolis From Greta's mind to her face: modelling the dynamics of affective states in a conversational embodied agent Int. J. Hum.-Comput. Stud. 59 1 2003 81 118
117. C. Breazeal, J. Velásquez, Toward teaching a robot "infant" using emotive communication acts, in: Proceedings of the 1998 Simulated Adaptive Behavior Workshop on Socially Situated Intelligence, 1998, pp. 25-40.
118. A. Batliner, C. Hacker, S. Steidl, E. Nöth, S. D'Arcy, M.J. Russell, M. Wong, "You stupid tin box"-children interacting with the AIBO robot: a cross-linguistic emotional speech corpus. in: LREC, 2004.
119. K. Komatani, R. Ito, T. Kawahara, and H.G. Okuno Recognition of emotional states in spoken dialogue with a robot Innovations in Applied Artificial Intelligence 2004 Springer 413 423
120. B.-C. Bae, A. Brunete, U. Malik, E. Dimara, J. Jermsurawong, N. Mavridis, Towards an empathizing and adaptive storyteller system, in: Eighth Artificial Intelligence and Interactive Digital Entertainment Conference, 2012.
121. T. Ruf, A. Ernst, and C. Küblbeck Face detection with the sophisticated high-speed object recognition engine (SHORE) Microelectronic Systems 2011 Springer 243 252
122. R. El Kaliouby, and P. Robinson Real-time inference of complex mental states from facial expressions and head gestures Real-Time Vision for Human-Computer Interaction 2005 Springer 181 200
123. C. Shan, S. Gong, and P.W. McOwan Facial expression recognition based on local binary patterns: a comprehensive study Image Vis. Comput. 27 6 2009 803 816
124. M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan Fully automatic facial action recognition in spontaneous behavior 7th International Conference on Automatic Face and Gesture Recognition FGR 2006 2006 IEEE 223 230
125. T. Wu, N.J. Butko, P. Ruvulo, M.S. Bartlett, and J.R. Movellan Learning to make facial expressions IEEE 8th International Conference on Development and Learning ICDL 2009 2009 IEEE 1 6
126. M.-J. Han, C.-H. Lin, and K.-T. Song Robotic emotional expression generation based on mood transition and personality model IEEE Trans. Cybern. 43 4 2013 1290 1303
127. T. Baltrušaitis, L.D. Riek, and P. Robinson Synthesizing expressions using facial feature point tracking: how emotion is conveyed Proceedings of the 3rd International Workshop on Affective Interaction in Natural Environments 2010 ACM 27 32
128. G. Littlewort, M.S. Bartlett, I. Fasel, J. Susskind, and J. Movellan Dynamics of facial expression extracted automatically from video Image Vis. Comput. 24 6 2006 615 625
129. M. Schröder Expressive speech synthesis: past, present, and possible futures Affective Information Processing 2009 Springer 111 126
130. D. Ververidis, and C. Kotropoulos Emotional speech recognition: resources, features, and methods Speech Commun. 48 9 2006 1162 1181
131. S. Roehling, B. MacDonald, C. Watson, Towards expressive speech synthesis in English on a robotic platform, in: Proceedings of the Australasian International Conference on Speech Science and Technology, 2006, pp. 130-135.
132. A. Chella, R.E. Barone, G. Pilato, R. Sorbello, An emotional storyteller robot. in: AAAI Spring Symposium: Emotion, Personality, and Social Behavior, 2008, pp. 17-22.
133. B. Pang, and L. Lee Opinion mining and sentiment analysis Found. Trends Inf. Retr. 2 1-2 2008 1 135
134. T. Wilson, J. Wiebe, and P. Hoffmann Recognizing contextual polarity: an exploration of features for phrase-level sentiment analysis Comput. Linguist. 35 3 2009 399 433
135. M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede Lexicon-based methods for sentiment analysis Comput. Linguist. 37 2 2011 267 307
136. R.W. Picard Measuring affect in the wild Affective Computing and Intelligent Interaction 2011 Springer 3
137. S.H. Fairclough Fundamentals of physiological computing Interact. Comput. 21 1 2009 133 145
138. C.L. Bethel, and R.R. Murphy Affective expression in appearance constrained robots Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction ser. HRI'06 2006 ACM New York, NY, USA 327 328 [Online]. Available: http://doi.acm.org/10.1145/1121241.1121299
139. C.L. Bethel, and R.R. Murphy Non-facial/non-verbal methods of affective expression as applied to robot-assisted victim assessment Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'07 2007 ACM New York, NY, USA 287 294 [Online]. Available: http://doi.acm.org/10.1145/1228716.1228755
140. D. Arroyo, C. Lucho, S.J. Roncal, and F. Cuellar Daedalus: a sUAV for human-robot interaction Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'14 2014 ACM New York, NY, USA 116 117 [Online]. Available: http://doi.acm.org/10.1145/2559636.2563709
141. R. Read, and T. Belpaeme How to use non-linguistic utterances to convey emotion in child-robot interaction Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'12 2012 ACM New York, NY, USA 219 220 [Online]. Available: http://doi.acm.org/10.1145/2157689.2157764
142. R. Read, and T. Belpaeme People interpret robotic non-linguistic utterances categorically Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'13 2013 IEEE Press Piscataway, NJ, USA 209 210 [Online]. Available: http://dl.acm.org/citation.cfm?id=2447556.2447641
143. H.A. Elfenbein, and N. Ambady On the universality and cultural specificity of emotion recognition: a meta-analysis Psychol. Bull. 128 2 2002 203
144. D. McNeill Hand and Mind: What Gestures Reveal about Thought 1992 University of Chicago Press
145. P. Weil About face, computergraphic synthesis and manipulation of facial imagery (Ph.D. dissertation) 1982 Massachusetts Institute of Technology
146. J. Lewis, P. Purcell, Soft machine: a personable interface, in: Proc. of Graphics Interface, vol. 84. Citeseer, 1984, pp. 223-226.
147. J.P. Lewis, and F.I. Parke Automated lip-synch and speech synthesis for character animation ACM SIGCHI Bull. 17 SI 1987 143 147
148. C. Bregler, and Y. Konig "Eigenlips" for robust speech recognition 1994 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP-94, vol. 2 1994 IEEE II-669
149. H. McGurk, and J. MacDonald Hearing lips and seeing voices Nature 1976 746 748
150. G.A. Calvert, C. Spence, and B.E. Stein The Handbook of Multisensory Processes 2004 MIT Press
151. M. Tsakiris, and P. Haggard The rubber hand illusion revisited: visuotactile integration and self-attribution J. Exp. Psychol. [Hum. Percept.] 31 1 2005 80
152. N. Segato, A. Krogsager, D.G. Jensen, and M. Rehm The role of physical embodiment of humanoid robot interaction: focusing on backchannel head nods in danish first meeting encounters HCI International 2014-Posters' Extended Abstracts 2014 Springer 583 587
153. K.R. Thorisson Communicative humanoids: a computational model of psychosocial dialogue skills (Ph.D. dissertation) 1996 Massachusetts Institute of Technology
154. G. Lidoris, F. Rohrmuller, D. Wollherr, and M. Buss The autonomous city explorer (ACE) project - mobile robot navigation in highly populated urban environments IEEE International Conference on Robotics and Automation ICRA'09 2009 IEEE 1416 1422
155. W.-M. Roth Gestures: their role in teaching and learning Rev. Educ. Res. 71 3 2001 365 392
156. J. Cassell, T. Bickmore, M. Billinghurst, L. Campbell, K. Chang, H. Vilhjálmsson, and H. Yan Embodiment in conversational interfaces: Rea Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 1999 ACM 520 527
157. J. Cassell, H.H. Vilhjálmsson, and T. Bickmore BEAT: the behavior expression animation toolkit Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques ser. SIGGRAPH'01 2001 ACM New York, NY, USA 477 486
158. N. Rossini Patterns of synchronization of non-verbal cues and speech in ECAs: towards a more "natural" conversational agent Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces. Theoretical and Practical Issues 2011 Springer 96 103
159. S.R. Fussell, R.E. Kraut, and J. Siegel Coordination of communication: effects of shared visual context on collaborative work Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work 2000 ACM 21 30
160. S.E. Brennan, X. Chen, C.A. Dickinson, M.B. Neider, and G.J. Zelinsky Coordinating cognition: the costs and benefits of shared gaze during collaborative search Cognition 106 3 2008 1465 1477
161. C. Breazeal, C.D. Kidd, A.L. Thomaz, G. Hoffman, and M. Berlin Effects of nonverbal communication on efficiency and robustness in human-robot teamwork 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems IROS 2005 2005 IEEE 708 713
162. D. Das, M.M. Hoque, Y. Kobayashi, and Y. Kuno Attention control system considering the target person's attention level Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'13 2013 IEEE Press Piscataway, NJ, USA 111 112 [Online]. Available: http://dl.acm.org/citation.cfm?id=2447556.2447592
163. M.M. Hoque, T. Onuki, D. Das, Y. Kobayashi, and Y. Kuno Attracting and controlling human attention through robot's behaviors suited to the situation Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'12 2012 ACM New York, NY, USA 149 150 [Online]. Available: http://doi.acm.org/10.1145/2157689.2157729
164. T. Onuki, T. Ishinoda, Y. Kobayashi, and Y. Kuno Design of robot eyes suitable for gaze communication Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'13 2013 IEEE Press Piscataway, NJ, USA 203 204 [Online]. Available: http://dl.acm.org/citation.cfm?id=2447556.2447638
165. A. Moon, D.M. Troniak, B. Gleeson, M.K. Pan, M. Zheng, B.A. Blumer, K. MacLean, and E.A. Croft Meet me where i'm gazing: how shared attention gaze affects human-robot handover timing Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'14 2014 ACM New York, NY, USA 334 341 [Online]. Available: http://doi.acm.org/10.1145/2559636.2559656
166. H. Admoni, A. Dragan, S.S. Srinivasa, and B. Scassellati Deliberate delays during robot-to-human handovers improve compliance with gaze communication Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction ser. HRI'14 2014 ACM New York, NY, USA 49 56 [Online]. Available: http://doi.acm.org/10.1145/2559636.2559682
167. J.E. Hanna, and S.E. Brennan Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation J. Memory Lang. 57 4 2007 596 615
168. J.E. Hanna, and M.K. Tanenhaus Pragmatic effects on reference resolution in a collaborative task: evidence from eye movements Cogn. Sci. 28 1 2004 105 115
169. B. Mutlu, T. Shiwa, T. Kanda, H. Ishiguro, and N. Hagita Footing in human-robot conversations: how robots might shape participant roles using gaze cues Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction 2009 ACM 61 68
170. L.B. Adamson, and R. Bakeman The development of shared attention during infancy Ann. Child Dev. 8 1991 1 41
171. B. Scassellati, Mechanisms of shared attention for a humanoid robot, in: Embodied Cognition and Action: Papers from the 1996 AAAI Fall Symposium, vol. 4, no. 9, 1996, p. 21.
172. B. Scassellati Imitation and mechanisms of joint attention: a developmental structure for building social skills on a humanoid robot Computation for Metaphors, Analogy, and Agents 1999 Springer 176 195
173. G.O. Deák, I. Fasel, J. Movellan, The emergence of shared attention: using robots to test developmental theories, in: Proceedings 1st International Workshop on Epigenetic Robotics: Lund University Cognitive Studies, vol. 85, 2001, pp. 95-104.
174. I. Fasel, G.O. Deák, J. Triesch, and J. Movellan Combining embodied models and empirical research for understanding the development of shared attention The 2nd International Conference on Development and Learning, Proceedings 2002 IEEE 21 27
175. M.W. Hoffman, D.B. Grimes, A.P. Shon, and R.P. Rao A probabilistic model of gaze imitation and shared attention Neural Netw. 19 3 2006 299 310
176. C. Peters, S. Asteriadis, K. Karpouzis, E. de Sevin, Towards a real-time gaze-based shared attention for a virtual agent, in: International Conference on Multimodal Interfaces, 2008.
177. C. Peters, S. Asteriadis, and K. Karpouzis Investigating shared attention with a virtual agent using a gaze-based interface J. Multimodal User Interfaces 3 1-2 2010 119 130
178. D. Premack, and G. Woodruff Does the chimpanzee have a theory of mind? Behav. Brain Sci. 1 04 1978 515 526
179. H.M. Wellman, The child's theory of mind, 2011.
180. B.M. Scassellati Foundations for a theory of mind for a humanoid robot (Ph.D. dissertation) 2001 Massachusetts Institute of Technology
181. B. Scassellati Theory of mind for a humanoid robot Auton. Robots 12 1 2002 13 24
182. L.F. Marin-Urias, E.A. Sisbot, A.K. Pandey, R. Tadakuma, and R. Alami Towards shared attention through geometric reasoning for human robot interaction 9th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2009 2009 IEEE 331 336
183. K. Santner, G. Fritz, L. Paletta, and H. Mayer Visual recovery of saliency maps from human attention in 3D environments 2013 IEEE International Conference on Robotics and Automation, ICRA 2013 IEEE 4297 4303
184. S. Baron-Cohen Mindblindness: An Essay on Autism and Theory of Mind 1997 MIT Press
185. S.E. Baron-Cohen, H.E. Tager-Flusberg, and D.J. Cohen Understanding Other Minds: Perspectives from Developmental Cognitive Neuroscience 2000 Oxford University Press
186. B. Robins, K. Dautenhahn, R. Te Boekhorst, and A. Billard Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? Univers. Access Inform. Soc. 4 2 2005 105 120
187. G. Bird, J. Leighton, C. Press, and C. Heyes Intact automatic imitation of human and robot actions in autism spectrum disorders Proc. R. Soc. Biol. Sci. Ser. B 274 1628 2007 3027 3031
188. B. Robins, P. Dickerson, P. Stribling, and K. Dautenhahn Robot-mediated joint attention in children with autism: a case study in robot-human interaction Interact. Stud. 5 2 2004 161 198
189. B. Robins, K. Dautenhahn, and P. Dickerson From isolation to communication: a case study evaluation of robot assisted play for children with autism with a minimally expressive humanoid robot Second International Conferences on Advances in Computer-Human Interactions ACHI'09 2009 IEEE 205 211
190. H. Admoni, and B. Scassellati Demo: toward a data-driven generative behavior model for human-robot interaction Proceedings of the 2014 Workshop on Mobile Augmented Reality and Robotic Technology-Based Systems 2014 ACM 19 20
191. S. Russell, and P. Norvig Artificial Intelligence: A Modern Approach 2009 Prentice Hall PA
192. D. Jurafsky, and H. James Speech and Language Processing an Introduction to Natural Language Processing, Computational Linguistics, and Speech 2000
193. P.R. Cohen, and C.R. Perrault Elements of a plan-based theory of speech acts Cogn. Sci. 3 3 1979 177 212
194. J.J. Kuffner Jr., and S.M. LaValle RRT-connect: an efficient approach to single-query path planning IEEE International Conference on Robotics and Automation, Proceedings. ICRA'00, vol. 2 2000 IEEE 995 1001
195. S. Garrido, L. Moreno, M. Abderrahim, and F. Martin Path planning for mobile robot navigation using Voronoi diagram and fast marching 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems 2006 IEEE 2376 2381
196. L.P. Kaelbling, M.L. Littman, and A.R. Cassandra Planning and acting in partially observable stochastic domains Artif. Intell. 101 1 1998 99 134
197. S. Young, Using POMDPs for dialog management. in: SLT, 2006, pp. 8-13.
198. F. Bacchus Lp, a logic for representing and reasoning with statistical knowledge Comput. Intelligence 6 4 1990 209 231
199. N. Mavridis, and H. Dong To ask or to sense? Planning to integrate speech and sensorimotor acts 2012 4th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops, ICUMT 2012 IEEE 227 233
200. V. Klingspor, J. Demiris, and M. Kaiser Human-robot communication and machine learning Appl. Artif. Intell. 11 7 1997 719 746
201. C. Breazeal, Imitation as social exchange between humans and robots, in: Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, 1999, pp. 96-104.
202. C.L. Isbell Jr., M. Kearns, S. Singh, C.R. Shelton, P. Stone, and D. Kormann Cobot in LambdaMOO: an adaptive social statistics agent Auton. Agents Multi-Agent Syst. 13 3 2006 327 354
203. E. Guizzo Robots with their heads in the clouds IEEE Spectr. 48 3 2011 16 18
204. N. Mavridis, T. Bourlai, and D. Ognibene The human-robot cloud: situated collective intelligence on demand 2012 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems, CYBER 2012 IEEE 360 365
205. N. Mitsunaga, Z. Miyashita, K. Shinozawa, T. Miyashita, H. Ishiguro, and N. Hagita What makes people accept a robot in a social environment-discussion from six-week study in an office IEEE/RSJ International Conference on Intelligent Robots and Systems IROS 2008 2008 IEEE 3336 3343
206. N. Mavridis, A. AlDhaheri, L. AlDhaheri, M. Khanii, and N. AlDarmaki Transforming ibnsina into an advanced multilingual interactive android robot 2011 IEEE GCC Conference and Exhibition, GCC 2011 IEEE 120 123
207. M. Waibel, M. Beetz, J. Civera, R. D'Andrea, J. Elfring, D. Galvez-Lopez, K. Haussermann, R. Janssen, J. Montiel, and A. Perzylo Roboearth IEEE Robot. Autom. Mag. 18 2 2011 69 82
208. D. Hunziker, M. Gajamohan, M. Waibel, R. D'Andrea, Rapyuta: the RoboEarth cloud engine, in: Proc. IEEE Int. Conf. on Robotics and Automation, ICRA, Karlsruhe, Germany, 2013.
209. H. Sacks, E.A. Schegloff, and G. Jefferson A simplest systematics for the organization of turn-taking for conversation Language 1974 696 735
210. E.A. Schegloff Overlapping talk and the organization of turn-taking for conversation Lang. Soc. 29 1 2000 1 63
211. Y. Matsusaka, S. Fujie, T. Kobayashi, Modeling of conversational strategy for the robot participating in the group conversation, in: INTERSPEECH, vol. 1, 2001, pp. 2173-2176.
212. C. Chao, A.L. Thomaz, Turn taking for human-robot interaction, in: AAAI Fall Symposium on Dialog with Robots, 2010, pp. 132-134.
213. C.L. Sidner, C. Lee, C.D. Kidd, N. Lesh, and C. Rich Explorations in engagement for humans and robots Artif. Intell. 166 1 2005 140 164
214. J.-M. Valin, F. Michaud, J. Rouat, and D. Létourneau Robust sound source localization using a microphone array on a mobile robot 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2003, Proceedings, vol. 2 2003 IEEE 1228 1233
215. K. Nakadai, D. Matsuura, H.G. Okuno, and H. Kitano Applying scattering theory to robot audition system: robust sound source localization and extraction 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2003, Proceedings, vol. 2 2003 IEEE 1147 1152
216. J.-M. Valin, F. Michaud, B. Hadjou, and J. Rouat Localization of simultaneous moving sound sources for mobile robot using a frequency-domain steered beamformer approach 2004 IEEE International Conference on Robotics and Automation, Proceedings. ICRA'04, vol. 1 2004 IEEE 1033 1038
217. J.-M. Valin, F. Michaud, and J. Rouat Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering Robot. Auton. Syst. 55 3 2007 216 228
218. K. Nakadai, T. Takahashi, H.G. Okuno, H. Nakajima, Y. Hasegawa, and H. Tsujino Design and implementation of robot audition system 'HARK' - open source software for listening to three simultaneous speakers Adv. Robot. 24 5-6 2010 739 761
219. D.A. Reynolds, and R.C. Rose Robust text-independent speaker identification using Gaussian mixture speaker models IEEE Trans. Speech Audio Process. 3 1 1995 72 83
220. D.A. Reynolds, T.F. Quatieri, and R.B. Dunn Speaker verification using adapted Gaussian mixture models Digit. Signal Process. 10 1 2000 19 41
221. M. Ji, S. Kim, H. Kim, and H.-S. Yoon Text-independent speaker identification using soft channel selection in home robot environments IEEE Trans. Consum. Electron. 54 1 2008 140 144
222. Y. Matsusaka, T. Tojo, S. Kubota, K. Furukawa, D. Tamiya, K. Hayata, Y. Nakano, T. Kobayashi, Multi-person conversation via multi-modal interface-a robot who communicate with multi-user-. in: EUROSPEECH, vol. 99, 1999, pp. 1723-1726.
223. K. Nakadai, D. Matsuura, H.G. Okuno, and H. Tsujino Improvement of recognition of simultaneous speech signals using av integration and scattering theory for humanoid robots Speech Commun. 44 1 2004 97 112
224. M. Katzenmaier, R. Stiefelhagen, and T. Schultz Identifying the addressee in human-human-robot interactions based on head pose and speech Proceedings of the 6th International Conference on Multimodal Interfaces 2004 ACM 144 151
225. H. Holzapfel, Towards development of multilingual spoken dialogue systems, in: Proceedings of the 2nd Language and Technology Conference, 2005.
226. C. Cullen, C. Goodman, P. McGloin, A. Deegan, and E. McCarthy Reusable, interactive, multilingual online avatars Conference for Visual Media Production CVMP'09 2009 IEEE 152 158
227. K.R. Echavarria, M. Genereux, D.B. Arnold, A.M. Day, J.R. Glauert, Multilingual virtual city guides, in: Proceedings Graphicon, Novosibirsk, Russia, 2005.
228. T. Starner, J. Weaver, and A. Pentland Real-time American sign language recognition using desk and wearable computer based video IEEE Trans. Pattern Anal. Mach. Intell. 20 12 1998 1371 1375
229. C. Vogler, and D. Metaxas Handshapes and movements: multiple-channel American sign language recognition Gesture-Based Communication in Human-Computer Interaction 2004 Springer 247 258
230. G. Murthy, and R. Jadon A review of vision based hand gestures recognition Int. J. Inf. Technol. Knowl. Manag. 2 2 2009 405 410
231. H. Brashear, T. Starner, P. Lukowicz, H. Junker, Using multiple sensors for mobile sign language recognition, 2003.
232. R. Plamondon, and S.N. Srihari Online and off-line handwriting recognition: a comprehensive survey IEEE Trans. Pattern Anal. Mach. Intell. 22 1 2000 63 84
233. T. Plötz, and G.A. Fink Markov models for offline handwriting recognition: a survey Int. J. Doc. Anal. Recogn. (IJDAR) 12 4 2009 269 298
234. L.M. Lorigo, and V. Govindaraju Offline arabic handwriting recognition: a survey IEEE Trans. Pattern Anal. Mach. Intell. 28 5 2006 712 724
235. N. Mavridis Autonomy, isolation, and collective intelligence J. Artif. Gen. Intell. 3 1 2011 1 9