Smile and the world will smile with you - The effects of a virtual agent's smile on users' evaluation and behavior

International Journal of Human Computer Studies

Volumn 71, Issue 3, 2013, Pages 335-349

  Krämer, Nicole ^a   Kopp, Stefan ^b   Becker Asano, Christian ^c   Sommer, Nicole ^a  

^a UNIVERSITY OF DUISBURG ESSEN   (Germany)

^b BIELEFELD UNIVERSITY   (Germany)

^c UNIVERSITY OF FREIBURG   (Germany)

Author keywords

Embodied conversational agents; Media equation; Mimicry; Nonverbal behavior; Smiling

Indexed keywords

EMBODIED CONVERSATIONAL AGENT; MEDIA EQUATION; MIMICRY; NONVERBAL BEHAVIOR; SMILING;

COMMUNICATION;

VIRTUAL REALITY;

EID: 84871816391     PISSN: 10715819     EISSN: 10959300     Source Type: Journal    
DOI: 10.1016/j.ijhcs.2012.09.006     Document Type: Article

References (111)

1. M. Argyle, and M. Cook Gaze and Mutual Gaze 1976 Cambridge University Press Cambridge
2. J.N. Bailenson, and N. Yee Digital chameleons. Automatic assimilation of nonverbal gestures in immersive virtual environments Psychological Science 16 2005 814 819
3. J.N. Bailenson, J. Blascovich, A.C. Beall, and J.M. Loomis Interpersonal distance in immersive virtual environments Personality and Social Psychology Bulletin 29 2003 1 15
4. J.A. Bargh The four horsemen of automaticity: awareness, intention, efficiency, and control in social cognition R.S. Wyer, T.K. Scrull, Handbook of Social Cognition Vol. 1 1994 Basic processes. LEA, Hillsdale 1 40
5. J.B. Bavelas, A. Black, N. Chovil, C.R. Lemery, and J. Mullett Form and function in motor mimicry: topographic evidence that the primary function is communicative Human Communication Research 14 3 1988 275 299
6. J.B. Bavelas, A. Black, C.R. Lemery, and J. Mullett I show how you feel: motor mimicry as a communicative act Journal of Personality and Social Psychology 50 1986 322 329
7. J. Baudouin, D. Gilibert, S. Sansone, and G. Tiberghien When the smile is a cue to familiarity Memory 8 2000 285 292
8. Becker, C., Kopp, S., Wachsmuth, I., 2004. Simulating the emotion dynamics of a multimodal conversational agent. In: Affective Dialogue Systems: Tutorial and Research Workshop (ADS 2004). Springer LNAI 3068, Berlin, pp. 154-165.
9. C. Becker-Asano, S. Kopp, and I. Wachsmuth Why emotions should be integrated into conversational agents T. Nishida, Conversational Informatics: An Engineering Approach 2007 John Wiley & Sons 49 68
10. G. Bente, N.C. Krämer, A. Petersen, and J.P. de Ruiter Computer animated movement and person perception. methodological advances in nonverbal behavior research Journal of Nonverbal Behavior 25 3 2001 151 166
11. F.J. Bernieri, J.S. Gilles, J.M. Davis, and J.E. Grahe Dyad rapport and the accuracy of its judgment across situations: a lens model analysis Journal of Personality and Social Psychology 71 1996 110 129
12. F.J. Bernieri, and R. Rosenthal Interpersonal coordination: behavioral matching and interactional synchrony R.S. Feldman, B. Rimé, Fundamentals of Nonverbal Behavior 1991 Cambridge University Press Cambridge 401 432
13. Bhatt, K., Evens, M. & Argamon, S., 2004. Hedged responses and expressions of affect in human/human and human/computer tutorial interactions. Proceedings of the 26th Annual Meeting of the Cognitive Science Society, August 2004.
14. Bickmore, T., 2002. Towards the design of multimodal interfaces for handheld conversational characters. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 788-789.
15. T. Bickmore, and T. Giorgino Health dialog systems for patients and consumers Journal of Biomedical Informatics 39 5 2006 556 571
16. J. Blascovich, J. Loomis, A.C. Beall, K.R. Swinth, C.L. Hoyt, and J.N. Bailenson Immersive virtual environment technology as a methodological tool for social psychology Psychological Inquiry 13 2002 103 124
17. J.K. Burgoon, L. Dillman, and L. Stern Adaptation in dyadic interaction: defining and operationalizing patterns of reciprocity and compensation Communication Theory 4 1993 295 316
18. Cañamero, L., 2002. Playing the emotion game with Feelix: What can a LEGO robot tell us about emotion?, in: Dautenhahn, K., Bond, A., Canamero, L., Edmonds, B. (Eds.), Socially intelligent agents: creating relationships with computers and robots. Kluwer, Norwell, Ma, pp. 69-76. Available: 〈http://homepages.feis.herts.ac.uk/∼comqlc/papers/〉 [3.9.2006].
19. J.N. Cappella Mutual adaptation and relativity of measurement B.M. Montgomery, S. Duck, Studying Interpersonal Interaction 1991 103 117
20. J. Cassell, and T. Bickmore External manifestations of trustworthiness in the interface Communications of the ACM 43 12 2000 50 56
21. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., Yan, H., 1999. Embodiment in conversational interfaces: Rea. CHI'99 Conference Proceedings. Association for Computing Machinery, pp. 520-527.
22. J. Cassell, T. Bickmore, L. Campbell, H. Vilhjálmsson, and H. Yan Human conversation as a system framework: designing embodied conversational agents J. Cassell, J. Sullivan, S. Prevost, E. Churchill, Embodied Conversational Agents 2000 MIT Press Cambridge 29 63
23. Cassell, J., Stocky, T., Bickmore, T., Gao, Y., Nakano, Y., Ryokai, K., Tversky, D., Vaucelle, C., Vilhjálmsson, H., 2002. MACK: media lab autonomous conversational Kiosk. Proceedings of Imagina02. February 12-15, Monte Carlo.
24. T.L. Chartrand, and J.A. Bargh The chameleon effect: the perception-behavior link and social interaction Journal of Personality and Social Psychology 76 6 1999 893 910
25. T.L. Chartrand, W.W. Maddux, and J.L. Lakin Beyond the perception-behavior link: the ubiquitous utility and motivational moderators of nonconscious mimicry R.R. Hassin, J.S. Uleman, J.A. Bargh, The New Unconscious 2005 Oxford University Press Oxford 334 361
26. Chiasson, S. & Gutwin, C., 2005. Testting the media equation with children. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 829-838.
27. W.S. Condon, and W.D. Ogston Sound-film analysis of normal and pathological behavior patterns Journal of Nervous and Mental Disease 143 1966 338 347
28. Cowell, A.J., Stanney, K.M., 2003. Embodiment and interaction guidelines for designing credible, trustworthy embodied conversational agents, in: Rist, T., Aylett, R., Ballin, D., Rickel, J. (Eds.), Proceedings of the Fourth International Working Conference on Intelligent Virtual Agents (IVA'2003). Springer, Berlin, pp. 301-309.
29. Dahlbäck, N., Jönsson, A., Ahrenberg, L., 1993. Wizard of Oz studies - why and how. In Proceedings of the ACM International Workshop on Intelligent User Interfaces. Available: 〈http://citeseer.ist.psu.edu/45570. html〉 [05.07.2006].
30. D.M. Dehn, and S. van Mulken The impact of animated interface agents: a review of empirical research International Journal of Human-Computer Studies 52 2000 1 22
31. F.M. Deutsch, D. LeBaron, and M.M. Fryer What is in a smile? Psychology of Women Quarterly 11 1987 341 352
32. U. Dimberg Facial electromyography and emotional responses Psychophysiology 27 1990 481 494
33. U. Dimberg, and M. Thunberg Rapid facial reactions to different emotionally relevant stimuli Scandinavian Journal of Psychology 39 1998 39 45
34. U. Dimberg, M. Thunberg, and K. Elmehed Unconscious facial reactions to emotional facial expressions Psychological Science 11 2001 86 89
35. Eibl-Eibesfeld, I., 1997. Die Biologie des menschlichen Verhaltens: Grundriss der Humanethologie. Piper, München.
36. P. Ekman, and W.V. Friesen The Facial Action Coding System 1978 Consulting Psychology Press Palo Alto, CA
37. S. Frey, H.-P. Hirsbrunner, A. Florin, W. Daw, and R. Crawford A unified approach to the investigation of nonverbal and verbal behavior in communication research W. Doise, S. Moscovici, Current Issues in European Social Psychology (S. 143-199) 1983 Cambridge University Press Cambridge
38. Garson, G. David (2010). Inter-rater reliability, from Statnotes: Topics in Multivariate Analysis. Retrieved 05/14/2010 from 〈http://faculty.chass. ncsu.edu/garson/ pa765/statnote.htm〉.
39. H. Giles Accommodation theory: some new directions M.W.S. Silva, Aspects of Linguistic Behavior: Festschrift Robert Le Page 1980 Papers in Linguistics New York 105 136
40. H. Giles, A. Mulac, J.J. Bradac, and P. Johnson Speech accommodation theory: the first decade and beyond M.L. McLaughlin, Communication Yearbook vol. 10 1987 Sage Beverland Hills, CA 13 48
41. A.C. Graesser, K. Wiemer-Hastings, P. Wiemer-Hastings, and R. Kreuz AutoTutor: a simulation of a human tutor Journal of Cognitive Systems Research 1 1999 35 51
42. Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S., Morales, M., van der Werf, R., Morency, L.P., 2006. Virtual Rapport, in: Gratch, J. et al. (Eds), 6th International Conference on Intelligent Virtual Agents. Springer: Berlin.
43. U. Hess, P. Philippot, and S. Blairy Mimicry: facts and fiction P. Phillipot, R.S. Feldman, E.J. Coats, The Social Context of Nonverbal Behaviour 1999 Cambridge University Press Cambridge 213 241
44. U. Hess, P. Phillippot, and S. Blairy Facial reactions to emotional facial expressions: affect or cognition? Cognition and Emotion 22 4 1998 509 531
45. J.L. Horn A rationale and a test for the number of factors in factor analysis Psychometrika 30 1965 179 185 10.1007/BF02289447
46. C.K. Hsee, E. Hatfield, J.G. Carlson, and C. Chemtob The effect of power on susceptibility to emotional contagion Cognition and Emotion 4 4 1990 327 340
47. D. Hurwitz, N. Hirschberg-Wiggins, and L.E. Jones A semantic differential for facial attribution: the face differential Bulletin of the Psychonomic Society 6 4A 1975 370 372
48. Kaiser, S., Wehrle, T., 2005. Animating and Analyzing Facial Expressions in Human-Computer Interactions: An Appraisal based Approach. Paper Presented at the ISRE General Meeting, July 2005, Bari.
49. Kanning, U.P., 1999. Die Psychologie der Personenbeurteilung. Hogrefe, Göttingen.
50. S. Katz The importance of being beautiful J.M. Henslin, Down to Earth Sociology. Introductory Readings eleventh ed. 2001 The Free Press New York 297 303
51. A. Kendon The role of visible behavior in the organization of social interaction M. von Cranach, I. Vine, Social Communication and Movement 1973 Wiley London 29 74
52. S. Kiesler, and L. Sproull Social human-computer interaction B. Friedman, Human Values and the Design of Computer Technology 1997 Cambridge University Press Cambridge 191 199
53. J.M. Kilner, Y. Pauligan, and S.J. Blakemore An interference effect of observed biological movement on action Current Biology 13 2003 522 525
54. Kopp, S., Gesellensetter, L., Krämer, N.C., Wachsmuth, I., 2005. A conversational agent as museum guide - design and evaluation of a real-world application, in: Aylett, R. et al. (Eds.), IVA 2005, International Working Conference on Intelligent Virtual Agents, Springer, Berlin, pp. 329-343.
55. Kopp, S., Tepper, P., Cassell, J., 2004. Towards integrated microplanning of language and iconic gesture for multimodal output. Proceedings of the International Conference on Multimodal Interfaces (ICMI'04), Penn States, PA. Available: 〈http://www.techfak.uni-bielefeld.de/~skopp/publications. html〉 [3.9.2006].
56. N.C. Krämer Social communicative effects of a virtual program guide T. Panayiotopoulos, Intelligent Virtual Agents 2005 2005 Springer Hamburg 442 543
57. Krämer, N.C., Bente, G., Piesk, J., 2003a. The Ghost in the Machine. The Influence of Embodied Conversational Agents on user Expectations and user Behaviour in a TV/VCR Application, in Bieber, G., Kirste, T. (eds.), IMC Workshop 2003, Assistance, Mobility, Applications. Rostock, pp. 121-128.
58. N.C. Krämer, L. Hoffmann, and S. Kopp Know your users! Empirical results for tailoring an agent's nonverbal behavior to different user groups J. Allbeck, N. Badler, T. Bickmore, C. Pelachaud, A. Safonova, Intelligent Virtual Agents 2010 Springer Berlin 468 474
59. N.C. Krämer, and J. Nitschke Ausgabemodalitäten im Vergleich: verändern sie das Eingabeverhalten der Benutzer? R. Marzi, V. Karavezyris, H.-H. Erbe, K.P. Timpe, Bedienen und Verstehen. 4. Berliner Werkstatt Mensch-Maschine-Systeme 2002 VDI-Verlag Düsseldorf 231 248
60. Krämer, N.C., Tietz, B., Bente, G., 2003b. Effects of embodied interface agents and their gestural activity., in: Aylett, R. et al. (Eds.), IVA 2005, International Working Conference on Intelligent Virtual Agents, Springer, Berlin, pp. 292-300.
61. N.C. Krämer Bewegende Bewegung. Sozio-emotionale Wirkungen nonverbalen Verhaltens und deren experimentelle Untersuchung mittels Computeranimation 2001 Pabst Lengerich
62. N.C. Krämer, N. Simons, and S. Kopp The effects of an embodied agent's nonverbal behavior on user's evaluation and behavioural mimicry C. Pelachaud, Intelligent Virtual Agents 2007 2007 Springer Berlin 238 251
63. N.C. Krämer Social effects of virtual assistants. A review of empirical results with regard to communication H. Prendinger, J. Lester, M. Ishizuka, Intelligent Virtual Agents 2008 Springer Berlin 507 508
64. R.E. Kraut, and R.E. Johnston Social and emotional messages of smiling: an ethological approach Journal of Personality and Social Psychology 37 9 1979 1539 1553
65. M. LaFrance Posture mirroring and rapport J. Davis, Interaction Rhythms: Periodicity in Communicative Behaviour 1982 Human Sciences Press New York 279 298
66. M. LaFrance, and M. Broadbent Group rapport: posture sharing as a nonverbal indicator Group & Organization Studies 1 1976 328 333
67. M. LaFrance, and M.A. Hecht Why smiles generate leniency Personality and Social Psychology Bulletin 21 1995 207 214
68. M. LaFrance, and W. Ickes Posture mirroring and interactional involvement: sex and sex typing effects Journal of Nonverbal Behavior 5 3 1981 139 154
69. J.L. Lakin, and T.L. Chartrand Using nonconsious behavioral mimicry to create affiliation and rapport Psychological Science 14 4 2003 334 339
70. J.L. Lakin, V.E. Jefferis, C.M. Cheng, and T.L. Chartrand The chameleon effect as social glue: evidence for the evolutionary significance of nonconscious mimicry Journal of nonverbal Behavior 27 3 2003 145 162
71. E.J. Langer Mindfulness 1989 Addison-Wesley, Reading MA
72. Lester, J.C., Converse, S.A., Kahler, S.E., Barlow, S.T., Stone, B.A. & Bogal, R.S., 1997. The persona effect: affective impact of animated pedagogical agents, in: S. Pemberton (Ed.), Human Factors in Computing Systems: CHÍ97 Conference Proceedings. ACM Press, New York, pp. 59-366.
73. J.C. Lester, S.G. Towns, C.B. Callaway, J.L. Voerman, and P.J. FitzGerald Deictic and emotive communication in animated pedagogical agents J. Cassell, J. Sullivan, S. Prevost, E. Churchill, Embodied Conversational agents 2000 MIT Press Boston 123 154
74. K. Likowski, A. Mühlberger, B. Seibt, P. Pauli, and P. Weyers Modulation of facial mimicry by attitudes Journal of Experimental Social Psychology 44 2008 1065 1072
75. Lipps, T., 1907. Das Wissen von fremden Ichen, in: Lipps, T. (Ed.), Psychologische Untersuchungen, 1, Engelmann, Leipzig, pp. 694-722.
76. V. Manusov Reacting to changes in nonverbal behaviors: relational satisfaction and adaptation patterns in romantic dyads Human Communication Research 21 4 1995 456 477
77. J.C. McCroskey Scales for the measurement of ethos Speech Monographs 33 1966 65 72
78. A.E. Milewski, and S.H. Lewis Delegating to software agents International Journal of Human-Computer Studies 46 1997 485 500
79. A. Mehrabian Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament Current Psychology: Developmental, -learning, -Personality, -social 14 4 1996 261 292
80. R. Moreno Software agents in multimedia: an experimental study of their contributions to students' learning Human-Computer Interaction Proceedings 2001 275 277
81. C. Nass, and Y. Moon Machines and mindlessness: social responses to computers Journal of Social Issues 56 1 2000 81 103
82. C. Nass, Y. Moon, J. Morkes, E-Y. Kim, and B.J. Fogg Computers are social actors: a review of current research B. Friedman, Moral and Ethical Issues in Human-Computer Interaction. CSL 1997 Press Stanford, CA 137 162
83. Nass, C., Steuer, J., Tauber, E.R., 1994. Computers are social actors, in: Adelson, B., Dumais, S., Olson, J. (Eds.), Human Factors in Computing Systems: CHI′94 Conference Proceedings, ACM Press, pp. 72-78.
84. R. Neumann, and F. Strack Mood contagion: the automatic transfer of mood between persons Journal of Personality and Social Psychology 79 2 2000 211 223
85. C.E. Osgood Dimensionality of the semantic space for communication via facial expressions Scandinavian Journal of Psychology 7 1966 1 30
86. S.L. Oviatt, and B. Adams Designing and evaluating conversational interfaces with animated characters J. Cassell, J. Sullivan, S. Prevost, E. Churchill, Embodied Conversational Agents 2000 MIT Press Boston 319 345
87. S.L. Oviatt, C. Darves, and R. Coulston Toward adaptive conversational interfaces: modeling speech convergence with animated personas ACM Transactions on Computer-Human Interaction 3 2004 300 328
88. E. Oztop, D.W. Franklin, T. Chaminade, and G. Cheng Human-humanoid interaction: is a humanoid robot perceived as a human? International Journal of Humanoid Robotics 2 4 2005 537 559
89. S. Parise, S. Kiesler, L. Sproull, and K. Waters Cooperating with life-like interface agents Computers in Human Behavior 15 1999 123 142
90. Pelachaud, C., Carofiglio, V., De Carolis, B. de Rosis, F., Poggi, I., 2002. Embodied contextual agent in information delivering application. Proceedings of AAMAS′02: The First International Joint Conference on Autonomous Agents & Multi-Agent Systems, ACM, Bologna, pp. 758-765.
91. C. Press, G. Bird, R. Flach, and C. Heyes Robotic movement elicits automatic imitation Cognitive Brain Research 25 2005 632 640
92. C. Press, H. Gillmeister, and C. Heyes Bottom-up, not top-down, modulation of imitation by human and robotic models European Journal of Neuroscience 24 2006 2415 2419
93. B. Reeves, and C.I. Nass The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places 1996 Cambridge University Press New York
94. H.T. Reis, I.M. Wilson, C. Monestere, S. Bernstein, K. Clark, and E. Seidl What is smiling is beautiful and good European Journal of Social Psychology 20 3 1990 259
95. R. Rickenberg, and B. Reeves The effects of animated characters on anxiety, task performance, and evaluations of user interfaces Letters of CHI 2000 2000 49 56 April 2000
96. B. Rosemann, and M. Kerres Interpersonales Wahrnehmen und Verstehen 1986 Hans Huber Bern
97. A.E. Scheflen The significance of posture in communication systems Psychiatry 27 1964 316 321
98. L. Schilbach, A. Wohlschlaeger, N.C. Krämer, A. Newen, K. Zilles, J.N. Shah, G.R. Fink, and K. Vogeley Being with virtual others: neural correlates of social interaction Neuropsychologia 44 2006 718 730
99. Shechtman, N. & Horowitz, L.M., 2003. Media inequality in conversation: how people behave differently when interacting with computers and people, Proceedings of the SIGCHI conference on Human factors in computing systems, Ft. Lauderdale, Florida, USA.
100. Sidner, C.L., Lee, C., Morency, L-P, Forlines, C., 2006. The Effect of head-nod recognition in human-robot conversation, ACM SIGCHI/SIGART Conference on Human-Robot Interaction (HRI), 290-296.
101. M. Slater, and A. Steed Meeting people virtually: experiments in shared virtual environments R. Schroeder, The Social Life of Avatars: Presence and Interaction in Shared Virtual Environments 2002 Springer Berlin 145 171
102. J. Smith GrandChair: Conversational Collection of Family Stories. Media Lab 2000 MIT Cambridge, MA
103. L. Sproull, M. Subramani, S. Kiesler, J.H. Walker, and K. Waters When the interface is a face Human Computer Interaction 11 2 1996 97 124
104. Takeuchi, A., Naito, T., 1995. Situated facial displays: towards social interaction, in: Katz, I., Mack, R., Marks, L., Rosson, M.B., Nielsen, J. (Eds.), Human factors in computing Systems: CHI'95 Conference Proceedings. ACM Press, New York, pp. 450-455.
105. Thórisson, K.R., 1996. Communicative Humanoids. A Computational Model of Psychosocial Dialogue Skills. PH.D. Thesis, MIT.
106. L. Tickle-Degnen, and R. Rosenthal Group rapport and nonverbal behavior Review of Personality and Social Psychology 9 1987 113 136
107. Van Baaren, R., 2003. Mimicry: a social perspective. Available: 〈http://webdoc.ubn.kun.nl/mono/b/baaren-r-van/mimi.pdf〉 [10.02.2006].
108. R.B. Van Baaren, R.W. Holland, K. Kawakami, and A. van Knippenberg Mimicry and prosocial behavior Psychological Science 15 2004 71 74
109. H.G. Wallbott Congruence, contagion, and motor mimicry: mutualities in nonverbal exchange J. Markova, C.F. Graumann, K. Foppa, Mutalities in Dialogue 1995 Cambridge University Press Cambridge 82 98
110. P. Weyers, A. Mühlberger, C. Hefele, and P. Pauli Electromyographic responses to static and dynamic avatar emotional facial expressions Psychophysiology 43 2006 450 453
111. P. Weyers, A. Mühlberger, A. Kund, U. Hess, and P. Pauli Modulation of facial reactions to avatar emotional faces by nonconscious competition priming Psychophysiology 46 2009 328 335