A Framework for Emotions and Dispositions in Man-Companion Interaction

Coverbal Synchrony in Human-Machine Interaction

Volumn , Issue , 2013, Pages 99-140

  Traue, Harald C ^a   Ohl, Frank ^b   Brechmann, André ^c   Schwenker, Friedhelm ^a   Kessler, Henrik ^a   Limbrecht, Kerstin ^a   Hoffmann, Holger ^a   Scherer, Stefan ^d   Kotzyba, Michael ^b   Scheck, Andreas ^a   Walter, Steffen ^a  

^a UNIVERSITY OF ULM   (Germany)

^b OTTO VON GUERICKE UNIVERSITY   (Germany)

^c LEIBNIZ INSTITUTE FOR NEUROBIOLOGY   (Germany)

^d University of Ulm ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84922779065     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (107)

1. Abler, B. and H. Kessler. 2009. Emotion Regulation Questionnaire-Eine Deutsche Version des ERQ von Gross and John. Diagnostica, 55(3): 144-152.
2. Antonovsky, A. 1987. Unraveling the Mystery of Health-How People Manage Stress and Stay Well. Jossey-Bass Publishers, San Francisco.
3. Asendorpf, D. 2011. Netz auf Rädern. DIE ZEIT, March 10th, 40 p.
4. Aubrey, B. 2010. Managing Your Aspirations: Developing Personal Enterprise in the Global Workplace. McGraw-Hill Education, Singapore.
5. Bandura, A. 2001. Social cognitive theory: An agentic perspective. Ann. Rev. Psychol., 52:1-26.
6. Bar-Shalom, Y. and X.R. Li. 1995. Multitarget-Multisensor Tracking-Principles and Techniques. YBS, Urbana, IL.
7. Barth, A. and R. Herpers. 2005. Robust head detection and tracking in cluttered workshop environment using GMM, pp. 442-450. In DAGM symposium, LNCS 3663. Springer, New York, Berlin.
8. Bartlett, M. G. Littlewort, E. Vural, K. Lee, M. Cetin, A. Ercil and M. Movellan. 2008. Data mining spontaneous facial behavior with automatic expression coding. Lecture Notes in Computer Science 5042, Springer, Berlin, pp. 1-21.
9. Baum, S.R. and M.D. Pell. 1999. The neural bases of prosody: Insights from lesion studies and neuroimaging. Aphasiology, 13:581-608.
10. Becker-Asano, C. 2008. WASABI: Affect Simulation for Agents with Believable Interactivity. Dissertation der Universität Bielefeld.
11. Bierhoff, H.-W. 2011. Attribution: In Lexikon der Psychologie. http://www.wissenschaft-online.de/abo/lexikon/psycho/1584.
12. Bishop, C. 2006. Pattern Recognition and Machine Learning. Springer, New York, Berlin.
13. Biundo, S. and A. Wendemuth. 2010. Von kognitiven technischen Systemen zu Companion-Systemen. Künstliche Intelligenz, 24(4):335-339.
14. Bless, H., M. Wänke, G. Bohner and R.F. Fellhauer. 1994. Need for Cognition: Eine Skala zur Erfassung von Engagement und Freude bei Denkaufgaben. Zeitschrift für Sozialpsychologie, 25:147-154.
15. Böck, R., K. Limbrecht, S. Walter, S. Glüge, D. Hrabal, A. Wendemuth and H.C. Traue. 2012. Intraindividual and interindividual multimodal emotion analyses in human-machine-interaction. Proceedings of 2012 IEEE Conference on Cognitive Methods in Situation Awareness and Decision Support, pp. 59-64.
16. Borst, M., G. Langner and G. Palm. 2004. A biologically motivated neural network for phase extraction from complex sounds. Biological Cybernetics, 90:98-104.
17. Bradley, M.M. 2009. Natural selective attention: Orienting and emotion. Psychophysiology. 46(1):1-11.
18. Bradley, M.M. and P.J. Lang. 1994. Measuring emotion: The self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry, 25:49-59.
19. Bradley, M.M. and P.J. Lang. 2008. The International Affective Picture System (IAPS) in the study of emotion and attention, pp. 29-46. In: J.A. Coan and J.J. Allen (eds.), Handbook of Emotion Elicitation and Assessment. Oxford University Press, Oxford.
20. Bryson, J.J. 2010. Robots should be slaves, pp. 63-74. In: Y. Wilks (ed.), Close Engagements with Artificial Companions: Key Social, Psychological, Ethical and Design Issues. John Benjamins, Amsterdam.
21. Caridakis, G., G. Castellano, L. Kessous, A. Raouzaiou, L. Malatesta and S. Asteriadis. 2007. Multimodal emotion recognition from expressive faces, body gestures and speech, pp. 375-388. In: C. Boukis, L. Pnevmatikakis and L. Polymenakos (eds.), Artificial Intelligence and Innovations 2007: From Theory to Applications. Springer, Boston, Berlin.
22. Cohn, J.F. 2010. Advances in behavioral science using automated facial image analysis and synthesis. IEEE Signal Processing Magazine, 128(6):128-133.
23. Colombetti, G. 2009. From affect programs to dynamical discrete emotions. Philosophical Psychology, 22(4):407-445.
24. Corradini, A. 2001. Real-time gesture recognition by means of hybrid recognizers, pp. 34-46. Proceedings of the International Gesture Workshop on Gesture and Sign Languages in Human-Computer Interaction, USA.
25. Corradini, A. and H. Gross. 2000. Camera-based gesture recognition for robot control, pp. 133-138. In Proceedings of IJCNN’2000.
26. Costa, P.T. and R.R. McCrae. 2002. Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI) 2002. Psychological Assessment Resources, Odissa.
27. Cowie, R. and R.R. Cornelius. 2003. Describing the emotional states that are expressed in speech. Speech Communication, 40:5-32.
28. Deighton, R.M. and H.C. Traue. 2006. Emotionale Ambivalenz, Körperbeschwerden, Depressivität und soziale Interaktion: Untersuchungen zur deutschen Version des Ambivalence over Emotional Expressiveness Questionnaire AEQ-G18. Zeitschrift für Gesundheitspsychologie, 14:158-170.
29. Dietrich, C., G. Palm and F. Schwenker. 2003. Decision templates for the classification of bioacoustic time series. Information Fusion, 4:101-109.
30. Ekman, P. and W. Friesen. 1978. Facial Action Coding System: Investigator’s Guide. Palo Alto: Consulting Psychologists Press.
31. Frick, R.W. 1985. Communicating emotion: The role of prosodic features. Psychol. Bull., 97:412-429.
32. Friederici, A.D. and K. Alter. 2004. Lateralization of auditory language functions: A dynamic dual pathway model. Brain Lang, 89:267-276.
33. Frijda, N.H. 1988. The laws of emotion. Am. Psychol., 43:349-358.
34. Frijda, N.H. 2007. The Laws of Emotion. Erlbaum, Mahwah New Jersey and London.
35. Frijda, N.H. and M. Zeelenberg. 2001. Appraisal: What is the dependent? pp. 141-155. In: K.R. Scherer, A. Schorr and T. Johnstone (eds.), Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press, New York.
36. Garcia-Prieto, P. and K.R. Scherer. 2006. Connecting social identity theory and cognitive appraisal theory of emotions, pp. 189-208. In: R. Brown and D. Capozza (eds.), Social Identities: Motivational, Emotional, Cultural Influences. Psychology Press, New York.
37. Glodek, M., L. Bigalke, M. Schels and F. Schwenker. 2011a. Incorporating uncertainty in a layered HMM architecture for human activity recognition, pp. 33-34. Proceedings of the 2011 Joint ACM Workshop on Human Gesture and Behavior Understanding. 2011. ACM.
38. Glodek, M., S. Tschechne, G. Layher, M. Schels, T. Brosch, S. Scherer, M. Kächele, M. Schmidt, H. Neumann, G. Palm and F. Schwenker. 2011b. Multiple classifier systems for the classification of audio-visual emotional states, pp. 359-368. In: S.D’Mello et al. (eds.), ACII 2011, Part II, LNCS 6975, Springer, Heidelberg.
39. Gnjatovic, M. and D. Rösner. 2010. Inducing genuine emotions in simulated speech-based human-machine interaction: The NIMITEK Corpus. IEEE Transactions on Affective Computing, 1:132-144.
40. Gross, J.J. and O.P. John. 2003. Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. J. Pers. Soc. Psychol., 85:348-362.
41. Gu, Y., S.L. Tan, K.J. Wong, M.H.R. Ho and L. Qu. 2008. Emotion-aware technologies for consumer electronics. IEEE International Symposium on Consumer Electronics, Vilamoura, pp. 1-4.
42. Guralnik, D.B. 1970. Webster’s New World Dictionary. Foster, Toronto/Canada.
43. Haag, A., S. Goronzy, P. Schaich and J. Williams. 2004. Emotion recognition using biosensors: First step towards an automatic system. Affective Dialogue Systems, Tutorial and Research Workshop, Kloster Irsee, Germany, pp. 36-48.
44. Hrabal, D., S. Rukavina, K. Limbrecht, S. Walter, V. Hrabal, S. Gruss and H.C. Traue. 2012. Emotion Identification and Modelling on the Basis of Paired Physiological Data Features for Companion Systems. Proceedings of the MATHMOD 2012-7th Vienna International Conference on Mathematical Modelling. Vienna, Austria.
45. Izard, C.E., F.E. Dougherty, B.M. Bloxom and N.E. Kotsch. 1974. The Differential Emotions Scale: A Method of Measuring the Meaning of Subjective Experience of Discrete Emotions. Nashville: Vanderbilt University, Department of Psychology.
46. James, T.W., G.K. Humphrey, G.S. Gati, S.J.S. Ravi, R.S. Menon and M.A. Goodale. 2000. The effects of visual object priming on brain activation before and after recognition. Current Biol., 10:1017-1024.
47. Keller, J., S. Ringelhan and F. Blomann, 2011. Does skills-demands compatibility result in intrinsic motivation? Experimental test of a basic notion proposed in the theory of flow-experiences. Journal of Positive Psychology, 6(5):408-417.
48. Kernberg, O.F. 1992. Objektbeziehungen und Praxis der Psychoanalyse. Klett-Cotta Stuttgart.
49. Kessler, H., A. Festini, H.C. Traue, S. Filipic, M. Weber and H. Hoffmann. 2008. SIMPLEX-Simulation of Personal Emotion Experience, Affective Computing. pp. 255-271. In: Jimmy Or (ed.), Affective Computing: Emotion Modelling, Synthesis and Recognition, I-Tech Education and Publishing.
50. Kessler, H., H.C. Traue, M. Hopfensitz, C. Subic-Wrana and H. Hoffmann. 2010. Level of Emotional Awareness Scale-Computer (LEAS-C): Deutschsprachige digitale Version. Psychotherapeut, Bnd 55, Heft, 4:329-331.
51. Kim, J. and E. André. 2008. Emotion recognition based on physiological changes in music listening., IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:2067-2083.
52. Kohut, H. 1987. The Kohut Seminars on Self Psychology and Psychotherapy with Adolescents and Young Adults. W.W. Norton & Company, New York.
53. Kolodyazhniy, V., S. Kreibig, J.J. Gross, W. Roth and F.H. Wilhem. 2011. An affective computing approach to physiological emotion specificity: Toward subject-independent and stimulus-independent classification of film-induced emotions. Psychophysiology, 48:908-922.
54. Krapp, A. 2002. Structural and dynamic aspects of interest development: theoretical considerations from an ontogenetic perspective. Learning and Instruction. 12:383-409.
55. Kreibig, S.D., F.H. Wilhelm, W.T. Roth and J.J. Gross. 2007. Cardiovascular, electrodermal, and respiratory response patterns to fear and sadness-inducing films. Psychophysiology, 44:787-806.
56. Kreibig, S.D., T. Brosch and G. Schaefer, 2010. Psychophysiological response patterning in emotion: Implications for affective computing, pp. 105-130. In: K.R. Scherer, T. Baenziger and E. Roesch (eds.), A Blueprint for an Affectively Competent Agent: Cross-Fertilization Between Emotion Psychology, Affective Neuroscience, and Affective Computing. Oxford University Press, Oxford.
57. Kuhl, J. 2001. Motivation und Persönlichkeit. Interaktionen psychischer Systeme. Hogrefe, Göttingen.
58. Leventhal, H. and K. Scherer. 1987. The relationship of emotion to cognition: a functional approach to semantic controversy. Cognition and Emotion, (1):3-28.
59. Loewenstein, G. and J.S. Lerner, 2003. The role of affect in decision making, pp. 619-642. In: R.J. Davidson, K.R. Scherer and H.H. Goldsmith (eds.), Handbook of Affective Sciences. Oxford University Press, Oxford.
60. Maruta, T., R.C. Colligan, M. Malinchoc and K.P. Offord. 2002. Optimism-pessimism assessed in the 1960s and self-reported health status 30 years later. Mayo. Clin. Proc., 77:748-753.
61. Maslow, A.H. 1943. A theory of human motivation. Psychol. Rev., 50:370-396.
62. Mehrabian, A. 1996. Analysis of the big-five personality factors in terms of the PAD temperament model. Austral. J. Psychol., 48:86-92.
63. Miller, N.E. 1959. Liberalization of basic S-R concepts: Extension to conflict behavior, motivation, and social learning. In: S. Koch (ed.), Psychology: A Study of Science (Vol. 2. McGraw-Hill, New York).
64. Morguet, P. 2000. Stochastische Modellierung von Bildsequenzen zur Segmentierung und Erkennung dynamischer Gesten. Ph.D. thesis, TU München, München, Germany.
65. Nass, C. and S. Brave. 2005. Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship. MIT Press, Cambridge, MA.
66. Ong, S. and S. Ranganath, 2003. Classification of gesture with layered Meanings. Proceedings of International Gesture Workshop Singapore, pp. 239-246.
67. Ortony, A., G.L. Clore and A. Collins. 1988. The Cognitive Structure of Emotions. Cambrige: Cambrige University Press.
68. Pennebaker, J.W. and M.E. Francis. 1996. Cognitive, emotional, and language processes in disclosure. Cogn. Emot., 10:601-626.
69. Pentland, A. and S. Pentland. 2008. Honest Signals: How They Shape Our World. Bradford Books. Cambrige: MIT Press.
70. Picard, R. 2003. Affective computing: Challenges. International Journal of Human-Computer Studies, 59:55-64.
71. Picard, R.W., E. Vyzas and J. Healey. 2001. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23:1175-1191.
72. Puchalska-Wasyl, M. 2007. Types and functions of inner dialogues. Psychology of Language and Communication, 11:43-62.
73. Scherer, K.R. 2001. Appraisal considered as a process of multilevel sequential checking, pp. 92-120. In: K.R. Scherer, A. Schorr and T. Johnstone (eds.), Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press, New York.
74. Scherer, K.R. 2003a. Vocal communication of emotion: A review of research paradigms. Speech Communication, 40:227-256.
75. Scherer, K.R. 2003b. Introduction: Cognitive components of emotion, pp. 563571. In: R.J. Davidson, K.R. Scherer and H.H. Goldsmith (eds.), Handbook of Affective Sciences. Oxford University Press, Oxford.
76. Scherer, S., E. Trentin, F. Schwenker and G. Palm. 2009. Approaching emotion in human computer interaction. In: International Workshop on Spoken Dialogue Systems. (IWSDS’09), pp. 156-168.
77. Scherer, S., M. Glodek, M. Schels, M. Schmidt, G. Layher, F. Schwenker, et al. 2012a. A generic framework for the inference of user states in human computer interaction: How patterns of low level communicational cues support complex affective states. Journal on Multimodal User Interfaces, 6:117-141.
78. Scherer, S., M. Glodek, F. Schwenker, N. Campbell and G. Palm. 2012b. Spotting laughter in naturalistic multiparty conversations: A comparison of automatic online and offline approaches using audiovisual data. ACM Transactions on Interactive Intelligent Systems: Special Issue on Affective Interaction in Natural Environments, 2:111-144.
79. Scherer, S., M. Schels and G. Palm. 2011. How low level observations can help to reveal the user’s state in hci. In S. D’Mello, A. Graesser, B. Schuller, and J.-C.M. (eds.), Proceedings of the 4th International conference on Affective Computing and Intelligent Interaction (ACII’11), volume 2, pp. 81-90. Berlin: Springer.
80. Schuler, H. and M. Prochaska. 2000. Entwicklung und Konstruktvalidierung eines berufsbezogenen Leistungsmotivationstests. Diagnostica, 46:61-72.
81. Schuller, B., G. Rigoll and M. Lang, 2003. Hidden Markov model-based speech emotion recognition. Proceedings IEEE International Conference on Multimedia and Expo, pp. 401-404.
82. Schupp, H.T., B.N. Cuthbert, M.M. Bradley, C.H. Hillman, A.O. Hamm and P.J. Lang, 2004. Brain processes in emotional perception: Motivated attention. Cogn. Emot., 18:593-611.
83. Schwenker, F., C. Dietrich, C. Thiel and G. Palm. 2006. Learning of decision fusion mappings for pattern recognition. International Journal on Artificial Intelligence and Machine Learning (AIML), 6:17-21.
84. Skelley, J., R. Fischer, A. Sarma and B. Heisele. 2006. Recognizing expressions in a new database containing played and natural expressions. Proceedings of 18th International Conference on Pattern Recognition (ICPR), 1:1220-1225.
85. Sloman, A. 2010. Requirements for artificial companions: It’s harder than you think, pp. 179-200. In: Y. Wilks (ed.), Close Engagements with Artificial Companions. J. Benjamins Publishing Company, Amsterdam.
86. Stemmler, G. and J. Wacker. 2010. Personality, emotion, and individual differences in physiological responses. Biol. Psychol., 84:541-551.
87. Stenger, B., P. Mendonca and R. Cipolla. 2001. Model-based 3D tracking of an articulated hand. IEEE Conference on Computer Vision and Pattern Recognition, 2:310-315.
88. Tan, J.-W., S. Walter, A. Scheck, D. Hrabal, H. Hoffmann, H. Kessler and H.C. Traue. 2012. Repeatability of facial electromyography (EMG) activity over corrugator supercilii and zygomaticus major on differentiating various emotions. Journal of Ambient Intelligence and Humanized Computing, 3:3-10.
89. Traue, H.C. and R.M. Deighton. 2007. Emotional inhibition, pp. 908-913. In: George Fink (Editor-in-Chief), Encyclopedia of Stress, Second Edition, Volume 1. Academic Press, Oxford.
90. Traue, H.C. and H. Kessler. 2003. Psychologische Emotionskonzepte, pp. 20-33. In: A. Stephan and H. Walter (eds.), Natur und Theorie der Emotion. mentis Verlag, Paderborn.
91. Traue, H.C., H. Kessler and A.B. Horn. 2005. Emotion, Emotionsregulation und Gesundheit, pp. 149-171. In: R. Schwarzer (ed.), Gesundheitspsychologie. Enzyklopädie der Psychologie. Hogrefe, Göttingen.
92. Turtle, S. 2010. In good company, pp. 3-10. In: Y. Wilks (ed.), Close Engagements with Artificial Companions. J. Benjamins Publishing Company, Amsterdam.
93. Wagner, J. 2005. Vom physiologischen Signal zur Emotion: Implementierung und Vergleich ausgewählter Methoden zur Merkmalsextraktion und Klassifikation. M.S. thesis, Universität Augsburg, Augsburg, Germany.
94. Wagner, J., J. Kim and E. André. 2008. From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. Proceedings of IEEE International Conference on Multimedia and Expo ICME, pp. 940-943.
95. Walter, S., C. Wendt, J. Böhnke, S. Crawcour, J.W. Tan, A. Chan, K. Limbrecht, S. Gruss and H.C. Traue. 2013b. Similarities and differences of emotions in human-machine and human-human interactions: What kind of emotions are relevant for future companion systems? Ergonomics, Taylor & Francis, pp. 1-13.
96. Walter, S., D. Hrabal, A. Scheck, H. Kessler, G. Bertrain, F. Nodtdurft and H.C. Traue. 2010. Individual emotional profiles in Wizard-of-Oz-Experiments. In: F. Makedon, I. Maglogiannis and S. Kapidakis (eds.), Proceedings of the 3rd International Conference on Pervasive Technologies Related to Assistive Environments. S:58-63.
97. Walter, S., K. Jonghwa, D. Hrabal, S.C. Crawcour, H. Kessler, H.C. Traue. 2013a. Transsituational Individual-Specific Biopsychological Classification of Emotions. IEEE Transactions on Systems, Man and Cybernetics: Systems, 43(4):988-995.
98. Walter, S., S. Scherer, M. Schels, M. Glodek, D. Hrabal, M. Schmidt, R. Böck, K. Limbrecht, H.C. Traue and F. Schwenker. 2011. Multimodal Emotion Classification in Naturalistic User Behavior. In J.A. Jacko (ed.). Human Computer Interaction, Part III, HCI 2011, LNCS 6763, Springer-Verlag Berlin, pp. 603-611.
99. Weiner, B. 1986. An Attributional Theory of Motivation and Emotion. Springer, New York, Berlin.
100. Wendemuth, A. and S. Biundo. 2012. A Companion Technology for Cognitive Technical Systems. In: Cognitive Behavioural Systems. A. Esposito, A.M. Esposito, A. Vinciarelli, R. Hoffmann, V.C. Müller (eds.), LNCS 7403, Springer Berlin Heidelberg, pp. 89-103.
101. Wilks, Y. 2010. Introducing artificial companions, pp. 11-22. In: Y. Wilks (ed.), Close Engagements with Artificial Companions. J. Benjamins Publishing Company, Amsterdam.
102. Wimmer, M. and B. Radig. 2007. Automatically Learning the Objective Function for Model Fitting. Proceedings of the Meeting in Image Recognition and Understanding (MIRU) in Hiroshima, Japan.
103. Witherington, D.C. and J.A. Crichton. 2007. Framework for understanding emotions and their development: Functionalist and dynamic systems approaches. Emotion, 7:628-637.
104. Wolff, S., C. Kohrs, H. Scheich, A. Brechmann. 2011. Temporal contingency and prosodic modulation of feedback in human-computer interaction: Effects on brain activation and performance in cognitive tasks. Informatik, Lecture Notes in Informatics, Berlin, 192:238.
105. Woolf, B.P. 2010. Social and caring tutors: ITS 2010 Keynote Address, pp. 5-14. In: V. Aleven, J. Kay and J. Mostow (eds.), Intelligent Tutoring Systems, Part I, LNCS 6094. Springer, Berlin, Heidelberg.
106. Yanushevskaya, I., M. Tooher, C. Gobl and A. Ni Chasaide. 2007. Time- and amplitude-based voice source correlates of emotional portrayals, pp. 159170. In: A. Paiva, R. Prada and R.W. Picard (eds.), Affective Computing and Intelligent Interaction: Proceedings of the ACII 2007 (Vol. 4738). Springer-Verlag, Lisbon, Portugal.
107. Zajonc, R.B. 1980. Feeling and thinking: Preferences need no inferences. American Psychologist, 35(2):151-175