Categorical and dimensional affect analysis in continuous input: Current trends and future directions

Image and Vision Computing

Volumn 31, Issue 2, 2013, Pages 120-136

  Gunes, Hatice ^a   Schuller, Björn ^b  

^a QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^b TECHNICAL UNIVERSITY OF MUNICH   (Germany)

Author keywords

Automatic affect analysis; Categorical affect description; Continuous input; Dimensional affect description; Multiple modalities; Survey

Indexed keywords

COMPUTER APPLICATIONS; ELECTRICAL ENGINEERING; SURVEYING;

AFFECT ANALYSIS; CATEGORICAL AFFECT DESCRIPTION; CONTINUOUS INPUT; DIMENSIONAL AFFECT DESCRIPTION; MULTIPLE MODALITIES;

SURVEYS;

EID: 84886395193     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2012.06.016     Document Type: Article

References (206)

1. A.B. Ashraf, S. Lucey, J.F. Cohn, T. Chen, Z. Ambadar, K.M. Prkachin, P.E. Solomon, The painful face - pain expression recognition using active appearance models, Image Vision Comput. 27 (12) (2009) 1788-1796.
2. P. Lucey, J.F. Cohn, I. Matthews, S. Lucey, S. Sridharan, J. Howlett, K.M. Prkachin, Automatically detecting pain in video through facial action units, IEEE Trans. Syst. Man Cybern. Part B Cybern. 41 (3) (2011) 664-674.
3. S. Steidl, B. Schuller, D. Seppi, A. Batliner, The hinterland of emotions: facing the open-microphone challenge, Proc. Int. Conf. on Affective Computing and Intelligent Interaction, IEEE, Amsterdam, The Netherlands, 2009, pp. 690-697.
4. M. Pantic, A. Nijholt, A. Pentland, T. Huang, Human-centred intelligent human- computer interaction (hci2): how far are we from attaining it? Int. J. Auton. Adapt. Commun. Syst. (1) (2008) 168-187.
5. M. Schröder, E. Bevacqua, R. Cowie, F. Eyben, H. Gunes, D. Heylen, M. ter Maat, G. McKeown, S. Pammi, M. Pantic, C. Pelachaud, B. Schuller, E. de Sevin, M. Valstar, M. Wöllmer, Building autonomous sensitive artificial listeners, IEEE Trans. Affective Comput. (2012) 1-20.
6. M. Schröder, S. Pammi, H. Gunes, M. Pantic, M. Valstar, R. Cowie, G. McKeown, D. Heylen, M. Maat, F. Eyben, B. Schuller, M. Wöllmer, E. Bevacqua, C. Pelachaud, E. de Sevin, Have an emotional workout with sensitive artificial listeners! Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, p. 646.
7. M. Schröder, E. Bevacqua, F. Eyben, H. Gunes, D. Heylen, M. Maat, S. Pammi, M. Pantic, C. Pelachaud, B. Schuller, E. Sevin, M. Valstar, M. Wöllmer, A demonstration of audiovisual sensitive artificial listeners, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction, vol. 1, 2009, pp. 263-264.
8. H. Gunes, M. Pantic, Automatic, dimensional and continuous emotion recognition, Int. J. Synth. Emotions 1 (1) (2010) 68-99.
9. H. Gunes, M. Pantic, Automatic measurement of affect in dimensional and continuous spaces: why, what, and how? Proc. of Measuring Behavior, 2010, pp. 122-126.
10. H. Gunes, B. Schuller, M. Pantic, R. Cowie, Emotion representation, analysis and synthesis in continuous space: A survey, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 827-834.
11. D. Grandjean, D. Sander, K.R. Scherer, Conscious emotional experience emerges as a function of multilevel, appraisal-driven response synchronization, Conscious. Cogn. 17 (2) (2008) 484-495.
12. P. Ekman, W. Friesen, Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues, Prentice Hall, New Jersey, 1975.
13. S. Baron-Cohen, T. Tead, Mind Reading: The Interactive Guide to Emotion, Jessica Kingsley Publishers Ltd, 2003.
14. T. Pfister, P. Robinson, Real-time recognition of affective states from nonverbal features of speech and its application for public speaking skill analysis, IEEE Trans. Affective Comput. 2 (2) (2011) 66-78.
15. J.A. Russell, A circumplex model of affect, J. Pers. Soc. Psychol. 39 (1980) 1161-1178.
16. A. Mehrabian, Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament, Curr. Psychol. 14 (1996) 261-292.
17. K. Scherer, A. Schorr, T. Johnstone, Appraisal Processes in Emotion: Theory, Methods, Research, Oxford Univ. Press, Oxford/New York, 2001.
18. C. Yu, P.M. Aoki, A.Woodruff, Detecting user engagement in everyday conversations, Proc. of 8th Int. Conf. on Spoken Language Processing, 2004, pp. 1329-1332.
19. Y.-H. Yang, Y.-C. Lin, Y.-F. Su, H.H. Chen, Music Emotion Classification: A Regression Approach, Proc. of IEEE Int. Conf. on Multimedia and Expo, 2007, pp. 208-211.
20. J. Jia, S. Zhang, F. Meng, Y. Wang, L. Cai, Emotional audio-visual speech synthesis based on pad, IEEE Trans. Audio Speech Lang. Process. 19 (3) (2010) 570-582.
21. H. Espinosa, C. Garcia, L. Pineda, Features selection for primitives estimation on emotional speech, Proc. of IEEE Int. Conf. on Acoustics Speech and Signal Processing, 2010, pp. 5138-5141.
22. J.R. Fontaine, K.R. Scherer, E.B. Roesch, P. Ellsworth, The world of emotion is not two-dimensional, Psychol. Sci. 18 (2007) 1050-1057.
23. G. McKeown, M. Valstar, R. Cowie, M. Pantic, The semaine corpus of emotionally coloured character interactions, Proc. of IEEE Int. Conf. Multimedia and Expo, 2010, pp. 1079-1084.
24. R. Dietz, A. Lang, Affective agents: effects of agent affect on arousal, attention, liking and learning, Proc. of Cognitive Technology, 1999.
25. C. Kaernbach, On dimensions in emotion psychology, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 792-796.
26. N.H. Frijda, The Emotions, Cambridge Univ. Press, 1986.
27. D. Sander, D. Grandjean, K.R. Scherer, A systems approach to appraisal mechanisms in emotion, Neural Networks 18 (4) (2005) 317-352. (Pubitemid 40922644)
28. M. Mortillaro, B. Meuleman, K.R. Scherer, Advocating a componential appraisal model to guide emotion recognition, J. Synth. Emotions 3 (1) (2012) 18-32.
29. A. Vinciarelli, M. Pantic, H. Bourlard, Social signal processing: survey of an emerging domain, Image Vision Comput. 27 (12) (2009) 1743-1759.
30. A. Tarasov, S.J. Delany, Benchmarking classification models for emotion recognition in natural speech: A multi-corporal study, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 841-846.
31. B. Schuller, A. Batliner, S. Steidl, D. Seppi, Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge, J. Speech Commun. 53 (9-10) (2011) 1062-1087.
32. G. Chanel, J.J.M. Kierkels, M. Soleymani, T. Pun, Short-term emotion assessment in a recall paradigm, Int. J. Hum. Comput. Stud. 67 (8) (2009) 607-627.
33. G. Berntson, J. Bigger, D. Eckberg, P. Grossman, P. Kaufmann, M. Malik, H. Nagaraja, S. Porges, J. Saul, P. Stone, M. Van der Molen, Heart rate variability:origins, methods, and interpretive caveats, Psychophysiology 34 (6) (1997) 623-648. (Pubitemid 27514918)
34. L. Salahuddin, J. Cho, M.G. Jeong, D. Kim, Ultra short term analysis of heart rate variability for monitoring mental stress in mobile settings, Proc. of the IEEE Int. Conf. of Eng. Med. Biol. Soc., 2007, pp. 39-48.
35. A. Nakasone, H. Prendinger, M. Ishizuka, Emotion recognition from electromyography and skin conductance, Proc. of the 5th Int. Workshop on Biosignal Interpretation, 2005, pp. 219-222.
36. M. Nicolaou, H. Gunes, M. Pantic, Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space, IEEE Trans. Affective Comput. 2 (2) (2011) 92-105.
37. S. Petridis, H. Gunes, S. Kaltwang, M. Pantic, Static vs. dynamic modeling of human nonverbal behavior from multiple cues and modalities, Proc. of ACM Int. Conf. on Multimodal Interfaces, 2009, pp. 23-30.
38. B. Schuller, L. Devillers, Incremental acoustic valence recognition: An inter-corpus perspective on features, matching, and performance in a gating paradigm, Proc. of Interspeech, 2010, pp. 801-804.
39. H. Gunes, M. Piccardi, M. Pantic, Affective computing: focus on emotion expression, synthesis, and recognition, Real World: Affect Recognition using Multiple Cues and Modalities, I-Tech Education and Publishing, 2008, pp. 185-218.
40. Z. Zeng, M. Pantic, G. Roisman, T. Huang, A survey of affect recognition methods: Audio, visual, and spontaneous expressions, IEEE Trans. Pattern Anal. Mach. Intell. 31 (1) (2009) 39-58.
41. G. Chanel, K. Ansari-Asl, T. Pun, Valence-arousal evaluation using physiological signals in an emotion recall paradigm, Proc. of IEEE Int. Conf. on Systems, Man and Cybernetics, 2007, pp. 2662-2667. (Pubitemid 351409969)
42. A. Haag, S. Goronzy, P. Schaich, J. Williams, Emotion recognition using bio-sensors: first steps towards an automatic system, LNCS, 3068, 2004, pp. 36-48.
43. T. Pun, T. Alecu, G. Chanel, J. Kronegg, S. Voloshynovskiy, Brain-computer interaction research at the computer vision and multimedia laboratory, University of Geneva, IEEE Trans. Neural Syst. Rehabil. Eng. 14 (2) (2006) 210-213. (Pubitemid 44150244)
44. R. Davidson, N. Fox, Asymmetrical brain activity discriminates between positive and negative affective stimuli in human infants, Science 218 (1982) 1235-1237. (Pubitemid 13224836)
45. P. Tsiamyrtzis, J. Dowdall, D. Shastri, I. Pavlidis, M. Frank, P. Ekman, Imaging facial physiology for the detection of deceit, Int. J. Comput. Vision 71 (2) (2007) 197-214. (Pubitemid 44359121)
46. M. Kusserow, O. Amft, G. Troster, Bodyant: miniature wireless sensors for naturalistic monitoring of daily activity, Proc. of Int. Conf. on Body Area Networks, 2009, pp. 1-8.
47. Emotiv's epoc, http://www.emotiv.com/.
48. R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey, M. Schröder, Feeltrace: An instrument for recording perceived emotion in real time, Proc. of ISCA Workshop on Speech and Emotion, 2000, pp. 19-24.
49. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, J.G. Taylor, Emotion recognition in human-computer interaction, IEEE Signal Process. Mag. 18 (1) (2001) 33-80.
50. R. Calvo, S. D'Mello, Affect detection: An interdisciplinary review of models, methods, and their applications, IEEE Trans. Affective Comput. 1 (1) (2010) 18-37.
51. G.L. Huttar, Relations between prosodic variables and emotions in normal american english utterances, J. Speech Hear. Res. 11 (1968) 481-487.
52. J. Davitz, The communication of emotional meaning, Auditory Correlates of Vocal Expression of Emotional Feeling, McGraw-Hill, 1964, pp. 101-112.
53. K.R. Scherer, J.S. Oshinsky, Cue utilization in emotion attribution from auditory stimuli, Motiv. Emot. 1 (1977) 331-346.
54. J. Trouvain, W.J. Barry, The prosody of excitement in horse race commentaries, Proc. of ISCA Workshop Speech Emotion, 2000, pp. 86-91.
55. M. Schröder, Speech and emotion research: An overview of research frameworks and a dimensional approach to emotional speech synthesis, Ph.D. dissertation, Univ. of Saarland, Germany, 2003.
56. M. Schröder, D. Heylen, I. Poggi, Perception of non-verbal emotional listener feedback, R. Hoffmann, H. Mixdorff (Eds.), Speech Prosody, 2006, pp. 1-4.
57. M. Wöllmer, B. Schuller, F. Eyben, G. Rigoll, Combining long short-term memory and dynamic bayesian networks for incremental emotion-sensitive artificial listening, IEEE J. Sel. Top. Sign. Proces. 4 (5) (2010) 867-881.
58. B. Schuller, R. Müller, F. Eyben, J. Gast, B. Hörnler, M. Wöllmer, G. Rigoll, A. Höthker, H. Konosu, Being bored? Recognising natural interest by extensive audiovisual integration for real-life application, Image Vision Comput. J. 27 (12) (2009) 1760-1774.
59. A. Batliner, S. Steidl, F. Eyben, B. Schuller, On laughter and speech laugh, based on observations of child-robot interaction, J. Trouvain, N. Campbell (Eds.), The Phonetics of Laughing, Saarland University Press, Saarbrücken, 2011.
60. I. Becker, V. Aharonson, Last but definitely not least: on the role of the last sentence in automatic polarity-classification, Proc. of the ACL 2010 Conference, 2010, pp. 331-335.
61. K. Matsumoto, F. Ren, Estimation of word emotions based on part of speech and positional information, Comput. Hum. Behav. 27 (5) (2011) 1553-1564.
62. F. Benamara, C. Cesarano, A. Picariello, D. Reforgiato, V. Subrahmanian, Sentiment analysis: Adjectives and adverbs are better than adjectives alone, Proc. Int. Conf. on Weblogs and Social Media, 2007, pp. 1-7.
63. V. Subrahmanian, D. Reforgiato, AVA: Adjective-verb-adverb combinations for sentiment analysis, Intell. Syst. 23 (4) (2008) 43-50.
64. M. Pantic, M. Bartlett, Machine analysis of facial expressions, K. Delac, M. Grgic (Eds.), Face Recognition, I-Tech Education and Publishing, Vienna, Austria, 2007, pp. 377-416.
65. N. Dael, M. Mortillaro, K.R. Scherer, The body action and posture coding system (bap): Development and reliability, J. Nonverbal Behav. 36 (2) (2012) 97-121 (Springer).
66. H. Gunes, C. Shan, S. Chen, Y. Tian, Advances in emotion recognition, Bodily Expression for Automatic Affect Recognition, Wiley-Blackwell, 2012, pp. 1-35.
67. C. Darwin, The Expression of the Emotions in Man and Animals, John Murray, London, 1872.
68. H. Wallbott, Bodily expression of emotion, Eur. J. Soc. Psychol. 28 (1998) 879-896.
69. F. Pollick, H. Paterson, A. Bruderlin, A. Sanford, Perceiving affect from arm movement, Cognition 82 (2001) 51-61.
70. M. Coulson, Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence, Nonverbal Behav. 28 (2) (2004) 117-139. (Pubitemid 39015676)
71. A. Kleinsmith, N. Bianchi-Berthouze, Recognizing affective dimensions from body posture, Proc. of the Int. Conf. on Affective Computing and Intelligent Interaction, 2007, pp. 48-58.
72. H. Gunes, M. Piccardi, Automatic temporal segment detection and affect recognition from face and body display, IEEE Trans. Syst. Man Cybern. B Cybern. 39 (1) (2009) 64-84.
73. J. Cohn, L. Reed, T. Moriyama, J. Xiao, K. Schmidt, Z. Ambadar, Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles, Proc. of the IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2004, pp. 129-135.
74. R. Cowie, H. Gunes, G. McKeown, L. Vaclau-Schneider, J. Armstrong, E. Douglas-Cowie, The emotional and communicative significance of head nods and shakes in a naturalistic database, Proc. of LREC Int. Workshop on Emotion, 2010, pp. 42-46.
75. H. Gunes, M. Pantic, Dimensional emotion prediction from spontaneous head gestures for interaction with sensitive artificial listeners, Proc. of Int. Conf. on Intelligent Virtual Agents, 2010, pp. 371-377.
76. A. Kleinsmith, N. Bianchi-Berthouze, Affective body expression perception and recognition: A survey, IEEE Trans. Affective Comput. 99 (2012) 1-20.
77. M. Chen, J.A. Bargh, Consequences of automatic evaluation: immediate behavioral predispositions to approach or avoid the stimulus, Pers. Soc. Psychol. Bull. 25 (2) (1999) 215-224.
78. J. Forster, F. Strack, Influence of overt head movements on memory for valenced words: A case of conceptual-motor compatibility, J. Pers. Soc. Psychol. 71 (1996) 421-430. (Pubitemid 126443609)
79. C. Carver, Pleasure as a sign you can attend to something else: placing positive feelings within a general model of affect, Cogn. Emotion 17 (2) (2003) 241-261. (Pubitemid 36267018)
80. C. Hillman, K. Rosengren, D. Smith, Emotion and motivated behavior: postural adjustments to affective picture viewing, Biol. Psychol. 66 (2004) 51-62. (Pubitemid 38281180)
81. D. Janssen, W.I. Schllhorn, J. Lubienetzki, K. Flling, H. Kokenge, K. Davids, Recognition of emotions in gait patterns by means of artificial neural nets, J. Nonverbal Behav. 32 (2008) 79-92.
82. M. Karg, K. Khnlenz, M. Buss, Recognition of affect based on gait patterns, IEEE Trans. Syst. Man Cybern. B Cybern. 40 (2010) 1050-1061.
83. M. Inderbitzin, A. Väljamäe, J.M.B. Calvo, Expression of emotional states during locomotion based on canonical parameters, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 809-814.
84. M. Wöllmer, A. Metallinou, F. Eyben, B. Schuller, S. Narayanan, Context-sensitive multimodal emotion recognition from speech and facial expression using bidirectional LSTM modeling, Proc. of Interspeech, 2010, pp. 2362-2365.
85. A. Kleinsmith, P.R. De Silva, N. Bianchi-Berthouze, Recognizing emotion from postures: cross-cultural differences in user modeling, Proc. of the Conf. on User Modeling, 2005, pp. 50-59.
86. A. Metallinou, A. Katsamanis, Y. Wang, S. Narayanan, Tracking changes in continuous emotion states using body language and prosodic cues, Proc. of IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 2011, pp. 2288-2291.
87. H.K. Meeren, C.C. Van Heijnsbergen, B. De Gelder, Rapid perceptual integration of facial expression and emotional body language, Proc. Natl. Acad. Sci. U. S. A. 102 (2005) 16518-16523. (Pubitemid 41688946)
88. J. Van den Stock, R. Righart, B. De Gelder, Body expressions influence recognition of emotions in the face and voice, Emotion 7 (3) (2007) 487-494. (Pubitemid 47445349)
89. W. Labov, Language in use, Field Methods of the Project in Linguistic Change and Variation, Prentice-Hall, 1984, pp. 28-53.
90. H. Brugman, A. Russel, Annotating multi-media/multi-modal resources with ELAN, Proc. of Int. Conf. on Language Resources and Evaluation, 2004, pp. 2065-2068.
91. M. Kipp, Anvil - a generic annotation tool for multimodal dialogue, Proc. of the 7th European Conference on Speech Communication and Technology, 2001, pp. 1367-1370.
92. R. Cowie, G. McKeown, E. Douglas-Cowie, Tracing emotion: An overview, J. Synth. Emotions 3 (1) (2012) 1-17.
93. D. Kulic, E.A. Croft, Affective state estimation for human-robot interaction, IEEE Trans. Robot. 23 (5) (2007) 991-1000. (Pubitemid 47577975)
94. B. Schuller, S. Hantke, F. Weninger, W. Han, Z. Zhang, S. Narayanan, Automatic recognition of emotion evoked by general sound events, Proceedings 37th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012, 2012, pp. 341-344.
95. P. Lang, The Cognitive Psychophysiology of Emotion: Anxiety and the Anxiety Disorders, Lawrence Erlbaum, Hillside, NJ, 1985.
96. G. Laurans, P. Desmet, P. Hekkert, The emotion slider: A self-report device for the continuous measurement of emotion, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction Workshops, 2009, pp. 1-6.
97. J. Kessens, M. Neerincx, R. Looije, M. Kroes, G. Bloothooft, Perception of synthetic emotion expressions in speech: categorical and dimensional annotations, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction Workshops, 2009, pp. 1-5.
98. M. Grimm, K. Kroschel, Evaluation of natural emotions using self assessment manikins, Proc. ASRU, 2005, pp. 381-385.
99. M. Grimm, K. Kroschel, S. Narayanan, The Vera am Mittag German audio-visual emotional speech database, Proc. of IEEE Int. Conf. on Multimedia and Expo, 2008, pp. 865-868.
100. J. Cohn, Advances in behavioral science using automated facial image analysis and synthesis, IEEE Signal Process. Mag. (2010) 128-133.
101. J. Healey, Recording affect in the field: Towards methods and metrics for improving ground truth labels, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction, 2011, pp. 107-116.
102. P. Lucey, J.F. Cohn, K.M. Prkachin, P.E. Solomon, I. Matthews, Painful data: The UNBC-McMaster shoulder pain expression archive database, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 57-64.
103. M. Soleymani, J. Lichtenauer, T. Pun, M. Pantic, A multi-modal database for affect recognition and implicit tagging, IEEE Trans. Affective Comput. 3 (1) (2012) 42-55.
104. S. Koelstra, C. Muehl, I. Patras, EEG analysis for implicit tagging of video data, Proc. of the Affective Brain-Computer Interfaces Workshop, 2009, pp. 27-32.
105. S. Koelstra, C. Mühl, M. Soleymani, A. Yazdani, J.-S. Lee, T. Ebrahimi, T. Pun, A. Nijholt, I. Patras, DEAP: A database for emotion analysis using physiological signals, IEEE Trans. Affective Comput. 3 (1) (2012) 18-31.
106. G. McKeown, M. Valstar, R. Cowie, M. Pantic, M. Schröder, The SEMAINE database: Annotated multimodal records of emotionally coloured conversations between a person and a limited agent, IEEE Trans. Affective Comput. 3 (1) (2012) 5-17.
107. I. Sneddon, M. McRorie, G. McKeown, J. Hanratty, The Belfast induced natural emotion database, IEEE Trans. Affective Comput. 3 (1) (2012) 32-41.
108. A. Metallinou, C.-C. Lee, C. Busso, S. Carnicke, S. Narayanan, The USC CreativeIT database: A multimodal database of theatrical improvisation, Proc. of LREC Workshop on Multimodal Corpora, 2010.
109. X. Sun, J. Lichtenauer, M. Valstar, A. Nijholt, M. Pantic, A multimodal database for mimicry analysis, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction, 2011, pp. 367-376.
110. B. Schuller, E. Douglas-Cowie, A. Batliner, Guest editorial: special section on naturalistic affect resources for system building and evaluation, IEEE Trans. Affective Comput. 3 (1) (2012) 3-4.
111. H.P. Espinosa, L.V.P.C.A.R. Garcia, Bilingual acoustic feature selection for emotion estimation using a 3D continuous model, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 786-791.
112. M.C. Sezgin, B. Gnsel, G.K. Kurt, A novel perceptual feature set for audio emotion recognition, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 780-785.
113. J. Snel, C. Cullen, Obtaining speech assets for judgement analysis on low-pass filtered emotional speech, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 835-840.
114. M. Nicolaou, H. Gunes, M. Pantic, Output-associative RVM regression for dimensional and continuous emotion prediction, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 16-23.
115. G. Caridakis, K. Karpouzis, S. Kollias, User and context adaptive neural networks for emotion recognition, Neurocomputer 71 (13-15) (2008) 2553-2562.
116. I. Kanluan, M. Grimm, K. Kroschel, Audio-visual emotion recognition using an emotion recognition space concept, Proc. of the 16th European Signal Processing Conference, 2008.
117. K. Forbes-Riley, D. Litman, Predicting emotion in spoken dialogue from multiple knowledge sources, Proc. of Human Language Technology Conf. North Am. Chapter of the Assoc. Computational Linguistics, 2004, pp. 201-208.
118. K. Karpouzis, G. Caridakis, L. Kessous, N. Amir, A. Raouzaiou, L.Malatesta, S. Kollias, Modeling naturalistic affective states via facial, vocal and bodily expressions recognition, Lecture Notes in Artificial Intelligence, vol. 4451, 2007, pp. 92-116.
119. J. Kim, Robust speech recognition and understanding, Bimodal Emotion Recognition using Speech and Physiological Changes, I-Tech Education and Publishing, 2007, pp. 265-280.
120. M. Nicolaou, H. Gunes, M. Pantic, Audio-visual classification and fusion of spontaneous affective data in likelihood space, Proc. of IEEE Int. Conf. on Pattern Recognition, 2010, pp. 3695-3699.
121. C. Liu, P. Rani, N. Sarkar, An empirical study of machine learning techniques for affect recognition in human-robot interaction, Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2005, pp. 2662-2667.
122. R. Picard, E. Vyzas, J. Healey, Toward machine emotional intelligence: Analysis of affective physiological state, IEEE Trans. Pattern Anal. Mach. Intell. 23 (10) (2001) 1175-1191. (Pubitemid 33047955)
123. Y. Gu, S.-L. Tan, K.-J. Wong, M.-H.R. Ho, L. Qu, Emotion-aware technologies for consumer electronics, Proc. of IEEE Int. Symp. on Consumer Electronics, 2008, pp. 1-4.
124. R. Khosrowabadi, H.C. Quek, A. Wahab, K.K. Ang, EEG-based emotion recognition using self-organizing map for boundary detection, Proc. of Int. Conf. on Pattern Recognition, 2010, pp. 4242-4245.
125. C. Frantzidis, C. Bratsas, M. Klados, E. Konstantinidis, C. Lithari, A. Vivas, C. Papadelis, E. Kaldoudi, C. Pappas, P. Bamidis, On the classification of emotional biosignals evoked while viewing affective pictures: An integrated data-mining-based approach for healthcare applications, IEEE Trans. Inf. Technol. Biomed. 14 (2) (2010) 309-318.
126. D. Heger, F. Putze, T. Schultz, Online recognition of facial actions for natural EEG-based BCI applications, Proc. of ACII 2011 Affective Brain-Computer Interfaces Workshop, 2011, pp. 436-446.
127. B.Z Christian, Anton Nijholt, Affective brain-computer interfaces, Proc. of ACII 2011 Affective Brain-Computer Interfaces Workshop, 2011, p. 435.
128. T.O. Zander, C. Kothe, Towards passive brain-computer interfaces: Applying brain-computer interface technology to human-machine systems in general, J. Neural Eng. 8 (2011) 1-5.
129. B. Schuller, B. Vlasenko, F. Eyben, G. Rigoll, A. Wendemuth, Acoustic emotion recognition: A benchmark comparison of performances, Proc. of IEEE Workshop on Automatic Speech Recognition and Understanding, 2009, pp. 552-557.
130. M. Wöllmer, F. Eyben, S. Reiter, B. Schuller, C. Cox, E. Douglas-Cowie, R. Cowie, Abandoning emotion classes - towards continuous emotion recognition with modelling of long-range dependencies, Proc. of 9th Interspeech Conf., 2008, pp. 597-600.
131. M. Grimm, K. Kroschel, Emotion estimation in speech using a 3D emotion space concept, Proc. of IEEE Automatic Speech Recognition and Understanding Workshop, 2005, pp. 381-385.
132. E. Douglas-Cowie, R. Cowie, I. Sneddon, C. Cox, L. Lowry, M. McRorie, L. Jean-Claude Martin, J.-C. Devillers, A. Abrilian, S. Batliner, A. Noam, K. Karpouzis, The humaine database: Addressing the needs of the affective computing community, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction, 2007, pp. 488-500.
133. A. Reyes, P. Rosso, Linking humour to blogs analysis: Affective traits in posts, Proc. Int. Workshop on Opinion Mining and Sentiment Analysis, 2009, pp. 205-212.
134. C. Strapparava, R. Mihalcea, Annotating and identifying emotions in text, G. Armano, M. de Gemmis, G. Semeraro, E. Vargiu (Eds.), Intelligent Information Access, of Studies in Computational Intelligence, Springer, Berlin, Heidelberg, 2010, pp. 21-38.
135. A. Balahur, J.M. Hermida, A. Montoyo, Detecting emotions in social affective situations using the EmotiNet knowledge base, Proc. International Symposium on Neural Networks IEEE, vol. 3, Guilin, China, 2011, pp. 611-620.
136. B. Schuller, Recognizing affect from linguistic information in 3D continuous space, IEEE Trans. Affect. Comput. 2 (4) (2011) 192-205.
137. A. Batliner, S. Steidl, B. Schuller, D. Seppi, T. Vogt, J. Wagner, L. Devillers, L. Vidrascu, V. Aharonson, L. Kessous, N. Amir, Whodunnit - searching for the most important feature types signalling emotion-related user states in speech, Comp. Speech Lang. 25 (1) (2011) 4-28.
138. F. Eyben, M. Wöllmer, M. Valstar, H. Gunes, B. Schuller, M. Pantic, String-based audiovisual fusion of behavioural events for the assessment of dimensional affect, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 322-329.
139. F. Metze, A. Batliner, F. Eyben, T. Polzehl, B. Schuller, S. Steidl, Emotion recognition using imperfect speech recognition, Proc. Interspeech, Makuhari, Japan, 2010, pp. 478-481.
140. B. Schuller, T. Knaup, Learning and knowledge-based sentiment analysis in movie review key excerpts, A. Esposito, A. Esposito, R. Martone, V. Müller, G. Scarpetta (Eds.), Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces: Theoretical and Practical Issues, vol. 6456/2010, Springer Lecture Notes on Computer Science (LNCS), 2010, pp. 448-472.
141. E. Cambria, A. Hussain, C. Havasi, C. Eckl, Senticspace: visualizing opinions and sentiments in a multi-dimensional vector space, R. Setchi, I. Jordanov, R. Howlett, L. Jain (Eds.), Knowledge-Based and Intelligent Information and Engineering Systems, Vol. 6279 of Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2010, pp. 385-393.
142. R. Poppe, Vision-based human motion analysis: An overview, Comput. Vision and Image Understanding 108 (1-2) (2007) 4-18. (Pubitemid 47333391)
143. R. Poppe, A survey on vision-based human action recognition, Image Vision Comput. 28 (6) (2010) 976-990.
144. D. McDuff, R. El Kaliouby, K. Kassam, R. Picard, Affect valence inference from facial action unit spectrograms, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition Workshops, 2010, pp. 17-24.
145. D. Glowinski, A. Camurri, G. Volpe, N. Dael, K. Scherer, Technique for automatic emotion recognition by body gesture analysis, Proc. of Computer Vision and Pattern Recognition Workshops, 2008, pp. 1-6.
146. S. Ioannou, A. Raouzaiou, V. Tzouvaras, T. Mailis, K. Karpouzis, S. Kollias, Emotion recognition through facial expression analysis based on a neurofuzzy method, J. Neural Networks 18 (2005) 423-435. (Pubitemid 40922649)
147. S. Arifin, P. Cheung, Affective level video segmentation by utilizing the pleasure- arousal-dominance information, IEEE Trans. Multimedia 10 (7) (2008) 1325-1341.
148. M. Nicolaou, H. Gunes, M. Pantic, Automatic segmentation of spontaneous data using dimensional labels from multiple coders, Proc. of LREC Int. Workshop on Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality, 2010, pp. 43-48.
149. M. Kipp, J.-C. Martin, Gesture and emotion: can basic gestural form features discriminate emotions? Proc. of Int. Conf. on Affective Computing and Intelligent Interaction Workshops, 2009, pp. 1-8.
150. Y. Yoshitomi, S.I. Kim, T. Kawano, T. Kitazoe, Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face, IEEE Int. Workshop on Robot and Human Interactive Communication, 2000, pp. 178-183.
151. M.M. Khan, R.D. Ward, M. Ingleby, Infrared thermal sensing of positive and negative affective states, Proc. of the IEEE Int. Conf. on Robotics, Automation and Mechatronics, 2006, pp. 1-6.
152. B. Nhan, T. Chau, Classifying affective states using thermal infrared imaging of the human face, IEEE Trans. Biomed. Eng. 57 (4) (2010) 979-987.
153. A. Merla, G. Romani, Thermal signatures of emotional arousal: A functional infrared imaging study, Proc. of Int. Conf. of the IEEE Engineering in Medicine and Biology Society, 2007, pp. 247-249.
154. S. Gilroy, M. Cavazza, M. Niiranen, E. Andre, T. Vogt, J. Urbain, M. Benayoun, H. Seichter, M. Billinghurst, Pad-based multimodal affective fusion, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction Workshops, 2009, pp. 1-8.
155. G. Caridakis, L. Malatesta, L. Kessous, N. Amir, A. Raouzaiou, K. Karpouzis, Modeling naturalistic affective states via facial and vocal expressions recognition, Proc. of ACM Int. Conf. on Multimodal Interfaces, 2006, pp. 146-154. (Pubitemid 47113445)
156. A.M. Oliveira, M.P. Teixeira, I.B. Fonseca, M. Oliveira, Joint model-parameter validation of self-estimates of valence and arousal: probing a differential-weighting model of affective intensity, Proc. of the 22nd Annual Meeting of the Int. Society for Psychophysics, 2006, pp. 245-250.
157. N. Alvarado, Arousal and valence in the direct scaling of emotional response to film clips, Motiv. Emot. 21 (4) (1997) 323-348.
158. B. Schuller, M. Valstar, R. Cowie, M. Pantic, Avec 2011 - the first audio/visual emotion challenge and workshop - an introduction, Proc. of 1st Int'l. Audio- Visual Emotion Challenge and Workshop, 2011, pp. 415-424.
159. M. Nicolaou, H. Gunes, M. Pantic, Designing frameworks for automatic affect prediction and classification in dimensional space, Proc. of IEEE CVPR Workshop on Gesture Recognition, 2011, pp. 20-26.
160. J. Kim, E. Andre, Emotion recognition based on physiological changes in music listening, IEEE Trans. Pattern Anal. Mach. Intell. 30 (12) (2008) 2067-2083.
161. M. Nicolaou, H. Gunes, M. Pantic, A multi-layer hybrid framework for dimensional emotion classification, Proc. of ACM Multimedia, 2011, pp. 933-936.
162. S. Bermejo, J. Cabestany, Oriented principal component analysis for large margin classifiers, Neural Networks 14 (10) (2001) 1447-1461. (Pubitemid 33101880)
163. M. Schröder, The SEMAINE API: Towards a standards-based framework for building emotion-oriented systems, Adv. Hum. Mach. Interact. 2010 (2010) 1-21.
164. F. Eyben, M. Wöllmer, B. Schuller, Open SMILE - the munich versatile and fast open-source audio feature extractor, Proc. of ACM Multimedia, 2010, pp. 1459-1462.
165. T. Taleb, D. Bottazzi, N. Nasser, A novel middleware solution to improve ubiquitous healthcare systems aided by affective information, IEEE Trans. Inf. Technol. Biomed. 14 (2) (2010) 335-349.
166. G. Littlewort, J. Whitehill, T. Wu, I.R. Fasel, M.G. Frank, J.R. Movellan, M.S. Bartlett, The computer expression recognition toolbox (cert), Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 298-305.
167. D. Glowinski, N. Dael, A. Camurri, G. Volpe, M. Mortillaro, K. Scherer, Towards a minimal representation of affective gestures, IEEE Trans. Affective Comput. 2 (2) (2011) 106-118.
168. Humaine: http://emotion-research.net.
169. T. Huang, M. Hasegawa-Johnson, S. Chu, Z. Zeng, H. Tang, Sensitive talking heads, IEEE Signal Process. Mag. 26 (4) (2009) 67-72.
170. X. Shen, X. Fu, Y. Xuan, Do different emotional valences have same effects on spatial attention? Proc. of Int. Conf. on Natural Computation, vol. 4, 2010, pp. 1989-1993.
171. K.C. Wassermann, K. Eng, P.F.M.J. Verschure, Live soundscape composition based on synthetic emotions, IEEE Multi Media 10 (4) (2003) 82-90.
172. A. Beck, L. Canamero, K. Bard, Towards an affect space for robots to display emotional body language, Proc. of IEEE Int. Symp. in Robot and Human Interactive Communication, 2010, pp. 464-469.
173. J. Sanghvi, G. Castellano, I. Leite, A. Pereira, P.W. McOwan, A. Paiva, Automatic analysis of affective postures and body motion to detect engagement with a game companion, Proc. of ACM/IEEE Int. Conf. on Human-Robot Interaction, 2011, pp. 305-311.
174. M. Karg, M. Schwimmbeck, K. Kühnlenz, M. Buss, Towards mapping emotive gait patterns from human to robot, Proc. of IEEE Int. Symp. in Robot and Human Interactive Communication, 2010, pp. 258-263.
175. M. Mihelj, D. Novak, M. Munih, Emotion-aware system for upper extremity rehabilitation, Proc. of Int. Conf. on Virtual Rehabilitation, 2009, pp. 160-165.
176. T.-C. Tsai, J.-J. Chen, W.-C. Lo, Design and implementation of mobile personal emotion monitoring system, Proc. of Int. Conf. on Mobile Data Management: Systems, Services and Middleware, 2009, pp. 430-435.
177. B. Grundlehner, L. Brown, J. Penders, B. Gyselinckx, The design and analysis of a real-time, continuous arousal monitor, Proc. of Int. Workshop on Wearable and Implantable Body Sensor Networks, 2009, pp. 156-161.
178. R. Picard, Future affective technology for autism and emotion communication, Philos. Trans. R. Soc. Lond. B Biol. Sci. 364 (1535) (2009) 3575-3584.
179. G. Littlewort, M.S. Bartlett, L.P. Salamanca, J. Reilly, Automated measurement of children's facial expressions during problem solving tasks, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 30-35.
180. J. Whitehill, Z. Serpell, A. Foster, Y. Lin, B. Pearson, M.S. Barlett, J.R. Movellan, Toward an optimal affect-sensitive instructional system of cognitive skills, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition Workshop on Human Communicative Behavior, 2011, pp. 20-25.
181. U. Faghihi, P. Fournier-Viger, R. Nkambou, P. Poirier, A. Mayers, How emotional mechanism helps episodic learning in a cognitive agent, Proc. of IEEE Symp. on Intelligent Agents, 2009, pp. 23-30.
182. F. Eyben, M. Wöllmer, T. Poitschke, B. Schuller, C. Blaschke, B. Färber, N. Nguyen-Thien, Emotion on the road - necessity, acceptance, and feasibility of affective computing in the car, Adv. Hum. Mach. Interact. 2010 (2010) 1-17.
183. K. Sun, J. Yu, Y. Huang, X. Hu, An improved valence-arousal emotion space for video affective content representation and recognition, Proc. of IEEE Int. Conf. on Multimedia and Expo, 2009, pp. 566-569.
184. J. Kierkels, M. Soleymani, T. Pun, Queries and tags in affect-based multimedia retrieval, Proc. of IEEE Int. Conf. on Multimedia and Expo, 2009, pp. 1436-1439.
185. M. Soleymani, J. Davis, T. Pun, A collaborative personalized affective video retrieval system, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction Workshops, 2009, pp. 1-2.
186. M. Soleymani, S. Koelstra, I. Patras, T. Pun, Continuous emotion detection in response to music videos, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 803-808.
187. M. Rehm, M. Wissner, Gamble-a multiuser game with an embodied conversational agent, Lecture Notes in Computer Science, vol. 3711, 2005, pp. 180-191.
188. A. Kleinsmith, N. Bianchi-Berthouze, A. Steed, Automatic recognition of non-acted affective postures, IEEE Trans. Syst. Man Cybern. B Cybern. 41 (2011) 1027-1038.
189. Affectiva's homepage: http://www.affectiva.com/.
190. B. Schuller, S. Steidl, A. Batliner, The interspeech 2009 emotion challenge, Proceedings Interspeech 2009, 10th Annual Conference of the International Speech Communication Association, 2009, pp. 312-315.
191. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Muller, S. Narayanan, The interspeech 2010 paralinguistic challenge, Proc. of Interspeech, 2010, pp. 2794-2797.
192. J. Gratch, IEEE Trans. Affective Comput. 1 (1) (2010) 1-10.
193. A.A. Salah, T. Gevers, A. Vinciarelli, Introduction to the affect-based human behavior understanding special issue, IEEE Trans. Affective Comput. 2 (2) (2011) 64-65.
194. B. Schuller, S. Steidl, A. Batliner, Introduction to the special issue on sensing emotion and affect - facing realism in speech processing, Speech Commun. 53 (9/10) (2011) 1059-1061.
195. J. Epps, R. Cowie, S. Narayanan, B. Schuller, J. Tao, Editorial emotion and mental state recognition from speech, EURASIP J. Adv. Signal Process. 2012 (15) (2012) 1-2.
196. E. Hudlicka, H. Gunes, Benefits and limitations of continuous representations of emotions in affective computing: introduction to the special issue, J. Synth. Emotions 3 (1) (2012).
197. D. Gokcay, G. Yildirim, Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives, IGI Global, 2011.
198. K.R. Scherer, T. B-nziger, E. Roesch, A Blueprint for Affective Computing, Oxford University Press, 2011.
199. A. Konar, A. Chakraborty, Advances in Emotion Recognition, Wiley-Blackwell, 2012.
200. M.E. Hoque, R.W. Picard, Acted vs. natural frustration and delight: many people smile in natural frustration, Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011, pp. 354-359.
201. B. Schuller, Z. Zhang, F. Weninger, G. Rigoll, Using multiple databases for training in emotion recognition: To unite or to vote? Proc. Interspeech, 2011, pp. 1553-1556.
202. Z. Zhang, F. Weninger, M. Wöllmer, B. Schuller, Unsupervised learning in cross-corpus acoustic emotion recognition, Proc. of IEEE Workshop on Automatic Speech Recognition and Understanding, 2011, pp. 523-528.
203. B. Schuller, F. Burkhardt, Learning with synthesized speech for automatic emotion recognition, Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 2010, pp. 5150-5153.
204. X. Sun, A. Nijholt, K.P. Truong, M. Pantic, Automatic understanding of affective and social signals by multimodal mimicry recognition, Proc. of Int. Conf. on Affective Computing and Intelligent Interaction, 2011, pp. 289-296.
205. P. Wang, Q. Ji, Performance modeling and prediction of face recognition systems, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, 2006, pp. 1566-1573.
206. D. Heylen, M. Theune, R. Akker, A. Nijholt, Social agents: The first generations, Proc. of Affective Computing and Intelligent Interaction Workshops, 2009, pp. 1-7.