Affect detection: An interdisciplinary review of models, methods, and their applications

IEEE Transactions on Affective Computing

Volumn 1, Issue 1, 2010, Pages 18-37

  Calvo, Rafael A ^a   D'Mello, Sidney ^b  

^a UNIVERSITY OF SYDNEY   (Australia)

^b UNIVERSITY OF MEMPHIS   (United States)

Author keywords

affect sensing and analysis; Affective computing; emotion detection; emotion theory; multimodal recognition

Indexed keywords

AFFECT SENSING AND ANALYSIS; AFFECTIVE COMPUTING; EMOTION DETECTION; EMOTION THEORY; MULTIMODAL RECOGNITION;

ELECTROPHYSIOLOGY; SURVEYS;

RESEARCH;

EID: 79953822842     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/T-AFFC.2010.1     Document Type: Article

References (210)

1. S. D'Mello, S. Craig, B. Gholson, S. Franklin, R. Picard, and A. Graesser, "Integrating Affect Sensors in an Intelligent Tutoring System," Proc. Computer in the Affective Loop Workshop at 2005 Int'l Conf. Intelligent User Interfaces, pp. 7-13, 2005.
2. C. Darwin, Expression of the Emotions in Man and Animals. Oxford Univ. Press, Inc., 2002.
3. C. Darwin, The Expression of the Emotions in Man and Animals. John Murray, 1872.
4. W. James, "What Is an Emotion?" Mind, vol. 9, pp. 188-205, 1884.
5. S. Turkle, The Second Self: Computers and the Human Spirit. Simon & Schuster, 1984.
6. K.R. Scherer, "Studying the Emotion-Antecedent Appraisal Process: An Expert System Approach," Cognition and Emotion, vol. 7, pp. 325-355, 1993.
7. R.W. Picard, Affective Computing. The MIT Press, 1997.
8. K. Boehner, R. DePaula, P. Dourish, and P. Sengers, "Affect: From Information to Interaction," Proc. Fourth Decennial Conf. Critical Computing: Between Sense and Sensibility, pp. 59-68, 2005.
9. J. Tao and T. Tan, "Affective Computing: A Review," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, J. Tao, T. Tan, and R.W. Picard, eds., pp. 981-995, 2005. (Pubitemid 43763427)
10. M. Pantic and L. Rothkrantz, "Toward an Affect-Sensitive Multimodal Human-Computer Interaction," Proc. IEEE, vol. 91, no. 9, pp. 1370-1390, Sept. 2003. (Pubitemid 40890819)
11. N. Sebe, I. Cohen, and T.S. Huang, "Multimodal Emotion Recognition," Handbook of Pattern Recognition and Computer Vision, World Scientific, 2005.
12. Z. Zeng, M. Pantic, G.I. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
13. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. Taylor, "Emotion Recognition in Human- Computer Interaction," IEEE Signal Processing Magazine, vol. 18, no. 1, pp. 32-80, 2001. (Pubitemid 32287669)
14. B. Pang and L. Lee, "Opinion Mining and Sentiment Analysis," Foundations and Trends in Information Retrieval, vol. 2, pp. 1-135, 2008.
15. J. Gratch and S. Marsella, Social Emotions in Nature and Artifact: Emotions in Human and Human-Computer Interaction. Oxford Univ. Press, to be published.
16. K. Scherer, T. Bänziger, and R. EB, Blueprint for Affective Computing: A Source Book. Oxford Univ. Press, to be published.
17. D. Gkay and G. Yildirim, Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives. Information Science Publishing, 2010.
18. New Perspectives on Affect and Learning Technologies, R.A. Calvo and S. D'Mello, eds., Springer, to be published.
19. J.A. Russell, "Core Affect and the Psychological Construction of Emotion," Psychological Rev., vol. 110, pp. 145-172, 2003. (Pubitemid 37170695)
20. J.A. Russell, J.A. Bachorowski, and J.M. Fernandez-Dols, "Facial and Vocal Expressions of Emotion," Ann. Rev. of Psychology, vol. 54, pp. 329-349, 2003. (Pubitemid 37417497)
21. L. Barrett, "Are Emotions Natural Kinds?" Perspectives on Psychological Science, vol. 1, pp. 28-58, 2006.
22. L. Barrett, B. Mesquita, K. Ochsner, and J. Gross, "The Experience of Emotion," Ann. Rev. of Psychology, vol. 58, pp. 373-403, 2007.
23. T. Dalgleish, B. Dunn, and D. Mobbs, "Affective Neuroscience: Past, Present, and Future," Emotion Rev., vol. 1, pp. 355-368, 2009.
24. T. Dalgleish and M. Power, Handbook of Cognition and Emotion. John Wiley & Sons, Ltd., 1999.
25. M. Lewis, J. Haviland-Jones, and L. Barrett, Handbook of Emotions, third ed. Guilford Press, 2008.
26. R.J. Davidson, K.R. Scherer, and H.H. Goldsmith, Handbook of Affective Sciences. Oxford Univ. Press, 2003.
27. A. Ortony and T. Turner, "What's Basic about Basic Emotions," Psychological Rev., vol. 97, pp. 315-331, July 1990.
28. J. Russell, "Is There Universal Recognition of Emotion from Facial Expression?: A Review of the Cross-Cultural Studies," Psychological Bull., vol. 115, pp. 102-141, 1994. (Pubitemid 24976739)
29. P. Ekman, Universals and Cultural Differences in Facial Expressions of Emotion. Univ. of Nebraska Press, 1971.
30. P. Ekman and W.V. Friesen, Unmasking the Face. Malor Books, 2003.
31. S.S. Tomkins, Affect Imagery Consciousness: Volume I, The Positive Affects. Tavistock, 1962.
32. C. Izard, The Face of Emotion. Appleton-Century-Crofts, 1971.
33. C. Izard, "Innate and Universal Facial Expressions: Evidence from Developmental and Cross-Cultural Research," Psychological Bull., vol. 115, pp. 288-299, 1994. (Pubitemid 24976748)
34. N. Frijda, "Emotion, Cognitive Structure, and Action Tendency," Cognition and Emotion, vol. 1, pp. 115-143, 1987.
35. I. Arroyo, D.G. Cooper, W. Burleson, B.P. Woolf, K. Muldner, and R. Christopherson, "Emotion Sensors Go to School," Proc. 14th Conf. Artificial Intelligence in Education, pp. 17-24, 2009.
36. M.B. Arnold, Emotion and Personality Vol. 1 & 2. Columbia Univ. Press, 1960.
37. C. Smith and P. Ellsworth, "Patterns of Cognitive Appraisal in Emotion," J. Personality and Social Psychology, vol. 48, pp. 813-838, 1985.
38. K. Scherer, A. Schorr, and T. Johnstone, Appraisal Processes in Emotion: Theory, Methods, Research. Oxford Univ. Press, 2001.
39. I.J. Roseman, M.S. Spindel, and J.P.E., "Appraisals of Emotion Eliciting Events: Testing a Theory of Discrete Emotions," J. Personality and Social Psychology, vol. 59, pp. 899-915, 1990.
40. S. Schachter and J.E. Singer, "Cognitive, Social, and Physiological Determinants of Emotional State," Psychological Rev., vol. 69, pp. 379-399, 1962.
41. R. Lazarus, Emotion and Adaptation. Oxford Univ. Press, 1991.
42. I.J. Roseman, "Cognitive Determinants of Emotion," Emotions, Relationships and Health, pp. 11-36, Beverly Hills Publishing, 1984.
43. A. Ortony, G. Clore, and A. Collins, The Cognitive Structure of Emotions. Cambridge Univ. Press, 1988.
44. J. Ledoux, The Emotional Brain: The Mysterious Underpinnings of Emotional Life. Simon & Schuster, 1998.
45. R. Zajonc, "On the Primacy of Affect," Am. Psychologist, vol. 39, pp. 117-123, 1984.
46. R. Zajonc, "Feeling and Thinking: Preferences Need No Inferences," Am. Psychologist, vol. 33, pp. 151-175, 1980.
47. C. Conati, "Probabilistic Assessment of User's Emotions during the Interaction with Educational Games," J. Applied Artificial Intelligence, special issue on merging cognition and affect in HCI, vol. 16, pp. 555-575, 2002. (Pubitemid 35019458)
48. J.R. Averill, "A Constructivist View of Emotion," Emotion: Theory, Research and Experience, pp. 305-339, Academic Press, 1980.
49. A. Glenberg, D. Havas, R. Becker, and M. Rinck, "Grounding Language in Bodily States: The Case for Emotion," The Grounding of Cognition: The Role of Perception and Action in Memory, Language, and Thinking, R.A.Z.D. Pecher, ed., pp. 115-128, Cambridge Univ. Press, 2005.
50. J. Stets and J. Turner, "The Sociology of Emotions," Handbook of Emotions, M. Lewis, J. Haviland-Jones, and L. Barrett, eds., third ed., pp. 32-46, The Guilford Press, 2008.
51. P. Salovey, "Introduction: Emotion and Social Processes," Handbook of Affective Sciences, R.J. Davidson, K.R. Scherer, and H.H. Goldsmith, eds., Oxford Univ. Press, 2003.
52. J.Y. Chiao, T. Iidaka, H.L. Gordon, J. Nogawa, M. Bar, E. Aminoff, N. Sadato, and N. Ambady, "Cultural Specificity in Amygdala Response to Fear Faces," J. Cognitive Neuroscience, vol. 20, pp. 2167- 2174, Dec. 2008.
53. G. Peterson, "Cultural Theory and Emotions," Handbook of the Sociology of Emotions, J. Stets and J. Turner, eds., pp. 114-134, Springer, 2006.
54. D. Matsumoto, "Cultural Similarities and Differences in Display Rules," Motivation and Emotion, vol. 14, pp. 195-214, 1990.
55. P. Ekman and W. Friesen, Unmasking the Face: A Guide to Recognizing Emotions from Facial Expressions. Prentice-Hall, 1975.
56. T.D. Kemper, "Predicting Emotions from Social-Relations," Social Psychology Quarterly, vol. 54, pp. 330-342, Dec. 1991.
57. J. Panksepp, Affective Neuroscience: The Foundations of Human and Animal Emotions. Oxford Univ. Press, 2004.
58. A. Damasio, Looking for Spinoza: Joy, Sorrow, and the Feeling Brain. Harcourt, Inc., 2003.
59. A. Ohman and J.J.F. Soares, "Emotional Conditioning to Masked Stimuli: Expectancies for Aversive Outcomes Following Nonrecognized Fear-Relevant Stimuli," J. Experimental Psychology: General, vol. 127, pp. 69-82, 1998. (Pubitemid 128460252)
60. M.G. Calvo and L. Nummenmaa, "Processing of Unattended Emotional Visual Scenes," J. Experimental Psychology: General, vol. 136, pp. 347-369, 2007.
61. M.E. Dawson, A.J. Rissling, A.M. Schell, and R. Wilcox, "Under What Conditions Can Human Affective Conditioning Occur without Contingency Awareness?: Test of the Evaluative Conditioning Paradigm," Emotion, vol. 7, pp. 755-766, 2007.
62. R.J. Davidson, P. Ekman, C.D. Saron, J.A. Senulis, and W.V. Friesen, "Approach-Withdrawal and Cerebral Asymmetry: Emotional Expression and Brain Physiology I," J. Personality and Social Psychology, vol. 58, pp. 330-341, 1990.
63. P.C. Petrantonakis and L.J. Hadjileontiadis, "Emotion Recognition from EEG Using Higher Order Crossings," IEEE Trans. Information Technology in Biomedicine, vol. 14, no. 2, pp. 186-194, Mar. 2010.
64. M.D. Lewis, "Bridging Emotion Theory and Neurobiology through Dynamic Systems Modeling," Behavioral and Brain Sciences, vol. 28, pp. 169-194, 2005. (Pubitemid 41309470)
65. J. Coan, "Emergent Ghosts of the Emotion Machine," Emotion Rev., to appear.
66. R.A. Calvo, "Latent and Emergent Models in Affective Computing," Emotion Rev., to appear.
67. P. Ekman, "An Argument for Basic Emotions," Cognition and Emotion, vol. 6, pp. 169-200, 1992.
68. P. Ekman, "Expression and the Nature of Emotion," Approaches to Emotion, K. Scherer and P. Ekman, eds., pp. 319-344, Erlbaum, 1984.
69. P. Ekman and W. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement: Investigator's Guide 2 Parts. Consulting Psychologists Press, 1978.
70. G. Donato, M.S. Bartlett, J.C. Hager, P. Ekman, and T.J. Sejnowski, "Classifying Facial Actions," IEEE Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 974-989, Oct. 1999. (Pubitemid 129304131)
71. A. Asthana, J. Saragih, M. Wagner, and R. Goecke, "Evaluating AAM Fitting Methods for Facial Expression Recognition," Proc. 2009 Int'l Conf. Affective Computing and Intelligent Interaction, 2009.
72. T. Brick, M. Hunter, and J. Cohn, "Get the FACS Fast: Automated FACS Face Analysis Benefits from the Addition of Velocity," Proc. 2009 Int'l Conf. Affective Computing and Intelligent Interaction, 2009.
73. M.E. Hoque, R. el Kaliouby, and R.W. Picard, "When Human Coders (and Machines) Disagree on the Meaning of Facial Affect in Spontaneous Videos," Proc. Ninth Int'l Conf. Intelligent Virtual Agents, 2009.
74. R. El Kaliouby and P. Robinson, "Generalization of a Vision-Based Computational Model of Mind-Reading," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, pp. 582-589, 2005. (Pubitemid 43763378)
75. R. El Kaliouby and P. Robinson, "Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures," Proc. Int'l Conf. Computer Vision and Pattern Recognition, vol. 3, p. 154, 2004.
76. B. McDaniel, S. D'Mello, B. King, P. Chipman, K. Tapp, and A. Graesser, "Facial Features for Affective State Detection in Learning Environments," Proc. 29th Ann. Meeting of the Cognitive Science Soc., 2007.
77. H. Aviezer, R. Hassin, J. Ryan, C. Grady, J. Susskind, A. Anderson, M. Moscovitch, and S. Bentin, "Angry, Disgusted, or Afraid? Studies on the Malleability of Emotion Perception," Psychological Science, vol. 19, pp. 724-732, 2008.
78. M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Fully Automatic Facial Action Recognition in Spontaneous Behaviour," Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 223-230, 2006. (Pubitemid 44711354)
79. M. Pantic and I. Patras, "Dynamics of Facial Expression: Recognition of Facial Actions and Their Temporal Segments from Face Profile Image Sequences," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 36, no. 2, pp. 433-449, Apr. 2006.
80. H. Gunes and M. Piccardi, "Bi-Modal Emotion Recognition from Expressive Face and Body Gestures," J. Network and Computer Applications, vol. 30, pp. 1334-1345, 2007. (Pubitemid 47263518)
81. P.N. Juslin and K.R. Scherer, "Vocal Expression of Affect," The New Handbook of Methods in Nonverbal Behavior Research, Oxford Univ. Press, 2005.
82. T. Johnstone and K. Scherer, "Vocal Communication of Emotion," Handbook of Emotions, pp. 220-235, Guilford Press, 2000.
83. R. Banse and K.R. Scherer, "Acoustic profiles in Vocal Emotion Expression," J. Personality and Social Psychology, vol. 70, pp. 614- 636, 1996. (Pubitemid 126420699)
84. C.M. Lee and S.S. Narayanan, "Toward Detecting Emotions in Spoken Dialogs," IEEE Trans. Speech and Audio Processing, vol. 13, no. 2, pp. 293-303, Mar. 2005. (Pubitemid 40320247)
85. L. Devillers, L. Vidrascu, and L. Lamel, "Challenges in Real-Life Emotion Annotation and Machine Learning Based Detection," Neural Networks, vol. 18, pp. 407-422, 2005. (Pubitemid 40922648)
86. L. Devillers and L. Vidrascu, "Real-Life Emotions Detection with Lexical and Paralinguistic Cues on Human-Human Call Center Dialogs," Proc. Ninth Int'l Conf. Spoken Language Processing, 2006.
87. B. Schuller, J. Stadermann, and G. Rigoll, "Affect-Robust Speech Recognition by Dynamic Emotional Adaptation," Proc. Speech Prosody, 2006.
88. D. Litman and K. Forbes-Riley, "Predicting Student Emotions in Computer-Human Tutoring Dialogues," Proc. 42nd Ann. Meeting on Assoc. for Computational Linguistics, 2004.
89. B. Schuller, R.J. Villar, G. Rigoll, and M. Lang, "Meta-Classifiers in Acoustic and Linguistic Feature Fusion-Based Affect Recognition," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 2005.
90. R. Fernandez and R.W. Picard, "Modeling Drivers' Speech under Stress," Speech Comm., vol. 40, pp. 145-159, 2003.
91. P. Bull, Posture and Gesture. Pergamon Press, 1987.
92. M. Demeijer, "The Contribution of General Features of Body Movement to the Attribution of Emotions," J. Nonverbal Behavior, vol. 13, pp. 247-268, 1989.
93. A. Mehrabian, Nonverbal Communication. Aldine-Atherton, 1972.
94. N. Bernstein, The Co-Ordination and Regulation of Movement. Pergamon Press, 1967.
95. M. Coulson, "Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence," J. Nonverbal Behavior, vol. 28, pp. 117-139, 2004. (Pubitemid 39015676)
96. J. Montepare, E. Koff, D. Zaitchik, and M. Albert, "The Use of Body Movements and Gestures as Cues to Emotions in Younger and Older Adults," J. Nonverbal Behavior, vol. 23, pp. 133-152, 1999.
97. R. Walk and K. Walters, "Perception of the Smile and Other Emotions of the Body and Face at Different Distances," Bull. Psychonomic Soc., vol. 26, p. 510, 1988.
98. P. Ekman and W. Friesen, "Nonverbal Leakage and Clues to Deception," Psychiatry, vol. 32, pp. 88-106, 1969.
99. S. Mota and R. Picard, "Automated Posture Analysis for Detecting Learner's Interest Level," Proc. Computer Vision and Pattern Recognition Workshop, vol. 5, p. 49, 2003.
100. S. D'Mello and A. Graesser, "Automatic Detection of Learner's Affect from Gross Body Language," Applied Artificial Intelligence, vol. 23, pp. 123-150, 2009.
101. N. Bianchi-Berthouze and C. Lisetti, "Modeling Multimodal Expression of User's Affective Subjective Experience," User Modeling and User-Adapted Interaction, vol. 12, pp. 49-84, Feb. 2002. (Pubitemid 34109558)
102. G. Castellano, M. Mortillaro, A. Camurri, G. Volpe, and K. Scherer, "Automated Analysis of Body Movement in Emotionally Expressive Piano Performances," Music Perception, vol. 26, pp. 103- 119, Dec. 2008.
103. J.P. Pinel, Biopsychology. Allyn and Bacon, 2004.
104. J.L. Andreassi, Human Behaviour and Physiological Response. Taylor & Francis, 2007.
105. R. Yerkes and J. Dodson, "The Relation of Strength of Stimulus to Rapidity of Habit-Formation," J. Comparative Neurology and Psychology, vol. 18, pp. 459-482, 1908.
106. R. Malmo, "Activation," Experimental Foundations of Clinical Psychology, pp. 386-422, Basic Books, 1962.
107. E. Duffy, "Activation," Handbook of Psychophysiology, pp. 577-622, Holt, Rinehart and Winston, 1972.
108. P. Ekman, R. Levenson, and W. Friesen, "Autonomic Nervous System Activity Distinguishes among Emotions," Science, vol. 221, pp. 1208-1210, 1983. (Pubitemid 13001240)
109. R. Croyle and J. Cooper, "Dissonance Arousal: Physiological Evidence," J. Personality and Social Psychology, vol. 45, pp. 782-791, 1983.
110. F. Nasoz, K. Alvarez, C.L. Lisetti, and N. Finkelstein, "Emotion Recognition from Physiological Signals Using Wireless Sensors for Presence Technologies," Cognition, Technology and Work, vol. 6, pp. 4-14, 2004. (Pubitemid 38305312)
111. O. Villon and C. Lisetti, "A User-Modeling Approach to Build User's Psycho-Physiological Maps of Emotions Using Bio- Sensors," Proc. IEEE RO-MAN 2006, 15th IEEE Int'l Symp. Robot and Human Interactive Comm., Session Emotional Cues in Human- Robot Interaction, pp. 269-276, 2006.
112. R.W. Picard, E. Vyzas, and J. Healey, "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175-1191, Oct. 2001. (Pubitemid 33047955)
113. J. Wagner, N.J. Kim, and E. Andre, "From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification," Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 940-943, 2005. (Pubitemid 44669023)
114. K. Kim, S. Bang, and S. Kim, "Emotion Recognition System Using Short-Term Monitoring of Physiological Signals," Medical and Biological Eng. and Computing, vol. 42, pp. 419-427, May 2004. (Pubitemid 38786706)
115. R.A. Calvo, I. Brown, and S. Scheding, "Effect of Experimental Factors on the Recognition of Affective Mental States through Physiological Measures," Proc. 22nd Australasian Joint Conf. Artificial Intelligence, 2009.
116. J.N. Bailenson, E.D. Pontikakis, I.B. Mauss, J.J. Gross, M.E. Jabon, C.A.C. Hutcherson, C. Nass, and O. John, "Real-Time Classification of Evoked Emotions Using Facial Feature Tracking and Physiological Responses," Int'l J. Human-Computer Studies, vol. 66, pp. 303-317, 2008.
117. C. Liu, K. Conn, N. Sarkar, and W. Stone, "Physiology-Based Affect Recognition for Computer-Assisted Intervention of Children with Autism Spectrum Disorder," Int'l J. Human-Computer Studies, vol. 66, pp. 662-677, 2008.
118. E. Vyzas and R.W. Picard, "Affective Pattern Classification," Proc. AAAI Fall Symp. Series: Emotional and Intelligent: The Tangled Knot of Cognition, pp. 176-182, 1998.
119. A. Haag, S. Goronzy, P. Schaich, and J. Williams, "Emotion Recognition Using Bio-Sensors: First Steps towards an Automatic System," Affective Dialogue Systems. pp. 36-48, Springer, 2004. (Pubitemid 38851666)
120. O. AlZoubi, R.A. Calvo, and R.H. Stevens, "Classification of EEG for Emotion Recognition: An Adaptive Approach," Proc. 22nd Australasian Joint Conf. Artificial Intelligence, pp. 52-61, 2009.
121. A. Heraz and C. Frasson, "Predicting the Three Major Dimensions of the Learner's Emotions from Brainwaves," World Academy of Science, Eng. and Technology, vol. 25, pp. 323-329, 2007.
122. J. Panksepp, Affective Neuroscience: The Foundations of Human and Animal Emotions. Oxford Univ. Press, 1998.
123. J. Panksepp, "Emotions as Natural Kinds within the Mammalian Brain," Handbook of Emotions, pp. 137-156, Guilford, 2000.
124. M.H. Immordino-Yang and A. Damasio, "We Feel, Therefore We Learn: The Relevance of Affective and Social Neuroscience to Education," Mind, Brain, and Education, vol. 1, pp. 3-10, 2007.
125. J.K. Olofsson, S. Nordin, H. Sequeira, and J. Polich, "Affective Picture Processing: An Integrative Review of ERP Findings," Biological Psychology, vol. 77, pp. 247-265, Mar. 2008.
126. A. Bashashati, M. Fatourechi, R.K. Ward, and G.E. Birch, "A Survey of Signal Processing Algorithms in Brain-Computer Interfaces Based on Electrical Brain Signals," J. Neural Eng., vol. 4, pp. R32-R57, June 2007. (Pubitemid 46603812)
127. F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi, "A Review of Classification Algorithms for EEG-Based Brain- Computer Interfaces," J. Neural Eng., vol. 4, pp. R1-R13, 2007. (Pubitemid 46603810)
128. C.E. Osgood, W.H. May, and M.S. Miron, Cross-Cultural Universals of Affective Meaning. Univ. of Illinois Press, 1975.
129. C. Lutz and G. White, "The Anthropology of Emotions," Ann. Rev. Anthropology, vol. 15, pp. 405-436, 1986.
130. C.E. Osgood, Language, Meaning, and Culture: The Selected Papers of C.E. Osgood. Praeger Publishers, 1990.
131. A.V. Samsonovich and G.A. Ascoli, "Cognitive Map Dimensions of the Human Value System Extracted from Natural Language," Proc. AGI Workshop Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms, pp. 111-124, 2006.
132. M.A. Cohn, M.R. Mehl, and J.W. Pennebaker, "Linguistic Markers of Psychological Change Surrounding September 11, 2001," Psychological Science, vol. 15, pp. 687-693, Oct. 2004. (Pubitemid 39443237)
133. J. Pennebaker, M. Mehl, and K. Niederhoffer, "Psychological Aspects of Natural Language Use: Our Words, Our Selves," Ann. Rev. Psychology, vol. 54, pp. 547-577, 2003. (Pubitemid 37417505)
134. J. Kahn, R. Tobin, A. Massey, and J. Anderson, "Measuring Emotional Expression with the Linguistic Inquiry and Word Count," Am. J. Psychology, vol. 120, pp. 263-286, Summer 2007.
135. C.G. Shields, R.M. Epstein, P. Franks, K. Fiscella, P. Duberstein, S.H. McDaniel, and S. Meldrum, "Emotion Language in Primary Care Encounters: Reliability and Validity of an Emotion Word Count Coding System," Patient Education and Counseling, vol. 57, pp. 232-238, May 2005. (Pubitemid 40724381)
136. Y. Bestgen, "Can Emotional Valence in Stories Be Determined from Words," Cognition and Emotion, vol. 8, pp. 21-36, Jan. 1994.
137. J. Hancock, C. Landrigan, and C. Silver, "Expressing Emotion in Text-Based Communication," Proc. SIGCHI, 2007.
138. J. Pennebaker, M. Francis, and R. Booth, Linguistic Inquiry and Word Count (LIWC): A Computerized Text Analysis Program. Erlbaum Publishers, 2001.
139. C. Chung and J. Pennebaker, "The Psychological Functions of Function Words," Social Comm., K. Fielder, ed., pp. 343-359, Psychology Press, 2007.
140. W. Weintraub, Verbal Behavior in Everyday Life. Springer, 1989.
141. G. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. Miller, "Introduction to Wordnet: An On-Line Lexical Database," J. Lexicography, vol. 3, pp. 235-244, 1990.
142. C. Strapparava and A. Valitutti, "WordNet-Affect: An Affective Extension of Word Net," Proc. Int'l Conf. Language Resources and Evaluation, vol. 4, pp. 1083-1086, 2004.
143. M.M. Bradley and P.J. Lang, "Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings," Technical Report C1, The Center for Research in Psychophysiology, Univ. of Florida, 1999.
144. R.A. Stevenson, J.A. Mikels, and T.W. James, "Characterization of the Affective Norms for English Words by Discrete Emotional Categories," Behavior Research Methods, vol. 39, pp. 1020-1024, 2007.
145. M.M. Bradley and P.J. Lang, "Affective Norms for English Text (ANET): Affective Ratings of Text and Instruction Manual," Technical Report No. D-1, Univ. of Florida, 2007.
146. A. Gill, R. French, D. Gergle, and J. Oberlander, "Identifying Emotional Characteristics from Short Blog Texts," Proc. 30th Ann. Conf. Cognitive Science Soc., B.C. Love, K. McRae, and V.M. Sloutsky, eds., pp. 2237-2242, 2008.
147. T. Landauer, D. McNamara, S. Dennis, and W. Kintsch, Handbook of Latent Semantic Analysis. Erlbaum, 2007.
148. K. Lund and C. Burgess, "Producing High-Dimensional Semantic Spaces from Lexical Co-Occurrence," Behavior Research Methods Instruments and Computers, vol. 28, pp. 203-208, 1996. (Pubitemid 126454814)
149. E. Breck, Y. Choi, and C. Cardie, "Identifying Expressions of Opinion in Context," Proc. 20th Int'l Joint Conf. Artificial Intelligence, 2007.
150. H. Liu, H. Lieberman, and S. Selker, "A Model of Textual Affect Sensing Using Real-World Knowledge," Proc. Eighth Int'l Conf. Intelligent User Interfaces, pp. 125-132, 2003.
151. M. Shaikh, H. Prendinger, and M. Ishizuka, "Sentiment Assessment of Text by Analyzing Linguistic Features and Contextual Valence Assignment," Applied Artificial Intelligence, vol. 22, pp. 558-601, 2008.
152. C. Akkaya, J. Wiebe, and R. Mihalcea, "Subjectivity Word Sense Disambiguation," Proc. 2009 Conf. Empirical Methods in Natural Language Processing, 2009.
153. A. Jaimes and N. Sebe, "Multimodal Human-Computer Interaction: A Survey," Computer Vision and Image Understanding, vol. 108, pp. 116-134, 2007. (Pubitemid 47333395)
154. R. Sharma, V.I. Pavlovic, and T.S. Huang, "Toward Multimodal Human-Computer Interface," Proc. IEEE, vol. 86, no. 5, pp. 853- 869, May. 1998. (Pubitemid 128720228)
155. C.O. Alm, D. Roth, and R. Sproat, "Emotions from Text: Machine Learning for Text-Based Emotion Prediction," Proc. Conf. Human Language Technology and Empirical Methods in Natural Language Processing, pp. 347-354, 2005.
156. T. Danisman and A. Alpkocak, "Feeler: Emotion Classification of Text Using Vector Space Model," Proc. AISB 2008 Convention, Comm., Interaction and Social Intelligence, 2008.
157. C. Strapparava and R. Mihalcea, "Learning to Identify Emotions in Text," Proc. 2008 ACM Symp. Applied Computing, pp. 1556-1560, 2008.
158. S.K. D'Mello, S.D. Craig, J. Sullins, and A.C. Graesser, "Predicting Affective States Expressed through an Emote-Aloud Procedure from AutoTutor's Mixed-Initiative Dialogue," Int'l J. Artificial Intelligence in Education, vol. 16, pp. 3-28, 2006.
159. S. D'Mello, N. Dowell, and A. Graesser, "Cohesion Relationships in Tutorial Dialogue as Predictors of Affective States," Proc. 2009 Conf. Artificial Intelligence in Education: Building Learning Systems That Care: From Knowledge Representation to Affective Modelling, V. Dimitrova, R. Mizoguchi, B. du Boulay, and A.C. Graesser, eds., pp. 9-16, 2009.
160. C. Ma, A. Osherenko, H. Prendinger, and M. Ishizuka, "A Chat System Based on Emotion Estimation from Text and Embodied Conversational Messengers," Proc. 2005 Int'l Conf. Active Media Technology, pp. 546-548, 2005. (Pubitemid 43995759)
161. A. Valitutti, C. Strapparava, and O. Stock, "Lexical Resources and Semantic Similarity for Affective Evaluative Expressions Generation," Proc. Int'l Conf. Affective Computing and Intelligent Interaction, pp. 474-481, 2005. (Pubitemid 43763364)
162. W.H. Lin, T. Wilson, J. Wiebe, and A. Hauptmann, "Which Side Are You On? Identifying Perspectives at the Document and Sentence Levels," Proc. 10th Conf. Natural Language Learning, pp. 109-116, 2006.
163. Z. Zeng, Y. Hu, G. Roisman, Z. Wen, Y. Fu, and T. Huang, "Audio- Visual Emotion Recognition in Adult Attachment Interview," Proc. Int'l Conf. Multimodal Interfaces, J.Y.F.K.H. Quek, D.W. Massaro, A.A. Alwan, and T.J. Hazen, eds., pp. 139-145, 2006. (Pubitemid 47113444)
164. Y. Yoshitomi, K. Sung-Ill, T. Kawano, and T. Kilazoe, "Effect of Sensor Fusion for Recognition of Emotional States Using Voice, Face Image and Thermal Image of Face," Proc. Int'l Workshop Robot and Human Interactive Comm., pp. 178-183, 2000.
165. L. Chen, T. Huang, T. Miyasato, and R. Nakatsu, "Multimodal Human Emotion/Expression Recognition," Proc. Third IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 366-371, 1998.
166. B. Dasarathy, "Sensor Fusion Potential Exploitation: Innovative Architectures and Illustrative Approaches," Proc. IEEE, vol. 85, no. 1, pp. 24-38, Jan. 1997. (Pubitemid 127745545)
167. G. Caridakis, L. Malatesta, L. Kessous, N. Amir, A. Paouzaiou, and K. Karpouzis, "Modeling Naturalistic Affective States via Facial and Vocal Expression Recognition," Proc. Int'l Conf. Multimodal Interfaces, pp. 146-154, 2006. (Pubitemid 47113445)
168. J. Liscombe, G. Riccardi, and D. Hakkani-Tür, "Using Context to Improve Emotion Detection in Spoken Dialog Systems," Proc. Ninth European Conf. Speech Comm. and Technology, 2005.
169. J. Ang, R. Dhillon, A. Krupski, E. Shriberg, and A. Stolcke, "Prosody-Based Automatic Detection of Annoyance and Frustration in Human-Computer Dialog," Proc. Int'l Conf. Spoken Language Processing, pp. 2037-2039, 2002.
170. K. Scherer and H. Ellgring, "Multimodal Expression of Emotion: Affect Programs or Componential Appraisal Patterns?," Emotion, vol. 7, pp. 158-171, 2007. (Pubitemid 46433172)
171. G. Castellano, L. Kessous, and G. Caridakis, "Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech," Affect and Emotion in Human-Computer Interaction, pp. 92-103, Springer, 2008.
172. S. Afzal and P. Robinson, "Natural Affect Data: Collection & Annotation in a Learning Context," Proc. Third Int'l Conf. Affective Computing and Intelligent Interaction and Workshops, pp. 1-7, 2009.
173. J. Cohn and K. Schmidt, "The Timing of Facial Motion in Posed and Spontaneous Smiles," Int'l J. Wavelets, Multiresolution and Information Processing, vol. 2, pp. 1-12, 2004.
174. P. Ekman, W. Friesen, and R. Davidson, "The Duchenne Smile-Emotional Expression and Brain Physiology .2," J. Personality and Social Psychology, vol. 58, pp. 342-353, 1990.
175. A. Kapoor and R.W. Picard, "Multimodal Affect Recognition in Learning Environments," Proc. 13th Ann. ACM Int'l Conf. Multimedia, pp. 677-682, 2005.
176. A. Kapoor, B. Burleson, and R. Picard, "Automatic Prediction of Frustration," Int'l J. Human-Computer Studies, vol. 65, pp. 724-736, 2007. (Pubitemid 46810294)
177. S. D'Mello and A. Graesser, "Multimodal Semi-Automated Affect Detection from Conversational Cues, Gross Body Language, and Facial Features," User Modeling and User-Adapted Interaction, vol. 10, pp. 147-187, 2010.
178. T. Baenziger, D. Grandjean, and K.R. Scherer, "Emotion Recognition from Expressions in Face, Voice, and Body. The Multimodal Emotion Recognition Test (MERT)," Emotion, vol. 9, pp. 691-704, 2009.
179. C. Busso et al., "Analysis of Emotion Recognition Using Facial Expressions, Speech and Multimodal Information," Proc. Int'l Conf. Multimodal Interfaces, T.D.R. Sharma, M.P. Harper, G. Lazzari, and M. Turk, eds., pp. 205-211, 2004. (Pubitemid 40371911)
180. Z.J. Chuang and C.H. Wu, "Multi-Modal Emotion Recognition from Speech and Text," Int'l J. Computational Linguistics and Chinese Language Processing, vol. 9, pp. 1-18, 2004.
181. A. Esposito, "Affect in Multimodal Information," Affective Information Processing, pp. 203-226, Springer, 2009.
182. R. Baker, S. D'Mello, M. Rodrigo, and A. Graesser, "Better to Be Frustrated than Bored: The Incidence and Persistence of Affect during Interactions with Three Different Computer-Based Learning Environments," Int'l J. Human-Computer Studies, vol. 68, no. 4, pp. 223-241, 2010.
183. S. D'Mello, R. Picard, and A. Graesser, "Towards an Affect- Sensitive AutoTutor," IEEE Intelligent Systems, vol. 22, no. 4, pp. 53-61, July 2007.
184. B. Lehman, M. Matthews, S. D'Mello, and N. Person, "What Are You Feeling? Investigating Student Affective States during Expert Human Tutoring Sessions," Proc. Ninth Int'l Conf. Intelligent Tutoring Systems, B. Woolf, E. Aimeur, R. Nkambou, and S. Lajoie, eds., pp. 50-59, 2008.
185. B. Lehman, S. D'Mello, and N. Person, "All Alone with Your Emotions: An Analysis of Student Emotions during Effortful Problem Solving Activities," Proc. Workshop Emotional and Cognitive Issues in ITS at the Ninth Int'l Conf. Intelligent Tutoring Systems, 2008.
186. S. D'Mello, S. Craig, K. Fike, and A. Graesser, "Responding to Learners' Cognitive-Affective States with Supportive and Shakeup Dialogues," Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction, pp. 595-604, Springer, 2009.
187. W. Burleson and R.W. Picard, "Gender-Specific Approaches to
188. S. D'Mello, B. Lehman, J. Sullins, R. Daigle, R. Combs, K. Vogt, L. Perkins, and A. Graesser, "A Time for Emoting: When Affect- Sensitivity Is and Isn't Effective at Promoting Deep Learning," Proc. 10th Int'l Conf. Intelligent Tutoring Systems, J. Kay and V. Aleven, eds., to appear.
189. I. Arroyo, B. Woolf, D. Cooper, W. Burleson, K. Muldner, and R. Christopherson, "Emotion Sensors Go to School," Proc. 14th Int'l Conf. Artificial Intelligence in Education, V. Dimitrova, R. Mizoguchi, B. Du Boulay, and A. Graesser, eds., pp. 17-24, 2009.
190. J. Robison, S. McQuiggan, and J. Lester, "Evaluating the Consequences of Affective Feedback in Intelligent Tutoring Systems," Proc. Int'l Conf. Affective Computing and Intelligent Interaction, pp. 37-42, 2009.
191. C. Conati and H. Maclaren, "Empirically Building and Evaluating a Probabilistic Model of User Affect," User Modeling and User- Adapted Interaction, vol. 19, pp. 267-303, 2009.
192. R. El Kaliouby, R. Picard, and S. Baron-Cohen, "Affective Computing and Autism," Annals New York Academy of Sciences, vol. 1093, pp. 228-248, 2006. (Pubitemid 47092380)
193. B. Parkinson, Ideas and Realities of Emotion. Routledge, 1995.
194. J. Gross, "Emotion Regulation," Handbook of Emotions, M. Lewis, J. Haviland-Jones, and L. Barrett, eds., third ed., pp. 497-512, Guilford, 2008.
195. J. Gross, "The Emerging Field of Emotion Regulation: An Integrative Review," Rev. of General Psychology, vol. 2, pp. 271- 299, 1998. (Pubitemid 128608316)
196. A. Ohman and J. Soares, "Unconscious Anxiety-Phobic Responses to Masked Stimuli," J. Abnormal Psychology, vol. 103, pp. 231-240, May 1994. (Pubitemid 24152796)
197. A. Ohman and J. Soares, "On the Automatic Nature of Phobic Fear-Conditioned Electrodermal Responses to Masked Fear- Relevant Stimuli," J. Abnormal Psychology, vol. 102, pp. 121-132, Feb. 1993. (Pubitemid 23089570)
198. M.S. Clark and I. Brissette, "Two Types of Relationship Closeness and Their Influence on People's Emotional Lives," Handbook of Affective Sciences, K.R.S. Richard, J. Davidson, and H. Hill Goldsmith, eds., pp. 824-835, Oxford Univ. Press, 2003.
199. A.J. Fridlund, P. Ekman, and H. Oster, "Facial Expressions of Emotion," Nonverbal Behavior and Communication, A.W. Siegman and S. Feldstein, eds., second ed., pp. 143-223, Erlbaum, 1987.
200. W. Ruch, "Will the Real Relationship between Facial Expression and Affective Experience Please Stand Up-The Case of Exhilaration," Cognition and Emotion, vol. 9, pp. 33-58, Jan. 1995.
201. K. Scherer and H. Ellgring, "Are Facial Expressions of Emotion Produced by Categorical Affect Programs or Dynamically Driven by Appraisal?" Emotion, vol. 7, pp. 113-130, Feb. 2007. (Pubitemid 46433169)
202. G. Bower, "Mood and Memory," Am. Psychologist, vol. 36, pp. 129- 148, 1981.
203. G. Mandler, "Emotion," Cognitive Science (Handbook of Perception and Cognition), B.M. Bly and D.E. Rumelhart, eds., second ed., Academic Press, 1999.
204. N. Stein and L. Levine, "Making Sense Out of Emotion," Memories Thoughts, and Emotions: Essays in Honor of George Mandler, A.O.W. Kessen and F. Kraik, eds., pp. 295-322, Erlbaum, 1991.
205. J. Bachorowski and M. Owren, "Vocal Expression of Emotion- Acoustic Properties of Speech Are Associated with Emotional Intensity and Context," Psychological Science, vol. 6, pp. 219-224, July 1995.
206. D. Keltner and J. Haidt, "Social Functions of Emotions," Emotions: Current Issues and Future Directions, T.J. Mayne and G.A. Bonanno, eds., pp. 192-213, Guilford Press, 2001.
207. C. Conati and H. Maclaren, "Modeling User Affect from Causes and Effects," Proc. First and 17th Int'l Conf. User Modeling, Adaptation and Personalization, 2009.
208. D.A. Norman, Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, 2005.
209. J. Cohen, "A Coefficient of Agreement for Nominal Scales," Educational and Psychological Measurement, vol. 20, pp. 37-46, 1960.
210. C. Robson, Real Word Research: A Resource for Social Scientist and Practitioner Researchers. Blackwell, 1993.