Emotion recognition based on physiological changes in music listening

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 30, Issue 12, 2008, Pages 2067-2083

  Kim, Jonghwa ^a   André, Elisabeth ^a  

^a UNIVERSITY OF AUGSBURG   (Germany)

Author keywords

Affective computing; Arousal; Autonomic nervous system; Biosignal; Electrocardiogram; Electromyogram; Emotion recognition; Human computer interaction; Musical emotion; Physiological signal; Respiration; Skin conductance; Valence

Indexed keywords

BIOSENSORS; COMPUTER GENERATED HOLOGRAPHY; DISCRIMINANT ANALYSIS; FACE RECOGNITION; KNOWLEDGE MANAGEMENT; SKIN;

AFFECTIVE COMPUTING; AROUSAL; AUTONOMIC NERVOUS SYSTEM; BIOSIGNAL; ELECTROCARDIOGRAM; ELECTROMYOGRAM; EMOTION RECOGNITION; MUSICAL EMOTION; PHYSIOLOGICAL SIGNAL; RESPIRATION; SKIN CONDUCTANCE; VALENCE;

HUMAN COMPUTER INTERACTION;

ADAPTATION; ALGORITHM; AROUSAL; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; EMOTION; HEARING; HUMAN; METHODOLOGY; MONITORING; MUSIC; PHYSIOLOGY;

ADAPTATION, PHYSIOLOGICAL; ALGORITHMS; AROUSAL; ARTIFICIAL INTELLIGENCE; AUDITORY PERCEPTION; EMOTIONS; HUMANS; MONITORING, PHYSIOLOGIC; MUSIC; PATTERN RECOGNITION, AUTOMATED;

EID: 56649124649     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2008.26     Document Type: Article

References (47)

1. W. James, The Principles of Psychology. Holt, 1890.
2. W.B. Cannon, "The James-Lange Theory of Emotions: A Critical Examination and an Alternative Theory," Am. J. Psychology, vol. 39, pp. 106-127, 1927.
3. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J.G. Taylor, "Emotion Recognition in Human-Computer Interaction," IEEE Signal Processing Magazine, vol. 18, pp. 32-80, 2001.
4. P. Ekman, "The Argument and Evidence about Universals in Facial Expressions of Emotion," Handbook of Social Psychophysiology, pp. 143-164, John Wiley & Sons, 1989.
5. R.W. Levenson, P. Ekman, P. Heider, and W.V. Friesen, "Emotion and Autonomic Nervous System Activity in the Minangkabau of West Sumatra," J. Personality and Social Psychology, vol. 62, pp. 972-988, 1992.
6. J.T. Cacioppo, D.J. Klein, G.G. Bemston, and E. Hatfield, "The Psychophysiology of Emotion," Handbook of Emotions, M. Lewis and J. Haviland, eds., pp. 119-142, Guilford Press, 1993.
7. R.W. Levenson, P. Ekman, and W.V. Friesen, "Voluntary Facial Action Generates Emotion-Specific Autonomic Nervous System Activity," Psychophysiology, vol. 27, pp. 363-384, 1990.
8. G. Stemmler, M. Heldmann, C.A. Pauls, and T. Scherer, "Constraints for Emotion Specificity in Fear and Anger: The Context Counts," Psychophysiology, vol. 38, pp. 275-291, 2001.
9. S. Schachter and J.E. Singer, "Cognitive, Social, and Physiological Determinants of Emotional State," Psychological Rev., vol. 69, pp. 379-399, 1962.
10. G. Stemmler, "The Autonomic Differentiation of Emotions Revisited: Convergent and Discriminant Validation," Psychophysiology, vol. 26, pp. 617-632, 1989.
11. M.P. Kastner and R.G. Crowder, "Perception of the Major/Minor Distinction: IV. Emotional Connotations in Young Children," Music Perception, vol. 8, pp. 189-201, 1990.
12. P. Kivy, Sound Sentiment: An Essay on the Musical Emotions. Temple Univ. Press, 1989.
13. C.L. Krumhansl, "An Exploratory Study of Musical Emotions and Psychophysiology," Canadian J. Experimental Psychology, vol. 51, pp. 336-352, 1997.
14. L.B. Meyer, Emotion and Meaning in Music. Univ. of Chicago Press, 1956.
15. D. Vaitl, W. Vehrs, and S. Sternagel, "Prompts-Leitmotif-Emotion: Play It Again, Richard Wagner," The Structure of Emotion: Psychophysiological, Cognitive, and Clinical Aspects, pp. 169-189, Hogrefe & Huber, 1993.
16. W.B. Davis and M.H. Thaut, "The Influence of Preferred Relaxing Music on Measures of State Anxiety, Relaxation, and Physiological Responses," J. Music Therapy, vol. 26, no. 4, pp. 168-187, 1989.
17. C.E. Guzzetta, "Effects of Relaxation and Music Therapy on Patients in a Coronary Care Unit with Presumptive Acute Myocardial Infarction," Heart and Lung: J. Critical Care, vol. 18, no. 6, pp. 609-616, 1989.
18. J. Healey and R.W. Picard, "Digital Processing of Affective Signals," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 3749-3752, 1998.
19. R. Picard, E. Vyzas, and J. Healy, "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175-1191, Oct. 2001.
20. F. Nasoz, K. Alvarez, C. Lisetti, and N. Finkelstein, "Emotion Recognition from Physiological Signals for Presence Technologies," Int'l J. Cognition, Technology, and Work, special issue on presence, vol. 6, no. 1, 2003.
21. J.J. Gross and R.W. Levenson, "Emotion Elicitation Using Films," Cognition and Emotion, vol. 9, pp. 87-108, 1995.
22. A. Haag, S. Goronzy, P. Schaich, and J. Williams, "Emotion Recognition Using Bio-Sensors: First Steps Towards an Automatic System," Proc. Ninth Int'l Conf. Reliable Software Technologies, pp. 36-48, 2004.
23. Center for the Study of Emotion and Attention (CSEA-NIMH), The International Affective Picture System: Digitized Photographs, Center for Research in Psychophysiology, Univ. of Florida, 1995.
24. K.H. Kim, S.W. Bang, and S.R. Kim, "Emotion Recognition System Using Short-Term Monitoring of Physiological Signals," Medical & Biological Eng. and Computing, vol. 42, pp. 419-427, 2004.
25. P. Lang, "The Emotion Probe: Studies of Motivation and Attention," Am. Psychologist, vol. 50, no. 5, pp. 372-385, 1995.
26. J. Tooby and L. Cosmides, "The Past Explains the Present: Emotional Adaptations and the Structure of Ancestral Environments," Ethology and Sociobiology, vol. 11, pp. 375-424, 1990.
27. H. Scholsberg, "Three Dimensions of Emotion," Psychological Rev. vol. 61, pp. 81-88, 1954.
28. B.L. Fredricson and R.W. Levenson, "Positive Emotions Speed Recovery from the Cardiovascular Sequelae of Negative Emotions," Cognition and Emotion, vol. 12, no. 2, pp. 191-220, 1998.
29. J.E. LeDoux, The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental Dysfunction, pp. 339-351. Wiley-Liss, 1992.
30. D.M. Sloan, "Emotion Regulation in Action: Emotional Reactivity in Experiential Avoidance," Behavior Research and Therapy, vol. 4, pp. 1257-1270, 2004.
31. B. Melin and U. Lundberg, "A Biopsychosocial Approach to Work-Stress and Musculoskeletal Disorders," J. Psychophysiology, vol. 11, no. 3, pp. 238-247, 1997.
32. H.G. McCurdy, "Consciousness and the Galvanometer," Psychological Rev., vol. 57, pp. 322-327, 1950.
33. A. Jain, R. Duin, and J. Mao, "Statistical Pattern Recognition: A Review," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4-37, Jan. 2000.
34. R.D. King, C. Feng, and A. Shutherland, "StatLog: Comparison of Classification Algorithms on Large Real-World Problems," Applied Artificial Intelligence, vol. 9, no. 3, pp. 259-287, May/June 1995.
35. M.B. Mcllory and M.D. Cheithin, Clinical Cardiology, fifth ed. VLANGE Medical Book, 1990.
36. J. Pan and W. Tompkins, "A Real-Time QRS Detection Algorithm," IEEE Trans. Biomedical Eng., vol. 32, no. 3, pp. 230-323, 1985.
37. P.W. Kamen, H. Krum, and A.M. Tonkin, "Poincare Plot of Heart Rate Variability Allows Quantitative Display of Parasympathetic Nervous Activity," Clinical Science, vol. 91, pp. 201-208, 1996.
38. J. Richmann and J. Moorman, "Physiological Time Series Analysis Using Approximate Entropy and Sample Entropy," Am. J. Physiology - Heart and Circulatory Physiology, vol. 278, p. H2039, 2000.
39. M. Costa, A.L. Goldberger, and C.-K. Peng, "Multiscale Entropy Analysis of Biological Signals," Physical Rev. E, vol. 71, no. 021906, 2005.
40. R. Thuraisingham and G. Gottwald, "On Multiscale Entropy Analysis for Physiological Data," Physica A, 2005.
41. A. Malliani, "The Pattern of Sympathovagal Balance Explored in the Frequency Domain," News in Physiological Science, vol. 14, pp. 111-117, 1999.
42. A. Jain and D. Zongker, "Feature Selection: Evaluation, Application, and Small Sample Performance," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 153-163, Feb. 1997.
43. F.J. Ferri, P. Pudil, M. Hatef, and J. Kittler, "Comparative Study of Techniques for Large-Scale Feature Selection," Pattern Recognition in Practice IV: Multiple Paradigms, Comparative Studies, and Hybrid Systems, S.E. Gelsema and L.N. Kanal, eds., pp. 403-413, 1994.
44. D.L. Swets and J. Weng, "Using Discriminant Eigenfeatures for Image Retrieval," IEEE Trans. Pattern Analysis and Machine Intelligence vol. 18, no. 8, pp. 831-836, Aug. 1996.
45. T. Hastie and R. Tibshirani, "Classification by Pairwise Coupling," The Annals of Statistics, vol. 26, no. 1, pp. 451-471, 1998.
46. J. Friedman, "Another Approach to Polychotomous Classification," technical report, Dept. of Statistics, Stanford Univ., 1996.
47. L. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms John Wiley & Sons, 2004.