Representative segment-based emotion analysis and classification with automatic respiration signal segmentation

IEEE Transactions on Affective Computing

Volumn 3, Issue 4, 2012, Pages 482-495

  Wu, Chi Keng ^a   Chung, Pau Choo Julia ^a   Wang, Chi Jen ^a  

^a NATIONAL CHENG KUNG UNIVERSITY   (Taiwan)

Author keywords

Constraint based clustering analysis; Dynamic time warping; Emotion classification; Motion artifacts; Mutual information; Segmentation algorithm

Indexed keywords

CLUSTERING ANALYSIS; EMOTION CLASSIFICATION; MOTION ARTIFACT; MUTUAL INFORMATIONS; SEGMENTATION ALGORITHMS;

BIOSENSORS; RHODIUM COMPOUNDS;

SIGNAL RECONSTRUCTION;

EID: 84872249747     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/T-AFFC.2012.14     Document Type: Article

References (67)

1. R.W. Picard, Affective Computing. MIT Press, 2000.
2. D. Kulic and E.A. Croft, "Affective State Estimation for Human- Robot Interaction," IEEE Trans. Robotics, vol. 23, no. 5, pp. 991- 1000, Oct. 2007. (Pubitemid 47577975)
3. A. Kapoor, W. Burleson, and R.W. Picard, "Automatic Prediction of Frustration," Int'l J. Human-Computer Studies, vol. 65, no. 8, pp. 724-736, Aug. 2007. (Pubitemid 46810294)
4. J. Scheirer, R. Fernandez, J. Klein, and R.W. Picard, "Frustrating the User on Purpose: A Step toward Building an Affective Computer," Interacting with Computers, vol. 14, no. 2, pp. 93-118, 2002. (Pubitemid 34199154)
5. G.E. Sakr, I.H. Elhajj, and H.A.-S. Huijer, "Support Vector Machines to Define and Detect Agitation Transition," IEEE Trans. Affective Computing, vol. 1, no. 2, pp. 98-108, July-Dec. 2010.
6. L.S. Greenberg, "The Clinical Application of Emotion in Psychotherapy," Handbook of Emotions, M. Lewis and J.M. Haviland- Jones, eds., third ed., pp. 88-101, 2008.
7. W. James, "What Is an Emotion?" Mind, vol. 9, pp. 188-205, 1884.
8. S. Schachter and J. Singer, "Cognitive, Social, and Physiological Determinants of Emotional State," Psychological Rev., vol. 69, pp. 379-399, 1962.
9. W.B. Cannon, "The James-Lange Theory of Emotions: A Critical Examination and an Alternative Theory," Am. J. Psychology, vol. 39, pp. 106-124, 1927.
10. J.L. Andreassi, "Heart Activity and Behavior: Stress, Emotions, Motivation, Personality, Social Factors, Brain Interactions, and Conditioning," Psychophysiology: Human Behavior and Physiological Response, pp. 241-249, 1995.
11. J.T. Larsen, G.G. Berntson, K.M. Poehlmann, and T.A. Ito, and J.T. Cacioppo, "The Psychophysiology of Emotion," Handbook of Emotions, M. Lewis and J.M. Haviland-Jones, eds., third ed., pp. 173-191, 2008.
12. S.D. Kreibig, "Autonomic Nervous System Activity in Emotion: A Review," Biological Psychology, vol. 84, no. 3, pp. 394-421, July 2010.
13. R.W. Picard, E. Vyzas, and J. Healey, "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175-1191, Oct. 2001. (Pubitemid 33047955)
14. J. Kim and E. André, "Emotion Recognition Based on Physiological Changes in Music Listening," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 12, pp. 2067-2083, Dec. 2008.
15. K.H. Kim, S.W. Bang, and S.R. Kim, "Emotion Recognition System Using Short-Term Monitoring of Physiological Signals," Medical and Biological Eng. and Computing, vol. 42, pp. 1672-1687, 2004.
16. R.A. Calvo and S. D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 18-37, Jan. 2010.
17. Z. Zeng, M. Pantic, G.l. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
18. P. Philippot and G. Chapelle, "Respiratory Feedback in the Generation of Emotion," Cognition and Emotion, vol. 16, no. 5, pp. 605-627, Aug. 2002. (Pubitemid 34965713)
19. F.A. Boiten, N.H. Frijda, and C.J.E. Wientjes, "Emotions and Respiratory Patterns: Review and Critical Analysis," Int'l J. Psychophysiology, vol. 17, pp. 103-128, 1994. (Pubitemid 24244976)
20. C.L. Lisetti and F. Nasoz, "Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals," EURASIP J. Applied Signal Processing vol. 11, pp. 1672-1687, 2004.
21. J. Rottenberg, R.D. Ray, and J.J. Gross, "Emotion-Elicitation Using Films," Handbook of Emotion Elicitation And Assessment, J.A. Coan, ed., Oxford Univ. Press, 2007.
22. P. Gomez, S. Shafy, and B. Danuser, "Respiration, Metabolic Balance, and Attention in Affective Picture Processing," Biological Psychology, vol. 78, pp. 138-149, 2008.
23. R.W. Picard, "Affective Computing: From Laughter to IEEE," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 11-17, Jan.-June 2010.
24. F.A. Boiten, "The Effects of Emotional Behaviour on Components of the Respiratory Cycle," Biological Psychology, vol. 49, pp. 29-51, 1998. (Pubitemid 28479797)
25. C.-K. Wu, P.-C. Chung, and C.-J. Wang, "Extracting Coherent Emotion Elicited Segments from Physiological Signals," Proc. IEEE Workshop Affective Computational Intelligence, 2011.
26. P. Gomez, P. Zimmermann, S. Guttormsen-Schar, and B. Danuser, "Respiratory Responses Associated with Affective Processing of Film Stimuli," Biological Psychology, vol. 68, no. 3, pp. 223-235, Mar. 2005. (Pubitemid 40023357)
27. P. Rainville, A. Bechara, N. Naqvi, and A.R. Damasio, "Basic Emotions Are Associated with Distinct Patterns of Cardiorespiratory Activity," Int'l J. Psychophysiology, vol. 61, pp. 5-18, 2006. (Pubitemid 44209107)
28. M.A. Nicolaou, H. Gunes, and M. Pantic, "Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space," IEEE Trans. Affective Computing, vol. 2, no. 2, pp. 92-105, Apr.-June 2011.
29. D. Michael and J. Houchin, "Automatic EEG Analysis: A Segmentation Procedure Based on the Autocorrelation Function," Electroencephalography and Clinical Neurophysiology, vol. 46, no. 2, pp. 232-235, Feb. 1979. (Pubitemid 9083411)
30. L. Wong and W. Abdulla, "Time-Frequency Evaluation of Segmentation Methods for Neonatal EEG Signals" Proc. IEEE Int'l Conf. Eng. in Medicine and Biology Soc., pp. 1303-1306, 2006. (Pubitemid 46520111)
31. U. Appel and A.v. Brandt, "A Comparative Analysis of Three Sequential Time-Series Segmentation Algorithms," Signal Processing, vol. 6, no. 1, pp. 45-60, Jan. 1984. (Pubitemid 14547830)
32. D. Popivnov and A. Mineva, "Testing Procedures for Nonstationary and Non-Linearity in Physiological Signals," Math. Biosciences, vol. 157, nos. 1/2, pp. 303-320, 1999. (Pubitemid 29165073)
33. E. Punskaya, C. Andrieu, A. Doucet, and W.J. Fitzgerald, "Bayesian Curve Fitting Using MCMC with Applications to Signal Segmentation," IEEE Trans. Signal Processing, vol. 50, no. 3, pp. 747-758, Mar. 2002. (Pubitemid 34295114)
34. M. Staudacher, S. Telser, A. Amann, H. Hinterhuber, and M. Ritsch-Marte, "A New Method for Change-Point Detection Developed for On-Line Analysis of the Heart Beat Variability during Sleep," Physica A, vol. 349, nos. 3/4, pp. 682-596, Apr. 2005.
35. C.K. Peng, S. Havlin, H.E. Stanley, and A.L. Goldberger, "Quantification of Scaling Exponents and Crossover Phenomena in Nonstationary Heartbeat Time-Series," Am. Inst. of Physics, vol. 5, pp. 82-87, 1995.
36. A. Kaplan, J. Roschke, B. Darkhovsky, and J. Fell, "Macrostructural EEG Characterization Based on Nonparametric Change Point Segmentation: Application to Sleep Analysis," J. Neuroscience Methods, vol. 106, no. 1, pp. 81-90, Mar. 2001. (Pubitemid 32193017)
37. E. Tan, "Film-Induced Affect as a Witness Emotion," Poetics, vol. 23, no. 1, pp. 7-32, 1995. (Pubitemid 43090374)
38. L. Canini, S. Gilroy, M. Cavazza, R. Leonardi, and S. Benini, "Users' Response to Affective Film Content: A Narrative Perspective," Proc. Int'l Workshop Content-Based Multimedia Indexing, 2010.
39. H. Peng, F. Long, and C. Ding, "Feature Selection Based on Mutual Information: Criteria of Max-Denendency, Max-Relevance, and Min-Redundancy, " IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1226-1238, Aug. 2005. (Pubitemid 41245053)
40. P. Shaver, J. Schwartz, and D. Kirson, "Emotion Knowledge: Further Exploration of a Prototype Approach," J. Personality and Social Psychology, vol. 52, no. 6, pp. 1061-1086, 1987.
41. S. Epstein, "Controversial Issues in Emotion Theory," Rev. of Personality and Social Psychology, vol. 5, pp. 64-88, 1984.
42. D.A. Sauter, F. Eisner, P. Ekman, and S.K. Scott, "Cross-Cultural Recognition of Basic Emotions through Nonverbal Emotional Vocalizations," Proc. Nat'l Academy of Sciences USA, vol. 107, no. 6, pp. 2408-2412, Feb. 2010.
43. B. Fehr and J.A. Russell, "Concept of Emotion Viewed from a Prototype Perspective," J. Experimental Psychology: General, vol. 113, pp. 464-486, 1984.
44. I. Bretherton and M. Beeghly, "Talking about Internal States: The Acquisition of an Explicit Theory of Mind," Developmental Psychology, vol. 18, no. 6, pp. 906-912, 1982.
45. J.A. Russell, "In Defense of a Prototype Approach to Emotion Concepts," J. Personality and Social Psychology, vol. 60, no. 1, pp. 37- 47, 1991.
46. A. Kappas, "Smile When You Read This, Whether You Like It or Not: Conceptual Challenges to Affect Detection," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 38-41, Jan. 2010.
47. C. Peter and A. Herbon, "Emotion Representation and Physiology Assignments in Digital Systems," Interacting with Computers, vol. 18, no. 2, pp. 139-170, Mar. 2006. (Pubitemid 43267407)
48. B.E. Brodsky and B.E. Darkhovsky, Nonparametric Methods in Change-Point Problems. Kluwer Academic Publishers, 1993.
49. A. Denis and F. Cremoux, "Using the Entropy of Curves to Segment a Time or Spatial Series," Math. Geology, vol. 34, no. 8, pp. 899-914, Nov. 2002. (Pubitemid 41444729)
50. M.D. Weber, L.M. Leemis, and R.K. Kincaid, "Minimum Kolmogorov-Smirnov Test Statistic Parameter Estimates," J. Statistical Computation and Simulation, vol. 76, no. 3, pp. 195- 206, Mar. 2006. (Pubitemid 43109045)
51. J. Richmann and J. Moorman, "Physiological Time-Series Analysis Using Approximate Entropy and Sample Entropy," Am. J. Physiology-Heart and Circulatory Physiology, vol. 278, no. 6, pp. H2039-H2049, June 2000. (Pubitemid 30439218)
52. D. Caldirola, L. Bellodi, A. Caumo, G. Migliarese, and G. Perna, "Approximate Entropy of Respiratory Patterns in Panic Disorder," Am. J. Psychiatry, vol. 161, no. 1, pp. 79-87, 2004. (Pubitemid 38419311)
53. C.D. Katsis, N. Katertsidis, G. Ganiatsas, and D.I. Fotiadis, "Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach," IEEE Trans. Systems, Man, and Cybernetics- Part A: Systems and Humans, vol. 38, no. 3, pp. 502-512, May 2008. (Pubitemid 351649883)
54. G.M. Smith, Film Structure and the Emotion System. Cambridge Univ. Press, 2003.
55. M. Soleymani, G. Chanel, J. Kierkels, and T. Pun, "Affective Ranking of Movie Scenes Using Physiological Signals and Content Analysis," Proc. ACM Workshop Multimedia Semantics, pp. 32-39, Oct. 2008.
56. J. Listgarten, R.M. Neal, S.T. Roweis, and A. Emili, "Multiple Alignment of Continuous Time Series," Neural Information Processing Systems, vol. 17, 2005.
57. R. JayaDevan, S.R. Kolhe, and P.M. Patil, "Dynamic Time Warping Based Static Hand Printed Signature Verification," J. Pattern Recognition Research, vol. 4, no. 1, pp. 52-65, 2009.
58. I. Guyon and A. Elisseeff, "An Introduction to Variable and Feature Selection," J. Machine Learning Research, vol. 3, pp. 1152- 1182, Mar. 2003.
59. H. Peng mRMR, http://penglab.janelia.org/proj/mRMR/, 2011.
60. D.L. Davies and D.W. Bouldin, "A Cluster Separation Measure," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, no. 2, pp. 224-227, Apr. 1979.
61. P. Rainville, A. Bechara, N. Naqvi, A.R. Damasio, "Basic Emotions Are Associated with Distinct Patterns of Cardiorespiratory Activity," Int'l J. Psychophysiology, vol. 61, no. 1, pp. 5-18, July 2006. (Pubitemid 44209107)
62. C.M. Bishop, "Probability Distributions," Pattern Recognition and Machine Learning, pp. 124-127, Springer, 2006.
63. D. Cheng, G. Liu, and Y. Qiu, "Applications of Particle Swarm Optimization and K-Nearest Neighbors to Emotion Recognition from Physiological Signals," Proc. Int'l Conf. Computational Intelligence and Security, 2008.
64. S. Chen, C.F.N. Cowan, and P.M. Grant, "Orthogonal Least Squares Learning Algorithm for Radial Basis Function Networks," IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 302-309, Mar. 1991.
65. N.H. Frijda, The Emotions, pp. 124-175. Cambridge Univ. Press, 1986.
66. P. Lang, "The Emotion Probe: Studies of Motivation and Attention," Am. Psychologist, vol. 50, no. 5, pp. 372-385, 1995.
67. C.C. Chang and C.J. Lin, "LIBSVM: A Library for Support Vector Machines Software," http://www.csie.ntu.edu.tw/~cjlin/ libsvm, 2011.