Revealing real-time emotional responses: A personalized assessment based on heartbeat dynamics

Scientific Reports

Volumn 4, Issue , 2014, Pages

  Valenza, Gaetano ^a,b,c   Citi, Luca ^a,b,d   Lanatá, Antonio ^c   Scilingo, Enzo Pasquale ^c   Barbieri, Riccardo ^a,b  

^a MASSACHUSETTS GENERAL HOSPITAL   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c UNIVERSITY OF PISA   (Italy)

^d UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; BIOLOGICAL MODEL; COMPUTER SIMULATION; ELECTROCARDIOGRAPHY; EMOTION; HEART RATE; HUMAN; NONLINEAR SYSTEM; PATTERN RECOGNITION; PHYSIOLOGY; SIGNAL PROCESSING; SUPPORT VECTOR MACHINE;

ALGORITHMS; COMPUTER SIMULATION; ELECTROCARDIOGRAPHY; EMOTIONS; HEART RATE; HUMANS; MODELS, CARDIOVASCULAR; NONLINEAR DYNAMICS; PATTERN RECOGNITION, PHYSIOLOGICAL; SIGNAL PROCESSING, COMPUTER-ASSISTED; SUPPORT VECTOR MACHINES;

EID: 84901251366     PISSN: None     EISSN: 20452322     Source Type: Journal    
DOI: 10.1038/srep04998     Document Type: Article

References (98)

1. Picard, R. W. Affective computing (MIT Press, Boston, MA, 1997).
2. Sauter, D., Eisner, F., Ekman, P. & Scott, S. Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. P. Natl. Acad. Sci. USA 107, 2408-2412 (2010).
3. Kessler, R. et al. Lifetime and 12-month prevalence of dsm-iii-r psychiatric disorders in the united states: results from the national comorbidity survey. Arch. gen. psychiat. 51, 8 (1994).
4. Wittchen, H. & Jacobi, F. Size and burden of mental disorders in europe: a critical review and appraisal of 27 studies. Eur. neuropsychopharm. 15, 357-356 (2005).
5. Tang, Y. et al. Central and autonomic nervous system interaction is altered by short-term meditation. P. Natl. Acad. Sci. USA 106, 8865 (2009).
6. Petrantonakis, P. C. & Hadjileontiadis, L. J. Emotion Recognition From EEG Using Higher Order Crossings. IEEE T. Inf. Technol. B. 14, 186-197 (2010).
7. Ko, K. E., Yang, H. C. & Sim, K. B. Emotion recognition using EEG signals with relative power values and Bayesian network. Int. J. Control Autom. 7, 865-870 (2009).
8. Li M., Chai, Q., Kaixiang, T.,Wahab, A. & Abut, H. [EEG Emotion Recognition System.] Vehicle Corpus and Signal Processing for Driver Behavior [Takeda, K. (ed.), Hansen, J. H. L. (ed.), Erdogan, H. (ed.), Abut, H. (ed.)] [125-135] (Springer, New York, 2009).
9. Lin, Y. P. et al. EEG-Based Emotion Recognition in Music Listening. IEEE T. Bio- Med Eng. 57, 1798-1806 (2010).
10. Petrantonakis, P. C. & Hadjileontiadis, L. J. A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition. IEEE T. Bio-Med Eng. 15, 737-746 (2011).
11. Park, K. S. et al. Emotion recognition based on the asymmetric left and right activation. Int. J. Med. Medical Sci. 3, 201-209 (2011).
12. Hsieha, S., Hornbergera, M., Pigueta, O. & Hodges, J. R. Brain correlates of musical and facial emotion recognition: Evidence from the dementias. Neuropsychologia 50, 1814-1822 (2012).
13. Ruiz-Padial, E., Vila, J. & Thayer, J. The effect of conscious and non-conscious presentation of biologically relevant emotion pictures on emotion modulated startle and phasic heart rate. Int. J. Psychophysiol. 79, 341-346 (2011).
14. Tamietto, M. et al. Unseen facial and bodily expressions trigger fast emotional reactions. P. Natl. Acad. Sci. USA 106, 17661-17666 (2009).
15. Heller, A. et al. Reduced capacity to sustain positive emotion in major depression reflects diminished maintenance of fronto-striatal brain activation. P. Natl. Acad. Sci. USA 106, 22445-22450 (2009).
16. Valenza, G. et al.Wearable monitoring for mood recognition in bipolar disorder based on history-dependent long-term heart rate variability analysis. IEEE J. Biomed. Health Inform., (2014).
17. Valenza, G. et al. Mood recognition in bipolar patients through the PSYCHE platform: preliminary evaluations and perspectives. Artif. Intell. Med. 57, 49-58 (2013).
18. Lanata, A., Valenza, G. & Scilingo, E. P. A novel eda glove based on textileintegrated electrodes for affective computing. Med. Biol. Eng. Comput. 50, 1163-1172 (2012).
19. Calvo, R. & D'Mello, S. Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE T. Affect. Comp. 1, 18-37 (2010).
20. Koolagudi, S. G. & Rao, K. S. Recognition of Emotion from Speech: A Review. Int J Speech Technol 15, 99-117 (2012).
21. Agrafioti, F., Hatzinakos, D. & Anderson, A. Ecg pattern analysis for emotion detection. IEEE T. Affect. Comp. 3, 102-115 (2012).
22. Wu, C. K., Chung, P. C. & Wang, C. J. Representative Segment-based Emotion Analysis and Classification with Automatic Respiration Signal Segmentation. IEEE T. Affect. Comp. 3, 482-495 (2012).
23. Katsis, C. D., Katertsidis, N. S. & Fotiadis, D. I. An integrated system based on physiological signals for the assessment of affective states in patients with anxiety disorders. Biomed. Signal Proces. 6, 261-268 (2011).
24. Task Force of the European Society of Cardiology, the North American Society of Pacing, and Electrophysiology, Heart rate variability: standards of measurement, physiological interpretation and clinical use. Circulation 93, 1043-65 (1996).
25. Acharya, U. R., Joseph, K. P., Kannathal, N., Lim, C. & Suri, J. Heart rate variability: a review. Med. Biol. Eng. Comput. 44, 1031-1051 (2006). (Pubitemid 47090818)
26. Chua, C. K., Chandran, V., Acharya, U. R. & Lim, C. Application of higher order statistics/spectra in biomedical signals-A review. Med. Eng. Phys. 32, 679-689 (2010).
27. Sunagawa, K., Kawada, T. & Nakahara, T. Dynamic nonlinear vago-sympathetic interaction in regulating heart rate. Heart Vessels 13, 157-174 (1998). (Pubitemid 29322458)
28. Atyabi, F., Livari, M., Kaviani, K. & Tabar, M. Two statistical methods for resolving healthy individuals and those with congestive heart failure based on extended self-similarity and a recursive method. J. Biol. Phys. 32, 489-495 (2006). (Pubitemid 46872925)
29. Glass, L. Synchronization and rhythmic processes in physiology. Nature 410, 277-284 (2001). (Pubitemid 32218701)
30. Costa, M., Goldberger, A. & Peng, C. K. Multiscale entropy analysis of complex physiologic time series. Phys. Rev. Lett. 89, 68102 (2002).
31. Poon, C.& Merrill, C. Decrease of cardiac chaos in congestive heart failure. Nature 389, 492-495 (1997). (Pubitemid 27435323)
32. Tulppo, M. et al. Physiological background of the loss of fractal heart rate dynamics. Circulation 112, 314 (2005).
33. Wu, G. et al. Chaotic signatures of heart rate variability and its power spectrum in health, aging and heart failure. PloS one 4, e4323 (2009).
34. Goldberger, A., Rigney, D. & West, B. Science in pictures: Chaos and fractals in human physiology. Sci. Am. 262, 42-49 (1990). (Pubitemid 20033153)
35. Sugihara, G., Allan, W., Sobel, D. & Allan, K. Nonlinear control of heart rate variability in human infants. P. Natl. Acad. Sci. USA 93, 2608 (1996).
36. Leistedt, S. J. J. et al. Decreased neuroautonomic complexity in men during an acute major depressive episode: analysis of heart rate dynamics. Translational Psych. 1, e27 (2011).
37. Barbieri, R., Matten, E., Alabi, A. & Brown, E. A point-process model of human heartbeat intervals: new definitions of heart rate and heart rate variability. Am. J. Physiol.-Heart C. 288, H424 (2005).
38. Barbieri, R. & Brown, E. Analysis of heartbeat dynamics by point process adaptive filtering. IEEE T. Bio-Med Eng. 53, 4-12 (2006). (Pubitemid 43074786)
39. Chen, Z., Brown, E. & Barbieri, R. Assessment of autonomic control and respiratory sinus arrhythmia using point process models of human heart beat dynamics. IEEE T. Bio-Med Eng. 56, 1791-1802 (2009).
40. Chen, Z., Brown, E. & Barbieri, R. Characterizing nonlinear heartbeat dynamics within a point process framework. IEEE T. Bio-Med Eng. 57, 1335-1347 (2010).
41. Valenza, G., Citi, L., Scilingo, E. P. & Barbieri, R. Point-Process NonlinearModels with Laguerre and Volterra Expansions: Instantaneous Assessment of Heartbeat Dynamics. IEEE T. Signal Proces. 61, 2914-2926 (2013).
42. Valenza, G., Citi, L., Scilingo, E. P. & Barbieri, R. Using Laguerre expansion within point-process models of heartbeat dynamics: a comparative study. Paper presented at the International IEEE Engineering in Medicine and Biology Conference, San Diego CA, USA, IEEE 2012, August 28-September 1.
43. Valenza, G., Citi, L., Scilingo, E. P. & Barbieri, R. Instantaneous Bispectral Characterization of the Autonomic Nervous System through a Point-Process NonlinearModel. Paper presented at theWorld Congress on Medical Physics and Biomedical Engineering, Beijing, China, (2012 May 21-31).
44. Valenza, G., Lanata, A.& Scilingo, E. P. The role of nonlinear dynamics in affective valence and arousal recognition. IEEE T. Affect. Comp. 3, 237-249 (2012).
45. Valenza, G., Allegrini, P., Lanata, A. & Scilingo, E. P. Dominant Lyapunov Exponent and Approximate Entropy in Heart Rate Variability during Emotional Visual Elicitation. Frontiers in Neuroeng. 5, 1-7 (2012).
46. Valenza, G., Lanata, A., Scilingo, E. P. & De Rossi, D. Towards a Smart Glove: Arousal Recognition based on Textile Electrodermal Response. Paper presented at the International IEEE Engineering in Medicine and Biology Conference, Buenos Aires, Argentina, IEEE (August 31-September 4).
47. Valenza, G., Citi, L., Lanata, A., Scilingo, E. P. & Barbieri, R. A Nonlinear Heartbeat Dynamics Model Approach for Personalized Emotion Recognition. Paper presented at International IEEE Engineering in Medicine and Biology Conference, Osaka, Japan, IEEE (2013, July 3-7).
48. Valenza, G., Lanata, A. & Scilingo, E. P. Oscillations of heart rate and respiration synchronize during affective visual stimulation. IEEE T. Inf. Technol. B. 16, 683 (2012).
49. Posner, J., Russell, J. & Peterson, B. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17, 715-734 (2005).
50. Lang, P., Bradley, M. & Cuthbert, B. International affective picture system iaps: Digitized photographs, instruction manual and affective ratings. Technical report A-6. Univ. Florida, (2005).
51. Kroenke, K., Spitzer, R. & Williams, J. The phq-9. J. Gen. Intern.Med. 16, 606613 (2001).
52. Citi, L., Brown, E. & Barbieri, R. A Real-Time Automated Point Process Method for Detection and Correction of Erroneous and Ectopic Heartbeats. IEEE T. Bio- Med Eng. 59, 2828-2837 (2012).
53. Barnett, A. & Wolff, R. A time-domain test for some types of nonlinearity. IEEE T. Signal Proces. 53, 26-33 (2005). (Pubitemid 40085330)
54. Billings, S. Identification of nonlinear system - a survey. Proc. IEEE 127, 272-285 (1980).
55. Marmarelis, V. Nonlinear dynamic modeling of physiological systems. (Wiley-IEEE Press, New York, 2004).
56. Nikias, C. & Mendel, J. Signal processing with higher-order spectra. IEEE Signal Proc. Mag. 10, 10-37 (1993).
57. Nikias, C. & Petropulu, A. Higher-order spectra analysis: A nonlinear signal processing framework. (PTR Prentice-Hall, Inc., New Jersey, USA, 1993).
58. Chua, K. C., Chandran, V., Acharya, U. R. & Lim, C. M. Cardiac state diagnosis using higher order spectra of heart rate variability. J. Med. Eng. Technol. 32, 145-155 (2008). (Pubitemid 351302367)
59. Lanata, A., Valenza, G., Mancuso, C. & Scilingo, E. P. Robust multiple cardiac arrhythmia detection through bispectrum analysis. Expert Syst. Appl. 38, 6798-6804 (2011).
60. Le Caillec, J. & Garello, R. Nonlinear system identification using autoregressive quadratic models. Signal Process. 81, 357-379 (2001). (Pubitemid 32196404)
61. Citi, L., Valenza, G. & Barbieri, R. Instantaneous estimation of high-order nonlinear heartbeat dynamics by Lyapunov exponents. Paper presented at International IEEE Engineering in Medicine and Biology Conference, San Diego, USA, IEEE (2012, August 28-September 1).
62. Chang, C. C. & Lin, C. J. LIBSVM: A library for support vector machines. ACM T. Intell. Syst. Tech. 2, 1-27 (2011). Software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm.
63. Van der Heijden, F., Duin, R. P. W., De Ridder, D. & Tax, D. M. J. Classification, parameter estimation and state estimation. (Wiley Online Library, New York, USA, 2004).
64. Schlogl, A. Time Series Analysis - A toolbox for the use with Matlab. Available online at http://pub.ist.ac.at/%7Eschloegl/matlab/tsa/ (Accessed: 18 February 2014).
65. Valenza, G., Citi, L. & Barbieri, R. Instantaneous Nonlinear Assessment of Complex Cardiovascular Dynamics by Laguerre-Volterra Point Process Models. Paper presented at International IEEE Engineering in Medicine and Biology Conference, Osaka, Japan, IEEE (July 2013).
66. Scholkopf, B., Smola, A., Williamson, R. & Bartlett, P. L. New support vector algorithms. Neural Comput. 12, 1207-124 (2000).
67. Duda, R., Hart, P. & Stork, D. G. Pattern classification. (JohnWiley & Sons, New York, 2012).
68. He, X., Deng, C. & Partha, N. Laplacian score for feature selection. Adv. Neur. In. 18, 507-514 (2005).
69. Granger, C. & Joyeux, R. An introduction to long-memory time series models and fractional differencing. J. Time Ser. Anal. 1, 15-29 (1980).
70. Chen, Z., Purdon, P., Brown, E. & Barbieri, R. A differential autoregressive modeling approach within a point process framework for non-stationary heartbeat intervals analysis. Paper presented at International IEEE Engineering in Medicine and Biology Conference, Buenos Aires, Argentina, IEEE (2010, August 31-September 4).
71. Valenza, G., Lanata, A. & Scilingo, E. P. Improving emotion recognition systems by embedding cardiorespiratory coupling. Physiol. Meas. 34, 449-464 (2013).
72. Bartsch, R., Schumann, A., Kantelhardt, J., Penzel, T. & Ivanov, P. Phase transitions in physiologic coupling. P. Natl. Acad. Sci. USA 109, 10181-10186 (2012).
73. Chen, Z. et al. Dynamic assessment of baroreflex control of heart rate during induction of propofol anesthesia using a point process method. Ann. Biomed. Eng. 39, 1-17 (2011).
74. Mendel, J. Tutorial on higher-order statistics (spectra) in signal processing and system theory: Theoretical results and some applications. Proc. IEEE 79, 278-305 (1991).
75. Loader, C. Local regression and likelihood. (Springer Verlag, Berlin, 1999).
76. Tibshirani, R. & Hastie, T. Local likelihood estimation. J. Am. Stat. Assoc. 82, 559-567 (1987).
77. Brown, E., Barbieri, R., Eden, U. & Frank, L. [Likelihood methods for neural spike train data analysis.] Computational neuroscience: A comprehensive approach [Feng, J.] [253-286] (CRC Press, Boca Raton, Florida, 2003).
78. Westwick, D. & Kearney, R. Explicit least-squares methods, in Identification of nonlinear physiological systems. (Wiley-IEEE Press, New York, 2003).
79. Akay, M. Nonlinear Biomedical Signal Processing II: Dynamic Analysis and Modeling. (Wiley-IEEE Press, New York, 2000).
80. Jo, J. et al. A nonlinear model of cardiac autonomic control in obstructive sleep apnea syndrome. Ann. Biomed. Eng. 35, 1425-1443 (2007). (Pubitemid 47115172)
81. Koukoulas, P. & Kalouptsidis, N. Nonlinear system identification using gaussian inputs. IEEE T. Signal Proces. 43, 1831-1841 (1995).
82. Nikias, C. & Raghuveer, M. Bispectrum estimation: A digital signal processing framework. Proc. IEEE 75, 869-891 (1987). (Pubitemid 17651268)
83. Nichols, J., Olson, C., Michalowicz, J. & Bucholtz, F. The bispectrum and bicoherence for quadratically nonlinear systems subject to non- gaussian inputs. IEEE T. Signal Proces. 57, 3879-3890 (2009).
84. Caillec, J. & Garello, R. Asymptotic bias and variance of conventional bispectrum estimates for 2-d signals. Multidim. Syst. Sign. P. 16, 49-84 (2005). (Pubitemid 40329463)
85. Brillinger, D. & Rosenblatt, M. Asymptotic theory of estimates of kth-order spectra. P. Natl. Acad. Sci. USA 57, 206 (1967).
86. Chandran, V. & Elgar, S. Pattern recognition using invariants defined from higher order spectra-one dimensional inputs. IEEE T. Signal Proces. 41, 1, 205 (1993).
87. Ng, T., Chang, S. & Sun, Q. Blind detection of photomontage using higher order statistics. Paper presented at the International Symposium on Circuits and Systems, Vancouver, Canada, IEEE (2004, May 23-26).
88. Chua, K., Chandran, V., Acharya, U. & Lim, C. Application of higher order statistics/spectra in biomedical signals-a review. Med. Eng. Phys. 32, 679-689 (2010).
89. Zhou, S., Gan, J. & Sepulveda, F. Classifying mental tasks based on features of higher-order statistics from eeg signals in brain-computer interface. Inform. Sciences 178, 1629-1640 (2008).
90. Akselrod, S. [Components of heart rate variability: basic studies.] Heart rate Variability [Malik, M. (ed.), Camm, A. J. (ed.)] [147-163] (Futura Publishing Company, New York, 1995).
91. Kearns, M. & Dana, R. Algorithmic stability and sanity-check bounds for leaveone- out cross-validation. Neural Comput. 11, 1427-1453 (1999).
92. Malliani, A. Association of heart rate variability components with physiological regulatory mechanisms. Heart rate Variability [Malik, M. (ed.), Camm, A. J. (ed.)] [173-188] (Futura Publishing Company, New York, 1995).
93. Malpas, S. Neural influences on cardiovascular variability: possibilities and pitfalls. Am. J. Physiol.-Heart C. 282, H6 (2002).
94. Ursino, M. & Magosso, E. Role of short-term cardiovascular regulation in heart period variability: a modeling study. Am. J. Physiol.-Heart C. 284, H1479 (2003).
95. Brillinger, D. The identification of polynomial systems by means of higher order spectra. J. Sound Vib. 12, 301-313 (1970).
96. Priestley, M. Spectral analysis and time series (Academic Press, London, UK, 1981).
97. Codispoti, M., Bradley, M. & Lang, P. J. Affective reactions to briefly presented pictures. Psychophysiology 38, 474-478 (2001). (Pubitemid 32422290)
98. Smith, J., Low, A., Bradley, M. & Lang, P. J. Rapid picture presentation and affective engagement. Emotion 6, 208-214 (2006). (Pubitemid 44060356)