Emotion recognition using wireless signals

Proceedings of the Annual International Conference on Mobile Computing and Networking, MOBICOM

Volumn 0, Issue 1, 2016, Pages 95-108

  Zhao, Mingmin ^a   Adib, Fadel ^a   Katabi, Dina ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Affective computing; Emotion recognition; Heart rate variability; Wireless sensing; Wireless signals

Indexed keywords

ARTIFICIAL INTELLIGENCE; ELECTROCARDIOGRAPHY; LEARNING SYSTEMS; SPEECH RECOGNITION;

AFFECTIVE COMPUTING; EMOTION RECOGNITION; HEART RATE VARIABILITY; WIRELESS SENSING; WIRELESS SIGNALS;

BIOMEDICAL SIGNAL PROCESSING;

EID: 84994165777     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2973750.2973762     Document Type: Conference Paper

References (65)

1. Microsoft project oxford emotion api. https://www.projectoxford.ai/emotion.
2. Noise robust differentiators for second derivative estimation. http://www.holoborodko.com/pavel/downloads/NoiseRobustSecondDerivative.
3. H. Abdelnasser, M. Youssef, and K. A. Harras. Wigest: A ubiquitous wifi-based gesture recognition system. In Computer Communications (INFOCOM), 2015 IEEE Conference on, pages 1472-1480. IEEE, 2015.
4. U. R. Acharya, K. P. Joseph, N. Kannathal, C. M. Lim, and J. S. Suri. Heart rate variability: a review. Medical and biological engineering and computing, 44(12):1031-1051, 2006.
5. F. Adib, Z. Kabelac, D. Katabi, and R. C. Miller. 3d tracking via body radio reflections. In 11th USENIX Symposium on Networked Systems Design and Implementation, 2014.
6. F. Adib and D. Katabi. See through walls with wifi! In Proceedings of the ACM SIGCOMM 2013, pages 75-86, New York, NY, USA, 2013. ACM.
7. F. Adib, H. Mao, Z. Kabelac, D. Katabi, and R. C. Miller. Smart homes that monitor breathing and heart rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 837-846. ACM, 2015.
8. F. Agrafioti, D. Hatzinakos, and A. K. Anderson. Ecg pattern analysis for emotion detection. Affective Computing, IEEE Transactions on, 3(1):102-115, 2012.
9. G. Alelis, A. Bobrowicz, and C. S. Ang. Exhibiting emotion: Capturing visitors' emotional responses to museum artefacts. In Design, User Experience, and Usability. User Experience in Novel Technological Environments, pages 429-438. Springer, 2013.
10. K. Ali, A. X. Liu, W. Wang, and M. Shahzad. Keystroke recognition using wifi signals. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 90-102. ACM, 2015.
11. L. Anitori, A. de Jong, and F. Nennie. Fmcw radar for life-sign detection. In Radar Conference, 2009 IEEE, pages 1-6. IEEE, 2009.
12. B. M. Appelhans and L. J. Luecken. Heart rate variability as an index of regulated emotional responding. Review of general psychology, 10(3):229, 2006.
13. O. Boric-Lubecke, W. Massagram, V. M. Lubecke, A. Host-Madsen, and B. Jokanovic. Heart rate variability assessment using doppler radar with linear demodulation. In Microwave Conference, 2008. EuMC 2008. 38th European, pages 420-423. IEEE, 2008.
14. R. A. Calvo and S. D'Mello. Affect detection: An interdisciplinary review of models, methods, and their applications. Affective Computing, IEEE Transactions on, 1(1):18-37, 2010.
15. G. Chittaranjan, J. Blom, and D. Gatica-Perez. Who's who with big-five: Analyzing and classifying personality traits with smartphones. In 2011 15th Annual International Symposium on Wearable Computers, pages 29-36. IEEE, 2011.
16. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor. Emotion recognition in human-computer interaction. Signal Processing Magazine, IEEE, 18(1):32-80, 2001.
17. P. De Chazal, E. O. Hare, N. Fox, and C. Heneghan. Assessment of sleep/wake patterns using a non-contact biomotion sensor. In Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE, pages 514-517. IEEE, 2008.
18. P. A. Devijver and J. Kittler. Pattern recognition: A statistical approach, volume 761. Prentice-Hall London, 1982.
19. A. D. Droitcour, O. Boric-Lubecke, and G. T. Kovacs. Signal-to-noise ratio in doppler radar system for heart and respiratory rate measurements. Microwave Theory and Techniques, IEEE Transactions on, 57(10):2498-2507, 2009.
20. A. D. Droitcour, O. Boric-Lubecke, V. M. Lubecke, J. Lin, and G. T. Kovacs. Range correlation and i/q performance benefits in single-chip silicon doppler radars for noncontact cardiopulmonary monitoring. Microwave Theory and Techniques, IEEE Transactions on, 52(3):838-848, 2004.
21. P. Ekman, W. V. Friesen, and P. Ellsworth. Emotion in the human face: Guidelines for research and an integration of findings. Elsevier, 2013.
22. M. El Ayadi, M. S. Kamel, and F. Karray. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition, 44(3):572-587, 2011.
23. H. A. Elfenbein and N. Ambady. On the universality and cultural specificity of emotion recognition: a meta-analysis. Psychological bulletin, 128(2):203, 2002.
24. A. Fernández-Caballero, J. M. Latorre, J. M. Pastor, and A. Fernández-Sotos. Improvement of the elderly quality of life and care through smart emotion regulation. In Ambient Assisted Living and Daily Activities, pages 348-355. Springer, 2014.
25. R. Fletcher and J. Han. Low-cost differential front-end for doppler radar vital sign monitoring. In Microwave Symposium Digest, 2009. MTT'09. IEEE MTT-S International, pages 1325-1328. IEEE, 2009.
26. I. Guyon and A. Elisseeff. An introduction to variable and feature selection. The Journal of Machine Learning Research, 3:1157-1182, 2003.
27. W. Hu, Z. Zhao, Y. Wang, H. Zhang, and F. Lin. Noncontact accurate measurement of cardiopulmonary activity using a compact quadrature doppler radar sensor. Biomedical Engineering, IEEE Transactions on, 61(3):725-735, 2014.
28. S. Jerritta, M. Murugappan, R. Nagarajan, and K. Wan. Physiological signals based human emotion recognition: a review. In Signal Processing and its Applications (CSPA), 2011 IEEE 7th International Colloquium on, pages 410-415. IEEE, 2011.
29. A. Jovic and N. Bogunovic. Electrocardiogram analysis using a combination of statistical, geometric, and nonlinear heart rate variability features. Artificial intelligence in medicine, 51(3):175-186, 2011.
30. S. E. Kahou, X. Bouthillier, P. Lamblin, C. Gulcehre, V. Michalski, K. Konda, S. Jean, P. Froumenty, Y. Dauphin, N. Boulanger-Lewandowski, et al. Emonets: Multimodal deep learning approaches for emotion recognition in video. Journal on Multimodal User Interfaces, pages 1-13, 2015.
31. O. Kaltiokallio, H. Yigitler, R. Jantti, and N. Patwari. Non-invasive respiration rate monitoring using a single cots tx-rx pair. In Information Processing in Sensor Networks, IPSN-14 Proceedings of the 13th International Symposium on, pages 59-69. IEEE, 2014.
32. P. W. Kamen, H. Krum, and A. M. Tonkin. Poincare plot of heart rate variability allows quantitative display of parasympathetic nervous activity in humans. Clinical science, 91(2):201-208, 1996.
33. T. B. Kashdan, A. Mishra, W. E. Breen, and J. J. Froh. Gender differences in gratitude: Examining appraisals, narratives, the willingness to express emotions, and changes in psychological needs. Journal of personality, 77(3):691-730, 2009.
34. J. Kim and E. André. Emotion recognition based on physiological changes in music listening. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(12):2067-2083, 2008.
35. S. D. Kreibig. Autonomic nervous system activity in emotion: A review. Biological psychology, 84(3):394-421, 2010.
36. D. E. Lake, J. S. Richman, M. P. Griffin, and J. R. Moorman. Sample entropy analysis of neonatal heart rate variability. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 283(3):R789-R797, 2002.
37. R. D. Lane, K. McRae, E. M. Reiman, K. Chen, G. L. Ahern, and J. F. Thayer. Neural correlates of heart rate variability during emotion. Neuroimage, 44(1):213-222, 2009.
38. P. J. Lang. The emotion probe: studies of motivation and attention. American psychologist, 50(5):372, 1995.
39. R. LiKamWa, Y. Liu, N. D. Lane, and L. Zhong. Moodscope: building a mood sensor from smartphone usage patterns. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services, pages 389-402. ACM, 2013.
40. W. Massagram, V. M. Lubecke, A. Host-Madsen, and O. Boric-Lubecke. Assessment of heart rate variability and respiratory sinus arrhythmia via doppler radar. Microwave Theory and Techniques, IEEE Transactions on, 57(10):2542-2549, 2009.
41. D. McDuff, R. El Kaliouby, J. F. Cohn, and R. W. Picard. Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads. Affective Computing, IEEE Transactions on, 6(3):223-235, 2015.
42. S. McKinley and M. Levine. Cubic spline interpolation. College of the Redwoods, 45(1):1049-1060, 1998.
43. P. Melillo, M. Bracale, L. Pecchia, et al. Nonlinear heart rate variability features for real-life stress detection. case study: students under stress due to university examination. Biomed Eng Online, 10(1):96, 2011.
44. M. O. Mendez, M. Matteucci, V. Castronovo, L. Ferini-Strambi, S. Cerutti, and A. Bianchi. Sleep staging from heart rate variability: time-varying spectral features and hidden markov models. International Journal of Biomedical Engineering and Technology, 3(3-4):246-263, 2010.
45. N. Patwari, L. Brewer, Q. Tate, O. Kaltiokallio, and M. Bocca. Breathfinding: A wireless network that monitors and locates breathing in a home. Selected Topics in Signal Processing, IEEE Journal of, 8(1):30-42, 2014.
46. T. Penzel, J. W. Kantelhardt, L. Grote, J.-H. Peter, and A. Bunde. Comparison of detrended fluctuation analysis and spectral analysis for heart rate variability in sleep and sleep apnea. Biomedical Engineering, IEEE Transactions on, 50(10):1143-1151, 2003.
47. R. W. Picard. Affective computing: from laughter to ieee. Affective Computing, IEEE Transactions on, 1(1):11-17, 2010.
48. R. W. Picard, E. Vyzas, and J. Healey. Toward machine emotional intelligence: Analysis of affective physiological state. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(10):1175-1191, 2001.
49. O. Postolache, P. S. Girão, G. Postolache, and J. Gabriel. Cardio-respiratory and daily activity monitor based on fmcw doppler radar embedded in a wheelchair. In Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pages 1917-1920. IEEE, 2011.
50. Q. Pu, S. Gupta, S. Gollakota, and S. Patel. Whole-home gesture recognition using wireless signals. In Proceedings of the 19th annual international conference on Mobile computing & networking, pages 27-38. ACM, 2013.
51. D. S. Quintana, A. J. Guastella, T. Outhred, I. B. Hickie, and A. H. Kemp. Heart rate variability is associated with emotion recognition: direct evidence for a relationship between the autonomic nervous system and social cognition. International Journal of Psychophysiology, 86(2):168-172, 2012.
52. D. P. Saha, T. L. Martin, and R. B. Knapp. Towards incorporating affective feedback into context-aware intelligent environments. In Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on, pages 49-55. IEEE, 2015.
53. T. Sakamoto, R. Imasaka, H. Taki, T. Sato, M. Yoshioka, K. Inoue, T. Fukuda, and H. Sakai. Feature-based correlation and topological similarity for interbeat interval estimation using ultra-wideband radar. Biomedical Engineering, IEEE Transactions on, 2015.
54. K. R. Scherer. Vocal communication of emotion: A review of research paradigms. Speech communication, 40(1):227-256, 2003.
55. M. Soleymani, M. Pantic, and T. Pun. Multimodal emotion recognition in response to videos. Affective Computing, IEEE Transactions on, 3(2):211-223, 2012.
56. L. Sun, S. Sen, D. Koutsonikolas, and K.-H. Kim. Widraw: Enabling hands-free drawing in the air on commodity wifi devices. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 77-89. ACM, 2015.
57. J. Sztajzel et al. Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system. Swiss medical weekly, 134:514-522, 2004.
58. D. Tse and P. Viswanath. Fundamentals of wireless communication. Cambridge university press, 2005.
59. W. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 65-76. ACM, 2015.
60. Y. Wang, J. Liu, Y. Chen, M. Gruteser, J. Yang, and H. Liu. E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In Proceedings of the 20th annual international conference on Mobile computing and networking, pages 617-628. ACM, 2014.
61. T. Wei and X. Zhang. mtrack: High-precision passive tracking using millimeter wave radios. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 117-129. ACM, 2015.
62. A. D. Wiens, M. Etemadi, S. Roy, L. Klein, and O. T. Inan. Toward continuous, noninvasive assessment of ventricular function and hemodynamics: Wearable ballistocardiography. Biomedical and Health Informatics, IEEE Journal of, 19(4):1435-1442, 2015.
63. A. Zaffaroni, P. De Chazal, C. Heneghan, P. Boyle, P. R. Mppm, and W. T. McNicholas. Sleepminder: an innovative contact-free device for the estimation of the apnoea-hypopnoea index. In Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE, pages 7091-9094. IEEE, 2009.
64. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(1):39-58, 2009.
65. J. Zhu, S. Rosset, T. Hastie, and R. Tibshirani. 1-norm support vector machines. Advances in neural information processing systems, 16(1):49-56, 2004.