Opportunistic and Context-Aware Affect Sensing on Smartphones

IEEE Pervasive Computing

Volumn 15, Issue 2, 2016, Pages 60-69

  Rana, Rajib ^a   Hume, Margee ^b   Reilly, John ^c   Jurdak, Raja ^d   Soar, Jeffrey ^e  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

^b Universiaty of Central Queensland   (Australia)

^c UNIVERSITY OF MELBOURNE   (Australia)

^d CSIRO   (Australia)

^e UNIVERSITY OF SOUTHERN QUEENSLAND   (Australia)

Author keywords

DSP coprocessor; GPU; healthcare; mental health; mobile; opportunistic affect sensing; pervasive computing; smartphone

Indexed keywords

HEALTH; HEALTH CARE; SIGNAL ENCODING; UBIQUITOUS COMPUTING;

AUDIO-VISUAL STIMULUS; CO-PROCESSORS; CONTEXTUAL INFORMATION; MENTAL HEALTH; MOBILE; OPPORTUNISTIC AFFECT SENSING; SENSING TECHNOLOGY; TECHNICAL CHALLENGES;

SMARTPHONES;

EID: 84964324877     PISSN: 15361268     EISSN: None     Source Type: Journal    
DOI: 10.1109/MPRV.2016.36     Document Type: Article

References (50)

1. Z. Zeng et al., "A Survey of Affect Recognition Methods: Audio, Visual, Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, 2009, pp. 39-58.
2. R. LiKamWa et al., "Moodscope: Building a Mood Sensor from Smartphone Usage Patterns," Proc. 11th Ann. Int'l Conf. Mobile Systems, Applications, Services, 2013, pp. 389-402.
3. P. Georgiev et al., "DSP. Ear: Leveraging Co-Processor Support for Continuous Audio Sensing on Smartphones," Proc. 12th ACM Conf. Embedded Network Sensor Systems, 2014, pp. 295-309.
4. K.-T. Cheng and Y.-C. Wang, "Using Mobile GPU for General-Purpose Computing-A Case Study of Face Recognition on Smartphones," Proc. 2011 Int'l Symp. VLSI Design, Automation and Test (VLSI-DAT), 2011, pp. 1-4.
5. N.D. Lane, "A Survey of Mobile Phone Sensing," IEEE Communications Magazine, vol. 48, no. 9, 2010, pp. 140-150.
6. A. Kolakowska, "A Review of Emotion Recognition Methods Based on Keystroke Dynamics and Mouse Movements," Proc. 2013 6th Int'l Conf. Human System Interaction, 2013, pp. 103-115.
7. R.W. Picard, "Affective Computing for HCI," Human-Computer Interaction, 1999, vol. 1, pp. 829-833.
8. P. Ekman et al., Emotion in the Human Face: Guidelines for Research and an Integration of Findings, vol. 11, Pergamon, 1972.
9. M.S. Harris and E. Thackery, The Gale Encyclopedia of Mental Disorders, Gale, 2003.
10. P. Ekman, "Universals and Cultural Differences in Facial Expressions of Emotion," Nebraska Symp. Motivation, Univ. Nebraska Press, 1971, pp. 712-717.
11. J.A. Russell, "A Circumplex Model of Affect," J. Personality and Social Psychology, vol. 39, no. 6, 1980, p. 1161.
12. G. Bauer and P. Lukowicz, "Can Smartphones Detect Stress-Related Changes in the Behaviour of Individuals" 2012 IEEE Int'l Conf. Pervasive Computing and Communications Workshops (PERCOM Workshops), 2012, pp. 423-426.
13. A. Bogomolov, B. Lepri, F. Pianesi, "Happiness Recognition from Mobile Phone Data," 2013 Int'l Conf. Social Computing (SocialCom), 2013, pp. 790-795.
14. R.R. McCrae and O.P. John, "An Introduction to the Five-Factor Model and Its Applications," J. Personality, vol. 60, no. 2, 1992, pp. 175-215.
15. H. Lee et al., "Towards Unobtrusive Emotion Recognition for Affective Social Communication," Proc. 2012 IEEE Consumer Communications and Networking Conf. (CCNC), 2012, pp. 260-264.
16. Y. Gao, N. Bianchi-Berthouze, H. Meng, "What Does Touch Tell Us about Emotions in Touchscreen-Based Gameplay" ACM Trans. Computer-Human Interaction (TOCHI), vol. 19, no. 4, 2012, p. 31.
17. A. Doryab et al., "Detection of Behavior Change in People with Depression," Proceedings of AAAI Workshop on Modern Artificial Intelligence for Health Analytics (MAIHA), 2014.
18. S. Vhaduri et al., "Estimating Drivers' Stress from GPS Traces," Proc. 6th Int'l Conf. Automotive User Interfaces and Interactive Vehicular Applications, 2014, pp. 1-8.
19. V. Alexandratos, Mobile Real-Time Stress Detection, Delft Univ. Technology, 2014.
20. A. Sano and R.W. Picard, "Stress Recognition Using Wearable Sensors and Mobile Phones," Proc. 5th Int'l Humaine Assoc. Conf. Affective Computing and Intelligent Interaction (ACII), 2013, pp. 671-676.
21. A. Grunerbl et al., "Smart-Phone Based Recognition of States and State Changes in Bipolar Disorder Patients," IEEE J. Biomedical and Health Informatics, vol. 19, no. 1, 2014, pp. 140-148.
22. S.C. Reid et al., "A Mobile Phone Program to Track Young People's Experiences of Mood, Stress and Coping," Social Psychiatry and Psychiatric Epidemiology, vol. 44, no. 6, 2009, pp. 501-507.
23. F. Španiel et al., "ITAREPS: Information Technology Aided Relapse Prevention Programme in Schizophrenia," Schizophrenia Research, 2008, vol. 98, no. 1, 2008, pp. 312-317.
24. R. Yamashita, S. Yamaguchi, K. Takami, "A Method of Inferring the Preferences and Mood of Mobile Phone Users by Analyzing Pictograms and Emoticons Used in Their Emails," Proc. 2010 3rd Int'l Conf. Advances in Human-Oriented and Personalized Mechanisms, Technologies and Services (CENTRIC), 2010, pp. 67-72.
25. S. Jimez-Serrano, S. Tortajada, J.M. Garc-Gez, "A Mobile Health Application to Predict Postpartum Depression Based on Machine Learning," Telemedicine and e-Health, 2015, pp. 567-574.
26. A.T. Campbell et al., "People-Centric Urban Sensing," Proc. 2nd Ann. Int'l Workshop Wireless Internet, 2006, pp. 2-5.
27. M.E. Hoque, R. El Kaliouby, R.W. Picard, "When Human Coders (and Machines) Disagree on the Meaning of Facial Affect in Spontaneous Videos," Intelligent Virtual Agents, Springer, 2009, pp. 14-16.
28. "This App Identifi es the Movie You're Watching and Lets You Buy What You See," WIRED Magazine, Dec. 2014.
29. A. Madan and A. Pentland, "Vibefones: Socially Aware Mobile Phones," Proc. 2006 10th IEEE Int'l Symp. Wearable Computers, 2006, pp . 109-112.
30. K.-Y. Lam et al., "SmartMood: Toward Pervasive Mood Tracking and Analysis for Manic Episode Detection," IEEE Trans. Human-Machine Systems, vol. 45, no. 1, 2014, pp. 126-131.
31. H. Lu et al., "StressSense: Detecting Stress in Unconstrained Acoustic Environments Using Smartphones," Proc. 2012 ACM Conf. Ubiquitous Computing, 2012, pp. 351-360.
32. R. Wang et al., "StudentLife: Assessing Mental Health, Academic Performance and Behavioral Trends of College Students Using Smartphones," Proc. 2014 ACM Int'l Joint Conf. Pervasive and Ubiquitous Computing, 2014, pp. 3-14.
33. K.-H. Chang et al., "AMMON: A Speech Analysis Library for Analyzing Affect, Stress, Mental Health on Mobile Phones," Proc. PhoneSense, 2011.
34. P. Prociow, K. Wac, J. Crowe, "Mobile Psychiatry: Towards Improving the Care for Bipolar Disorder," Int'l J. Mental Health Systems, vol. 6, no. 1, 2012, pp. 1-10.
35. K.-S. Cho, I.-H. Choi, Y.-G. Kim, "Robust Facial Expression Recognition Using a Smartphone Working against Illumination Variation," Applied Mathematics, vol. 6, no. 2S, 2012, pp. 403S-408S.
36. M.N. Burns et al., "Harnessing Context Sensing to Develop a Mobile Intervention for Depression," J. Medical Internet Research, vol. 13, no. 3, 2011.
37. E. Maier et al., "A Mobile Solution for Stress Recognition and Prevention," Proc. Int'l Conf. Health Informatics (Health-Inf), 2014, pp. 428-433.
38. A. Gaggioli et al., "A Mobile Data Collection Platform for Mental Health Research," Personal and Ubiquitous Computing, vol. 17, no. 2, 2013, pp. 241-251.
39. N.D. Lane et al., "BeWell: A Smartphone Application to Monitor, Model and Promote Wellbeing," Proc. 5th Int'l ICST Conf. Pervasive Computing Technologies for Healthcare, 2011, pp. 345-359.
40. A. Muaremi, B. Arnrich, G. Trter, "Towards Measuring Stress with Smartphones and Wearable Devices During Workday and Sleep," BioNanoScience, vol. 3, no. 2, 2013, pp. 172-183.
41. D. Christin et al., "A Survey on Privacy in Mobile Participatory Sensing Applications," J. Systems and Software, vol. 84, no. 11, 2011, pp. 1928-1946.
42. H. Lu et al., "SpeakerSense: Energy Effi-cient Unobtrusive Speaker Identifi cation on Mobile Phones," Pervasive Computing, 2011, pp. 188-205.
43. D. Chu et al., "Balancing Energy, Latency and Accuracy for Mobile Sensor Data Classifi cation," Proc. 9th ACM Conf. Embedded Networked Sensor Systems, 2011.
44. P.A. Tresadern, M.C. Ionita, T.F. Cootes, "Real-Time Facial Feature Tracking on a Mobile Device," Int'l J. Computer Vision, vol. 96, no. 3, 2012, pp. 280-289.
45. C. Shan, S. Gong, P.W. McOwan, "Robust Facial Expression Recognition Using Local Binary Patterns," Proc. IEEE Int'l Conf. Image Processing (ICIP), 2005, pp. 370-373.
46. J. Wright et al., "Robust Face Recognition via Sparse Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, 2009, vol. 31, no. 2, pp. 210-227.
47. B. Wei et al., "Distributed Sparse Approximation for Frog Sound Classifi cation," Proc. 11th Int'l Conf. Information Processing in Sensor Networks, 2012, pp. 105-106.
48. E. Maier, U. Reimer, M. Ridinger, "Smart Coping with Stress: Biofeedback via Smart Phone for Stress Reduction and Relapse Prevention in Alcohol Dependent Subjects," Proc. 24th Bled eConf., 2011.
49. M.X. Patel, V. Doku, L. Tennakoon, "Challenges in Recruitment of Research Participants," Advances in Psychiatric Treatment, vol. 9, no. 3, 2003, pp. 229-238.
50. R.B. Gul and P.A. Ali, "Clinical Trials: The Challenge of Recruitment and Retention of Participants," J. Clinical Nursing, vol. 19, no. 1-2, 2010, pp. 227-233.