Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Personal and Ubiquitous Computing

Volumn 19, Issue 7, 2015, Pages 1197-1212

  Kanjo, Eiman ^a   Al Husain, Luluah ^b   Chamberlain, Alan ^c  

^a NOTTINGHAM TRENT UNIVERSITY   (United Kingdom)

^b KING SAUD UNIVERSITY   (Saudi Arabia)

^c UNIVERSITY OF NOTTINGHAM   (United Kingdom)

Author keywords

Affect sensing; Affective computing; Applications; Context; Emotion recognition; Mobile; Mobile sensing; Pervasive computing; Ubiquitous computing

Indexed keywords

APPLICATIONS; DATA VISUALIZATION; REAL TIME SYSTEMS; SPEECH RECOGNITION; SURVEYS;

AFFECT SENSING; AFFECTIVE COMPUTING; CONTEXT; EMOTION RECOGNITION; MOBILE; MOBILE SENSING;

UBIQUITOUS COMPUTING;

EID: 84942366351     PISSN: 16174909     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00779-015-0842-3     Document Type: Article

References (75)

1. Jerritta S, Murugappan M, Nagarajan R, Wan K (2011) Physiological signals based human emotion recognition: a review. In: 2011 IEEE 7th international colloquium on signal processing and its applications (CSPA), pp 410–415
2. Lang PJ (1995) The emotion probe: studies of motivation and attention. Am Psychol 50:372–385
3. Mauss IB, Robinson MD (2009) Measures of emotion: a review. Cogn Emot 23:209–237
4. Peter C, Herbon A (2006) Emotion representation and physiology assignments in digital systems. Interact Comput 18:139–170
5. Picard RW, Papert S, Bender W, Blumberg B, Breazeal C, Cavallo D, Machover T, Resnick M, Roy D, Strohecker C (2004) Strohecker, affective learning—a manifesto. BT Technol J 22:253–269
6. Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25:49–59
7. Desmet P, Overbeeke K, Tax S (2001) Designing products with added emotional value: development and application of an approach for research through design. Des J 4:32–47
8. MacKerron G (2011) mappiness.org.uk
9. Mody RN, Willis KS, Kerstein V (2009) WiMo: location-based emotion tagging. In: Proceedings of the 8th international conference on mobile and ubiquitous multimedia, ACM, p 14
10. Affectiva Q. http://www.affectiva.com/q-sensor/
11. BioHarness. http://www.zephyranywhere.com/products/bioharness-3/
12. Bodymonitor Systeme (BMS). http://www.bodymonitor.de/
13. NeuroSky. http://www.neurosky.com/
14. Ayzenberg Y, Rivera JH, Picard R (2012) FEEL: frequent EDA and event logging—a mobile social interaction stress monitoring system. In: CHI ‘12 extended abstracts on human factors in computing systems, ACM, Austin, pp 2357–2362
15. Ståhl A, Höök K, Svensson M, Taylor AS, Combetto M (2009) Experiencing the affective diary. Personal Ubiquitous Comput 13:365–378
16. Yamamoto J, Kawazoe M, Nakazawa J, Takashio K, Tokuda H (2009) MOLMOD: analysis of feelings based on vital information for mood acquisition. Skin 4:6
17. Perttula A, Tuomi P, Suominen M, Koivisto A, Multisilta J (2010) Users as sensors: creating shared experiences in co-creational spaces by collective heart rate. In: Proceedings of the 14th international academic MindTrek conference: envisioning future media environments, ACM, pp 41–48
18. Matsumoto D, Willingham B (2009) Spontaneous facial expressions of emotion of congenitally and non-congenitally blind individuals. J Pers Soc Psychol 96(1):1–10
19. Valstar M, Pantic M (2006) Fully automatic facial action unit detection and temporal analysis. In: Proceedings of the 2006 conference on computer vision and pattern recognition workshop, June 2006
20. Yang S, Bhanu B (2011) Facial expression recognition using emotion avatar image. In: Recognition and workshops, Santa Barbara, pp 866–871
21. Affdex. http://www.affectiva.com/affdex/
22. Hernandez J Facereader Noduls
23. Hoque ME, Drevo W, Picard RW (2012) Mood meter: counting smiles in the wild. In: Proceedings of the 2012 ACM conference on ubiquitous computing, ACM, pp 301–310
24. Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. Affect Comput IEEE Trans 1:18–37
25. Juslin PN, Scherer KR (2005) Vocal expression of affect. In: Harrigan JA, Rosenthal R, Scherer KR (eds) The new handbook of methods in nonverbal behavior research. Oxford University Press, New York, pp 65–135
26. Bänziger T, Scherer KR (2005) The role of intonation in emotional expression. Speech Commun 46:252–267
27. Lee C, Narayanan S, Pieraccini R (2002) Classifying emotions in human-machine spoken dialogs. Presented at proceedings of international conference on multimedia and Expo, Lausanne, Switzerland, August 2002
28. Lu H, Rabbi M, Chittaranjan GT, Frauendorfer D, Mast MS, Campbell AT, Gatica-Perez D, Choudhury T (2012) StressSense: detecting stress in unconstrained acoustic environments using smartphones. In: Proceedings of 14th international conference ubiquitous computing
29. Rachuri KK, Musolesi M, Mascolo C, Rentfrow PJ, Longworth C, Aucinas A (2010) EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. In: Proceedings of the 12th ACM international conference on ubiquitous computing, ACM, 2010, pp 281–290
30. LiKamWa R, Liu Y, Lane ND, Zhong L (2011) Can your smartphone infer your mood. In: PhoneSense workshop, 2011
31. Yuanchao M, Bin X, Yin B, Guodong S, Run Z (2012) Daily mood assessment based on mobile phone sensing. In: Wearable and implantable body sensor networks (BSN), 2012 ninth international conference on, 2012, pp 142–147
32. Mislove A, Lehmann S, Ahn Y, Onnela J, Rosenquist J (2010) Pulse of the nation: US mood throughout the day inferred from Twitter 2010
33. Noulas A, Mascolo C, Frias-Martinez E (2013) Exploiting foursquare and cellular data to infer user activity in urban environments
34. Kanjo E, El-Mawass N, Craveiro J, Ramos F (2013) Social, disconnected or in between: mobile data reveals urban mood. In: The 3rd international conference on the analysis of mobile phone datasets (NetMob’13), MIT, MA, p 9
35. André E (2011) Experimental methodology in emotion-oriented computing. Pervasive Comput IEEE 10:54–57
36. Lee H, Choi YS, Lee S, Park I (2012) Towards unobtrusive emotion recognition for affective social communication. In: Consumer communications and networking conference (CCNC), 2012 IEEE, IEEE, pp. 260–264
37. Oh K, Park H-S, Cho S-B (2010) A mobile context sharing system using activity and emotion recognition with Bayesian networks. In: 2010 7th international conference on ubiquitous intelligence & computing and 7th international conference on autonomic & trusted computing (UIC/ATC), IEEE, 2010, pp 244–249
38. Kim H-J, Choi YS (2011) EmoSens: affective entity scoring, a novel service recommendation framework for mobile platform. In: Workshop on personalization in mobile application of the 5th international conference on recommender system
39. Al-Barrak L, Kanjo E (2013) NeuroPlace: making sense of a place. In: Proceedings of the 4th augmented human international conference, ACM, pp 186–189
40. Chang DFK-h, Canny J (2011) Ammon: a speech analysis library for analyzing affect, stress, and mental health on mobile phones. In: Proceedings of PhoneSense, 2011
41. Chang K-h, Fisher D, Canny J (2011) Bj, 246, r. Hartmann, How’s my mood and stress?: an efficient speech analysis library for unobtrusive monitoring on mobile phones. In: Proceedings of the 6th international conference on body area networks, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Beijing, 2011, pp 71–77
42. Patil SA, Hansen JH (2007) Speech under stress: analysis, modeling and recognition
43. Ekman P, Friesen WV Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto
44. Ellsworth PC, Smith CA (1988) From appraisal to emotion: differences among unpleasant feelings. Motiv Emot 12(1978):271–302
45. Ekman P (1992) An argument for basic emotions. Cogn Emot 6:169–200
46. Küblbeck C, Ernst A (2006) Face detection and tracking in video sequences using the modifiedcensus transformation. Image Vis Comput 24:564–572
47. Wilhelmer R, Bismarck J, Maus B (2008) Feel-o-meter: Stimmungsgasometer. http://www.fühlometer.de
48. Bradley MM, Lang PJ (1999) Affective norms for English words (ANEW): instruction manual and affective ratings. In: Technical report C-1, the center for research in psychophysiology, University of Florida, 1999
49. Thelwall Mike, Buckley Kevan, Paltoglou Georgios, Cai Di, Kappas Arvid (2010) Sentiment in short strength detection informal text. JASIST 61(12):2544–2558
50. Matei S, Ball-Rokeach SJ, Qiu JL (2001) Fear and misperception of Los Angeles urban space: a spatial-statistical study of communication-shaped mental maps. Commun Res 28:429–463
51. Nold C (2006) Bio mapping. http://biomapping.net/
52. Pressman SD, Cohen S (2005) Does positive affect influence health? Psychol Bull 131:925
53. Fogg B (2007) Mobile persuasion: 20 perspectives on the future of behavior change. Mobile Persuasion 2007
54. Gay G, Pollak J, Adams P, Leonard J (2011) Pilot study of aurora, a social, mobile-phone-based emotion sharing and recording system. J Diabetes Sci Technol 5:325–332
55. Chang T-R, Kaasinen E, Kaipainen K (2013) Persuasive design in mobile applications for mental well-being: multidisciplinary expert review. In: Godara B, Nikita K (eds) Wireless mobile communication and healthcare. Springer, Berlin Heidelberg, pp 154–162
56. Dickerson RF, Gorlin EI, Stankovic JA (2011) Empath: a continuous remote emotional health monitoring system for depressive illness. In: Proceedings of the 2nd conference on wireless health, ACM, 2011, p 5
57. O.A.P. Ltd., Optimism apps (2011)
58. Burns MN, Begale M, Duffecy J, Gergle D, Karr CJ, Giangrande E, Mohr DC (2011) Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res 13(3):e55
59. Gaggioli A, Pioggia G, Tartarisco G, Baldus G, Corda D, Cipresso P, Riva G (2013) A mobile data collection platform for mental health research. Personal Ubiquitous Comput 17:241–251
60. Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw W (2010) Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. J Med Internet Res 12(2):e10
61. Bergner BS, Exner J-P, Zeile P, Rumberg M (2012) Sensing the city–how to identify recreational benefits of urban green areas with the help of sensor technology. In: Proceedings REAL CORP, 2012
62. Klettner S, Huang H, Schmidt M, Gartner G (2013) Crowdsourcing affective responses to space. KN Kartographische Nachrichten. J Cartogr Geogr Inf 2
63. Gartner G (2012) Openemotionmap. org—Emotional response to space as an additional concept in cartography. Int Arch Photogramm Remote Sens Spat Inf Sci (ISPRS) 39-B4:473–476
64. Doerflinger J, Gross T, Lyra O, Karapanos E, Kostakos V, Cuadrado-Cordero I, Soria-Morillo LM, Gonzalez-Abril L, Ortega-Ramirez JA, Pau de la Cruz I (2012) ICTD work, plus mFeel, pervasive computing, IEEE, vol 11, pp 43–45. Replace with Sense of space paper
65. Killingsworth MA, Gilbert DT (2010) A wandering mind is an unhappy mind. Science 330:932
66. Christin D, Reinhardt A, Kanhere SS, Hollick M (2011) A survey on privacy in mobile participatory sensing applications. J Syst Softw 84:1928–1946
67. Crabtree A, Chamberlain A, Grinter R, Jones M, Rodden T, Rogers Y (eds) (2013) Special issue on “The Turn to the Wild Introduction”. ACM Transactions on Computer-Human Interaction—ToCHI 20(3), 13:1–13:4
68. Kanjo E, Alajmi N, El-Mawass N (2013) ShopMobia: emotion based shop rating system, affective computing and intelligent interaction conference. In Submission, Geneva, Switzerland
69. Lane ND, Xu Y, Lu H, Hu S, Choudhury T, Campbell AT, Zhao F (2011) Enabling large-scale human activity inference on smartphones using community similarity networks (csn). In: Proceedings of the 13th international conference on Ubiquitous computing, ACM, 2011, pp 355–364
70. Li Q, Cao G (2013) Providing privacy-aware incentives for mobile sensing. In: IEEE international conference on pervasive computing and communications (PerCom), 2013
71. Ra M-R, Priyantha B, Kansal A, Liu J (2012) Improving energy efficiency of personal sensing applications with heterogeneous multi-processors. In: Proceedings of the 2012 ACM conference on ubiquitous computing (UbiComp ‘12). ACM, New York, pp 1–10. doi:10.1145/2370216.2370218
72. Chamberlain A, Crabtree A, Rodden T, Jones M, Rogers Y (201) Research in the wild: understanding ‘in the wild’ approaches to design and development. In: Conference on designing interactive systems 2012 ACM DIS, ACM Press
73. Crabtree A, Chamberlain A, Grinter RR, Jones M, Rodden T, Rogers Y (eds) (2013) Special issue on “The Turn to the Wild” with authored introduction”. ACM Transactions on Computer-Human Interaction—ToCHI 20(3), 13:1–13:4
74. Crabtree A, Chamberlain A, Davies M, Glover K, Reeves S, Rodden T, Tolmie P, Jones M (2013) “Doing Innovation in the Wild”, CHItaly 2013. Trento, Italy—(ACM Library)
75. Tolmie P, Chamberlain A, Benford S (2014) Designing for Reportability: sustainable gamification, public engagement and promoting environmental debate. In Personal and ubiquitous computing journal, Springer, New York