Smartphone sensing methods for studying behavior in everyday life

Current Opinion in Behavioral Sciences

Volumn 18, Issue , 2017, Pages 83-90

  Harari, Gabriella M ^a   Müller, Sandrine R ^b   Aung, Min SH ^c   Rentfrow, Peter J ^b  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^c Department of Obstetrics Gynecology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIOR; BEHAVIOR ASSESSMENT; DAILY LIFE ACTIVITY; HUMAN; MOVEMENT (PHYSIOLOGY); PHYSICAL ACTIVITY; PRIORITY JOURNAL; REVIEW; SENSOR; SMARTPHONE; SOCIAL BEHAVIOR; SOCIAL INTERACTION;

EID: 85028757195     PISSN: None     EISSN: 23521546     Source Type: Journal    
DOI: 10.1016/j.cobeha.2017.07.018     Document Type: Review

References (73)

1. Furr, R.M., Personality psychology as a truly behavioural science. Eur J Personal 23 (2009), 369–401.
2. Funder, D.C., Sneed, C.D., Behavioral manifestations of personality: an ecological approach to judgmental accuracy. J Pers Soc Psychol 64 (1993), 479–490.
3. Gosling, S.D., John, O.P., Craik, K.H., Robins, R.W., Do people know how they behave? Self-reported act frequencies compared with on-line codings by observers. J Pers Soc Psychol 74 (1998), 1337–1349.
4. Letzring, T.D., Wells, S.M., Funder, D.C., Information quantity and quality affect the realistic accuracy of personality judgment. J Pers Soc Psychol 91 (2006), 111–123.
5. Reis, H.T., Why researchers should think “real-world”: a conceptual rationale. Mehl, M.R., Connor, T.S., (eds.) Handbook of Research Methods for Studying Daily Life, 2012, Guilford, New York.
6. Craik, K.H., The lived day of an individual: a person-environment perspective. Walsh, W.B., Craik, K.H., Price, R.H., (eds.) Person-Environment Psychology: New Directions and Perspectives, 2000, Erlbaum, Mahwah, NJ, 233–266.
7. Mehl, M.R., Gosling, S.D., Pennebaker, J.W., Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life. J Pers Soc Psychol, 90, 2006, 862.
8. Baumeister, R.F., Vohs, K.D., Funder, D.C., Psychology as the science of self-reports and finger movements: whatever happened to actual behavior?. Perspect Psycholog Sci 2 (2007), 396–403.
9. Paulhus, D.L., Vazire, S., The self-report method. Handbook of Research Methods in Personality Psychology, 2007, 224–239.
10. Vazire, S., Informant reports: a cheap, fast, and easy method for personality assessment. J Res Pers 40 (2006), 472–481.
11. Campbell, A.T., Eisenman, S.B., Lane, N.D., Miluzzo, E., Peterson, R.A., Lu, H., Zheng, X., Musolesi, M., Fodor, K., Ahn, G.S., The rise of people-centric sensing. IEEE Internet Comput 12 (2008), 12–21.
12. Lane, N.D., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., Campbell, A.T., A survey of mobile phone sensing. IEEE Commun Mag 48 (2010), 140–150, 10.1109/MCOM.2010.5560598.
13. Lathia, N., Rachuri, K.K., Mascolo, C., Rentfrow, P.J., Contextual dissonance: design bias in sensor-based experience sampling methods. Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2013, ACM, 183–192.
14. Miller, G., The smartphone psychology manifesto. Perspect Psycholog Sci 7 (2012), 221–237.
15. Harari, G.M., Lane, N.D., Wang, R., Crosier, B.S., Campbell, A.T., Gosling, S.D., Using smartphones to collect behavioral data in psychological science: opportunities, practical considerations, and challenges. Perspect Psycholog Sci 11 (2016), 838–854 The authors give an overview of different types of smartphone data and the behaviors they capture, describe areas of opportunity for psychological research using smartphone sensing methods, and describe key study design decisions and challenges to consider when conducting a smartphone sensing study.
16. Pew Research Center. Smartphone ownership and Internet usage continues to climb in emerging economies: but advanced economies still have higher rates of technology use. 2016, Jacob Poushter, Washington, DC Retrieved from http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/.
17. Rachuri, K.K., Mascolo, C., Musolesi, M., Rentfrow, P.J., Sociablesense: exploring the trade-offs of adaptive sampling and computation offloading for social sensing. Proceedings of the 17th Annual International Conference on Mobile Computing and Networking, 2011, ACM, 73–84.
18. Sandstrom, G.M., Lathia, N., Mascolo, C., Rentfrow, P.J., Putting mood in context: using smartphones to examine how people feel in different locations. J Res Personal, 2016.
19. Wrzus, C., Brandmaier, A.M., von Oertzen, T., Müller, V., Wagner, G.G., Riediger, M., A new approach for assessing sleep duration and postures from ambulatory accelerometry. PLoS ONE, 7, 2012, e48089, 10.1371/journal.pone.0048089.
20. Mast, M.S., Gatica-Perez, D., Frauendorfer, D., Nguyen, L., Choudhury, T., Social sensing for psychology automated interpersonal behavior assessment. Curr Dir Psycholog Sci 24 (2015), 154–160.
21. Harari GM, Müller SR, Gosling SD: Naturalistic assessment of situations using mobile sensing methods. Oxford Handbook of Psychological Situations.
22. Wang, R., Aung, M.S., Abdullah, S., Brian, R., Campbell, A.T., Choudhury, T., …, Tseng, V.W., CrossCheck: toward passive sensing and detection of mental health changes in people with schizophrenia. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, ACM, 886–897.
23. Wang, Q., Guo, B., Peng, G., Zhou, G., Yu, Z., CrowdWatch: pedestrian safety assistance with mobile crowd sensing. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, 2016, ACM, 217–220 The authors present CrowdWatch, a smartphone-based system designed to alert pedestrians to dangers in their environment. The system uses acceleration, phone orientation, and GPS location to identify overpasses, underpasses, traffic lights and dynamic barriers.
24. Kobayashi, T., Boase, J., Suzuki, T., Suzuki, T., Emerging from the cocoon? Revisiting the tele-cocooning hypothesis in the smartphone era. J Comput Mediat Commun 20 (2015), 330–345.
25. Lu, H., Pan, W., Lane, N.D., Choudhury, T., Campbell, A.T., SoundSense: scalable sound sensing for people-centric applications on mobile phones. Proceedings of the 7th International Conference on Mobile Systems, Applications, and Services, 2009, ACM, 165–178.
26. Miluzzo, E., Lane, N.D., Fodor, K., Peterson, R., Lu, H., Musolesi, M., Eisenman, S.B., Zheng, X., Campbell, A.T., Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application. Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems, 2008, ACM, 337–350, 10.1145/1460412.1460445.
27. Lu, H., Yang, J., Liu, Z., Lane, N.D., Choudhury, T., Campbell, A.T., The Jigsaw continuous sensing engine for mobile phone applications. Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems, 2010, ACM, 71–84.
28. Tseng, V.W., Merrill, M., Wittleder, F., Abdullah, S., Aung, M.H., Choudhury, T., Assessing mental health issues on college campuses: preliminary findings from a pilot study. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, 2016, ACM, 1200–1208 The paper shows how sleep, stress, and activity routines can be captured using passively sensed smartphone data fluctuate across the week, term, exam and break times for 22 students during an academic semester.
29. Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., …, Campbell, A.T., StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2014, 3–14 The paper presents relationships between mental health, educational outcomes, and passively collected behavioral data from smartphone sensors (e.g. sleep, daily activity, conversation patterns) for a cohort of 48 college students during a 10-week academic term. The StudentLife dataset is publicly available.
30. Lathia, N., Sandstrom, G.M., Mascolo, C., Rentfrow, P.J., Happier people live more active lives: using smartphones to link happiness and physical activity. PLoS ONE, 12, 2017, e0160589 This paper presents relationships between physical activity (measured via the smartphone's accelerometer) and happiness (self-reported via the application) in a sample with over 10,000 participants. The authors include information about validating accelerometer data and showcase participants’ diurnal patterns.
31. Wang, R., Harari, G., Hao, P., Zhou, X., Campbell, A.T., SmartGPA: how smartphones can assess and predict academic performance of college students. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, ACM, 295–306 This paper shows how automatically inferred behaviors from smartphones (e.g. activity, conversational interaction, mobility, class attendance, studying, and partying) can accurately predict GPA (within ±0.179 of the reported grade) for a cohort of 48 students across a 10 week term.
32. Hemminki, S., Nurmi, P., Tarkoma, S., Accelerometer-based transportation mode detection on smartphones. Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems, 2013, ACM, 13.
33. Mun, M., Reddy, S., Shilton, K., Yau, N., Burke, J., Estrin, D., …, Boda, P., PEIR, the personal environmental impact report, as a platform for participatory sensing systems research. Proceedings of the 7th International Conference on Mobile Systems, Applications, and Services, 2009, ACM, 55–68.
34. Biagioni, J., Krumm, J., Days of our lives: assessing day similarity from location traces. International Conference on User Modeling, Adaptation, and Personalization, 2013, Springer, Berlin, Heidelberg, 89–101.
35. Eagle, N., Pentland, A.S., Eigenbehaviors: identifying structure in routine. Behav Ecol Sociobiol 63 (2009), 1057–1066.
36. Saeb, S., Zhang, M., Karr, C.J., Schueller, S.M., Corden, M.E., Kording, K.P., Mohr, D.C., Mobile phone sensor correlates of depressive symptom severity in daily-life behavior: an exploratory study. J Med Internet Res, 17, 2015, e175, 10.2196/jmir.4273 This paper presents behavioral markers related to depression, such as regularity of circadian movement, mobility between favorite locations, location variance, phone usage duration and frequency. Presence of depressive symptoms in a sample of 28 participants over 2 weeks could be predicted with 86.5% accuracy.
37. Chon, J., Cha, H., Lifemap: a smartphone-based context provider for location-based services. IEEE Pervasive Comput 10 (2011), 58–67.
38. Canzian, L., Musolesi, M., Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, ACM, 1293–1304 This paper presents significant correlations between depressive mood and mobility trace characteristics (e.g., total distance covered, max. distance between locations, max. distance from home, number of different places visited, routine index) for 46 participants.
39. Chow, P.I., Fua, K., Huang, Y., Bonelli, W., Xiong, H., Barnes, L.E., Teachman, B.A., Using mobile sensing to test clinical models of depression, social anxiety, state affect, and social isolation among college students. J Med Internet Res, 19, 2017, e62.
40. Abdullah, S., Matthews, M., Frank, E., Doherty, G., Gay, G., Choudhury, T., Automatic detection of social rhythms in bipolar disorder. J Am Medical Inf Assoc 23 (2016), 538–543 Automated sensing of location, distance traveled, conversation frequency and time spent in transit were used to infer stability and rhythmicity of daily schedules for 7 participants with bipolar disorder over 4 weeks. Personalized models predicted stable and unstable states with high accuracy (precision of.85).
41. Abdullah, S., Murnane, E.L., Matthews, M., Kay, M., Kientz, J.A., Gay, G., Choudhury, T., Cognitive rhythms: unobtrusive and continuous sensing of alertness using a mobile phone. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, ACM This paper present an approach to predicting alertness from phone usage (e.g. usage duration, average time between usage sessions, short session frequency) and energy level (from time of day and sleep time) for 20 participants who were followed over the course of 40 days.
42. Mehl, M.R., Pennebaker, J.W., Crow, D.M., Dabbs, J., Price, J.H., The Electronically Activated Recorder (EAR): a device for sampling naturalistic daily activities and conversations. Behav Res Methods 33 (2001), 517–523.
43. Lu, H., Yang, J., Liu, Z., Lane, N.D., Choudhury, T., Campbell, A.T., The Jigsaw continuous sensing engine for mobile phone applications. Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems, 2010, ACM, 71–84.
44. Lu, H., Frauendorfer, D., Rabbi, M., Mast, M.S., Chittaranjan, G.T., Campbell, A.T., …, Choudhury, T., Stresssense: detecting stress in unconstrained acoustic environments using smartphones. Proceedings of the 2012 ACM Conference on Ubiquitous Computing, 2012, ACM, 351–360.
45. Rachuri, K.K., Musolesi, M., Mascolo, C., Rentfrow, P.J., Longworth, C., Aucinas, A., EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. Proceedings of the 12th ACM International Conference on Ubiquitous Computing, 2010, ACM, 281–290.
46. Chen, Z., Chen, Y., Hu, L., Wang, S., Jiang, X., Ma, X., Lane, N.D., Campbell, A.T., ContextSense: unobtrusive discovery of incremental social context using dynamic bluetooth data. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, 2014, ACM, 23–26, 10.1145/2638728.2638801.
47. Vanderhulst, G., Mashhadi, A., Dashti, M., Kawsar, F., Detecting human encounters from wifi radio signals. Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia, 2015, ACM, 97–108.
48. Chittaranjan, G., Blom, J., Gatica-Perez, D., Who's who with big-five: analyzing and classifying personality traits with smartphones. 15th Annual International Symposium Wearable Computers (ISWC), 2011, IEEE, 29–36.
49. Chittaranjan, G., Blom, J., Gatica-Perez, D., Mining large-scale smartphone data for personality studies. Pers Ubiquitous Comput 17 (2013), 433–450.
50. Eagle, N., Pentland, A.S., Reality mining: sensing complex social systems. Pers Ubiquitous Computi 10 (2006), 255–268.
51. Boase, J., Ling, R., Measuring mobile phone use: self report versus log data. J Comput Mediat Commun 18 (2013), 508–519.
52. Mehrotra, A., Müller, S.R., Harari, G.M., Gosling, S.D., Mascolo, C., Musolesi, M., Rentfrow, J., Understanding the role of places and activities on mobile phone interaction and usage patterns. Proceedings of the ACM on Interactive, Multimedia, Wearable and Ubiquitous Technologies (PACM IMWUT), 2017 This paper presents relationships between users’ receptivity and attentiveness towards new notifications, the location they are in, and the activity they are currently engaged in. For this study, 26 participants were monitored for 2 weeks.
53. Min, J.K., Wiese, J., Hong, J.I., Zimmerman, J., Mining smartphone data to classify life-facets of social relationships. Proceedings of the 2013 Conference on Computer Supported Cooperative Work, 2013, ACM, 285–294.
54. Welke, P., Andone, I., Blaszkiewicz, K., Markowetz, A., Differentiating smartphone users by app usage. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, ACM, 519–523.
55. Zhao, S., Ramos, J., Tao, J., Jiang, Z., Li, S., Wu, Z., Pan, G., Dey, A.K., Discovering different kinds of smartphone users through their application usage behaviors. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, ACM, 498–509.
56. Ferdous, R., Osmani, V., Mayora, O., Smartphone app usage as a predictor of perceived stress levels at workplace. 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2015, IEEE, 225–228.
57. Murnane, E.L., Abdullah, S., Matthews, M., Choudhury, T., Gay, G., Social (media) jet lag: how usage of social technology can modulate and reflect circadian rhythms. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, ACM, 843–854 This paper presents sleep sensing technology that can be used to predict sleep events and disruptions from daily phone usage patterns, and presents relationships between the sleep characteristics, cognitive performance, mood, and attention of 9 participants over 3 months.
58. Barata, F., Kowatsch, T., Tinschert, P., Filler, A., Personal MobileCoach: tailoring behavioral interventions to the needs of individual participants. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, 2016, ACM, 1089–1094 The paper presents an application that detects coughing via participants' smartphone microphone (detection accuracy 83.3%) based on supervised learning and is able to trigger individualised interventions based on the number of coughs observed. To train the system, 5 participants provided a total of 80 coughs.
59. Casaseca-de-la-Higuera, P., Lesso, P., McKinstry, B., Pinnock, H., Rabinovich, R., McCloughan, L., Monge-Álvarez, J., Effect of downsampling and compressive sensing on audio-based continuous cough monitoring. 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2015, IEEE, 6231–6235.
60. Sun, X., Lu, Z., Hu, W., Cao, G., SymDetector: detecting sound-related respiratory symptoms using smartphones. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, ACM, 97–108.
61. Korpela, J., Miyaji, R., Maekawa, T., Nozaki, K., Tamagawa, H., Evaluating tooth brushing performance with smartphone sound data. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, ACM, 109–120.
62. Jebara, S.B., Bio-mechanical characterization of voice for smoking detection. Proceedings of the 22nd European Signal Processing Conference (EUSIPCO), 2014, IEEE, 2475–2479.
63. Abdullah, S., Matthews, M., Murnane, E.L., Gay, G., Choudhury, T., Towards circadian computing: early to bed and early to rise makes some of us unhealthy and sleep deprived. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2014, ACM, 673–684.
64. Chen, Z., Lin, M., Chen, F., Lane, N.D., Cardone, G., Wang, R., Li, T., Chen, Y., Choudhury, T., Campbell, A.T., Unobtrusive sleep monitoring using smartphones. In the 2013 7th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2013, IEEE, 145–152.
65. Murnane, E.L., Abdullah, S., Matthews, M., Kay, M., Kientz, J.A., Choudhury, T., Gay, G., Cosley, D., Mobile manifestations of alertness: connecting biological rhythms with patterns of smartphone app use. Proceedings of the 18th International Conference on Human–Computer Interaction with Mobile Devices and Services, 2016, ACM This paper shows how phone usage patterns vary as a function of individual body clock types, alertness, and sleep characteristics for 20 participants over 40 days.
66. Mehrotra, A., Hendley, R., Musolesi, M., PrefMiner: mining user's preferences for intelligent mobile notification management. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, ACM, 1223–1234.
67. Mehrotra, A., Hendley, R., Musolesi, M., Towards multi-modal anticipatory monitoring of depressive states through the analysis of human-smartphone interaction. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, 2016, ACM, 1132–1138.
68. LiKamWa, R., Liu, Y., Lane, N.D., Zhong, L., Moodscope: building a mood sensor from smartphone usage patterns. Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services, 2013, ACM, 389–402.
69. Pielot, M., Dingler, T., Pedro, J.S., Oliver, N., When attention is not scarce-detecting boredom from mobile phone usage. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, ACM, 825–836.
70. Rabbi, M., Ali, S., Choudhury, T., Berke, E., Passive and in-situ assessment of mental and physical well-being using mobile sensors. Proceedings of the 13th International Conference on Ubiquitous Computing, 2011, ACM, 385–394.
71. Yan, Z., Yang, J., Tapia, E.M., Smartphone bluetooth based social sensing. Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, 2013, ACM, 95–98.
72. Werner, M., Kessel, M., Marouane, C., Indoor positioning using smartphone camera. 2011 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 2011, IEEE, 1–6.
73. Jones, S.L., Ferreira, D., Hosio, S., Goncalves, J., Kostakos, V., Revisitation analysis of smartphone app use. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, ACM, 1197–1208.