Stress measurement using speech: Recent advancements, validation issues, and ethical and privacy considerations

Stress

Volumn 22, Issue 4, 2019, Pages 408-413

  Slavich, George M ^a   Taylor, Sara ^b   Picard, Rosalind W ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MIT MEDIA LAB   (United States)

Author keywords

digital health; ethics; Life stress; privacy; speech; voice

Indexed keywords

ARTIFICIAL INTELLIGENCE; CONVERSATION; ENERGY; HUMAN; MEDICAL ETHICS; NERVOUSNESS; NOTE; PITCH; PRIORITY JOURNAL; PRIVACY; SOCIAL PSYCHOLOGY; SPEECH; SPEECH ANALYSIS; STRESS; VALIDATION STUDY; DEPRESSION; LONGITUDINAL STUDY; MENTAL STRESS; PSYCHOLOGY;

HYDROCORTISONE;

DEPRESSION; HUMANS; HYDROCORTISONE; LONGITUDINAL STUDIES; PRIVACY; SPEECH; STRESS, PSYCHOLOGICAL;

EID: 85063878574     PISSN: 10253890     EISSN: 16078888     Source Type: Journal    
DOI: 10.1080/10253890.2019.1584180     Document Type: Note

References (54)

1. Allen, A.P., Kennedy, P.J., Cryan, J.F., Dinan, T.G., & Clarke, G. (2014). Biological and psychological markers of stress in humans: Focus on the Trier Social Stress Test. Neuroscience and Biobehavioral Reviews, 38, 94–124. doi:10.1016/j.neubiorev.2013.11.005
2. Brown, D. (2018). Alexa goes to college: Echo Dots move into dorms on campus. USA Today. Retrieved from https://eu.usatoday.com/story/money/2018/09/06/college-students-echo-dots-dorm-rooms/1087251002
3. Burke, S. (2017). Google admits its new smart speaker was eavesdropping on users. CNN. Retrieved from https://money.cnn.com/2017/10/11/technology/google-home-mini-security-flaw/index.html
4. Charlton, A. (2018). Hacked Amazon Echo turned other Echoes on same Wi-Fi network into covert listening devices. GearBrain. Retrieved from https://www.gearbrain.com/defcon-amazon-echo-spy-hack-2595403129.html
5. Cohen, S., Janicki-Deverts, D., & Miller, G.E. (2007). Psychological stress and disease. Journal of the American Medical Association, 298, 1685–1687. doi:10.1001/jama.298.14.1685
6. Cohen, S., Kessler, R., Underwood, G., & Gordon, L. (1997). Measuring stress: A guide for health and social scientists. New York, NY: Oxford University Press.
7. Cummins, N., Scherer, S., Krajewski, J., Schnieder, S., Epps, J., & Quatieri, T.F. (2015). A review of depression and suicide risk assessment using speech analysis. Speech Communication, 71, 10–49. doi:10.1016/j.specom.2015.03.004
8. Dohrenwend, B.P. (2006). Inventorying stressful life events as risk factors for psychopathology: Toward resolution of the problem of intracategory variability. Psychological Bulletin, 132, 477–495. doi:10.1037/0033-2909.132.3.477
9. Epel, E.S., Crosswell, A.D., Mayer, S.E., Prather, A.A., Slavich, G.M., Puterman, E., & Mendes, W.B. (2018). More than a feeling: A unified view of stress measurement for population science. Frontiers in Neuroendocrinology, 49, 146–169. doi:10.1016/j.yfrne.2018.03.001
10. Eyben, F., Weninger, F., Gross, F., & Schuller, B. (2013). Recent developments in openSMILE, the munich open-source multimedia feature extractor. In MM 2013-proceedings of the ACM conference on multimedia (pp. 835–838). Barcelona, Spain: ACM.
11. Fernandez, R., & Picard, R.W. (2003). Modeling drivers’ speech under stress. Speech Communication, 40, 145–159. doi:10.1016/S0167-6393(02)00080-8
12. Geiker, N.R.W., Astrup, A., Hjorth, M.F., Sjödin, A., Pijls, L., & Markus, C.R. (2018). Does stress influence sleep patterns, food intake, weight gain, abdominal obesity and weight loss interventions and vice versa? Obesity Reviews, 19, 81–97. doi:10.1111/obr.12603
13. Giannakakis, G., Pediaditis, M., Manousos, D., Kazantzaki, E., Chiarugi, F., Simos, P.G., … Tsiknakis, M. (2017). Stress and anxiety detection using facial cues from videos. Biomedical Signal Processing and Control, 31, 89–101. doi:10.1016/j.bspc.2016.06.020
14. Giddens, C.L., Barron, K.W., Byrd-Craven, J., Clark, K.F., & Winter, A.S. (2013). Vocal indices of stress: A review. Journal of Voice, 27, 390.e21–390.e29. doi:10.1016/j.jvoice.2012.12.010
15. Han, H., Kyunggeun, B., & Hong-Goo, K. (2018). A deep learning-based stress detection algorithm with speech signal. In Proceedings of the 2018 workshop on audio-visual scene understanding for immersive multimedia (pp. 11–15). Seoul, Korea: ACM.
16. Hansen, J.H.L., & Patil, S.A. (2007). Speech under stress: Analysis, modeling and recognition. In Müller C. (Ed.), Speaker classification I. Lecture notes in computer science, vol. 4343 (pp. 108–137). Berlin, Heidelberg: Springer.
17. Hansen, J.H.L., & Womack, B.D. (1996). Feature analysis and neural network-based classification of speech under stress. IEEE Transactions on Speech and Audio Processing, 4, 307–313. doi:10.1109/89.506935
18. Holt-Lunstad, J., Robles, T.F., & Sbarra, D.A. (2017). Advancing social connection as a public health priority in the United States. American Psychologist, 72, 517–530. doi:10.1037/amp0000103
19. Irwin, M.R., & Slavich, G.M. (2017). Psychoneuroimmunology. In J. T., Cacioppo, L. G., Tassinary, & G. G., Berntson (Eds.), Handbook of psychophysiology, 4th ed. (pp. 377–398). New York: Cambridge University Press.
20. Ivanova, I. (2018). Amazon’s Alexa sent 1,700 recordings to the wrong person. CBS News. Retrieved from https://www.cbsnews.com/news/amazons-alexa-sent-1700-recordings-to-the-wrong-customer/
21. Johnson, L. (2019). Apple issues Group FaceTime bug mea culpa, promises software update next week. Macworld. Retrieved from https://www.macworld.com/article/3336865/ios/facetime-bug-lets-you-listen-in-on-people-you-call.html
22. Juster, R.P., McEwen, B.S., & Lupien, S.J. (2010). Allostatic load biomarkers of chronic stress and impact on health and cognition. Neuroscience and Biobehavioral Reviews, 35, 2–16. doi:10.1016/j.neubiorev.2009.10.002
23. Kelly-Irving, M., Lepage, B., Dedieu, D., Bartley, M., Blane, D., Grosclaude, P., … Delpierre, C. (2013). Adverse childhood experiences and premature all-cause mortality. European Journal of Epidemiology, 28, 721–734. doi:10.1007/s10654-013-9832-9
24. Lu, H., Frauendorfer, D., Rabbi, M., Mast, M.S., Chittaranjan, G.T., Campbell, A.T., … Choudhury, T. (2012). StressSense: Detecting stress in unconstrained acoustic environments using smartphones. In UbiComp 2012. Proceedings of the ACM conference on ubiquitous computing (pp. 351–360). Pittsburgh, PA: ACM.
25. Malat, J., Jacquez, F., & Slavich, G.M. (2017). Measuring lifetime stress exposure and protective factors in life course research on racial inequality and birth outcomes. Stress, 20, 379–385. doi:10.1080/10253890.2017.1341871
26. Mayer, S.E., Prather, A.A., Puterman, E., Lin, J., Arenander, J., Coccia, M., … Epel, E.S. (2019). Cumulative lifetime stress exposure and leukocyte telomere length attrition: The unique role of stressor duration and exposure timing. Psychoneuroendocrinology, 104, 210–218. doi:10.1016/j.psyneuen.2019.03.002
27. Mendoza, E., & Carballo, G. (1998). Acoustic analysis of induced vocal stress by means of cognitive workload tasks. Journal of Voice, 12, 263–273. doi:10.1016/S0892-1997(98)80017-9
28. Miller, G.E., Cohen, S., & Ritchey, A.K. (2002). Chronic psychological stress and the regulation of pro-inflammatory cytokines: A glucocorticoid-resistance model. Health Psychology, 21, 531–541. doi:10.1037//0278-6133.21.6.531
29. Monroe, S.M. (2008). Modern approaches to conceptualizing and measuring human life stress. Annual Review of Clinical Psychology, 4, 33–52. doi:10.1146/annurev.clinpsy.4.022007.141207
30. Monroe, S.M., & Slavich, G.M. (2016). Psychological stressors: Overview. In G., Fink (Ed.), Stress: Concepts, cognition, emotion, and behavior, 1st ed. (pp. 109–115). Cambridge, MA: Academic Press.
31. Monroe, S.M., & Slavich, G.M. (in press). Major life events: A review of conceptual, definitional, measurement issues, and practices. In K., Harkness & E. P., Hayden (Eds.), The Oxford handbook of stress and mental health. New York: Oxford University Press.
32. Montag, A. (2018). This university is putting Amazon Echo speakers in every dorm room. CNBC. Retrieved from https://www.cnbc.com/2018/08/21/this-university-is-putting-amazon-echo-speakers-in-every-dorm-room.html
33. Montany, B. (2018). More than 900,000 smart speakers to be used in healthcare facilities by 2021. IHS. Retrieved from https://technology.ihs.com/602327/more-than-900000-smart-speakers-to-be-used-in-healthcare-facilities-by-2021
34. Paulmann, S., Furnes, D., Bøkenes, A.M., & Cozzolino, P.J. (2016). How psychological stress affects emotional prosody. PloS One, 11, e0165022. doi:10.1371/journal.pone.0165022
35. Rivas, H., & Wac, K. (Eds.). (2018). Digital health: Scaling healthcare to the world. Cham, Switzerland: Springer.
36. Scherer, K.R., & Moors, A. (2019). The emotion process: Event appraisal and component differentiation. Annual Review of Psychology, 70, 719–745. doi:10.1146/annurev-psych-122216-011854
37. Shaban, H. (2018). An Amazon Echo recorded a family’s conversation, then sent it to a random person in their contacts, report says. Washington Post. Retrieved from https://www.washingtonpost.com/news/the-switch/wp/2018/05/24/an-amazon-echo-recorded-a-familys-conversation-then-sent-it-to-a-random-person-in-their-contacts-report-says/
38. Shields, G.S., & Slavich, G.M. (2017). Lifetime stress exposure and health: A review of contemporary assessment methods and biological mechanisms. Social and Personality Psychology Compass, 11, e12335. doi:10.1111/spc3.12335
39. Shirtcliff, E.A., Buck, R.L., Laughlin, M.J., Hart, T., Cole, C.R., & Slowey, P.D. (2015). Salivary cortisol results obtainable within minutes of sample collection correspond with traditional immunoassays. Clinical Therapeutics, 37, 505–514. doi:10.1016/j.clinthera.2015.02.014
40. Simons, D.J., Boot, W.R., Charness, N., Gathercole, S.E., Chabris, C.F., Hambrick, D.Z., & Stine-Morrow, E.A. (2016). Do “brain-training” programs work? Psychological Science in the Public Interest, 17, 103–186. doi:10.1177/1529100616661983
41. Slavich, G.M. (2015). Understanding inflammation, its regulation, and relevance for health: A top scientific and public priority. Brain, Behavior, and Immunity, 45, 13–14. doi:10.1016/j.bbi.2014.10.012
42. Slavich, G.M. (2016a). Life stress and health: A review of conceptual issues and recent findings. Teaching of Psychology, 43, 346–355. doi:10.1177/0098628316662768
43. Slavich, G.M. (2016b). Psychopathology and stress. In H. L., Miller (Ed.), The SAGE encyclopedia of theory in psychology, 1st ed. (pp. 762–764). Thousand Oaks, CA: SAGE Publications.
44. Slavich, G.M. (2019). Stressnology: The primitive (and problematic) study of life stress exposure and pressing need for better measurement. Brain, Behavior, and Immunity, 75, 3–5. doi:10.1016/j.bbi.2018.08.011
45. Slavich, G.M. (in press). Psychoneuroimmunology of stress and mental health. In K., Harkness & E. P., Hayden (Eds.), The Oxford handbook of stress and mental health. New York: Oxford University Press.
46. Slavich, G.M., & Auerbach, R.P. (2018). Stress and its sequelae: Depression, suicide, inflammation, and physical illness. In J. N., Butcher & J. M., Hooley (Eds.), APA handbook of psychopathology: Vol. 1. Psychopathology: Understanding, assessing, and treating adult mental disorders (pp. 375–402). Washington, DC: American Psychological Association.
47. Slavich, G.M., & Irwin, M.R. (2014). From stress to inflammation and major depressive disorder: A social signal transduction theory of depression. Psychological Bulletin, 140, 774–815. doi:10.1037/a0035302
48. Slavich, G.M., & Shields, G.S. (2018). Assessing lifetime stress exposure using the Stress and Adversity Inventory for Adults (Adult STRAIN): An overview and initial validation. Psychosomatic Medicine, 80, 17–27. doi:10.1097/PSY.0000000000000534
49. Sondhi, S., Khan, M., Vijay, R., & Salhan, A.K. (2015). Vocal indicators of emotional stress. International Journal of Computer Applications, 122, 38–43. doi:10.5120/21780-5056
50. Titze, I.R. (2000). Principles of voice production. Salt Lake City, UT: National Center for Voice and Speech.
51. Topol, E.J., Steinhubl, S.R., & Torkamani, A. (2015). Digital medical tools and sensors. Journal of the American Medical Association, 313, 353–354. doi:10.1001/jama.2014.17125
52. Toussaint, L., Shields, G.S., Dorn, G., & Slavich, G.M. (2016). Effects of lifetime stress exposure on mental and physical health in young adulthood: How stress degrades and forgiveness protects health. Journal of Health Psychology, 21, 1004–1014. doi:10.1177/1359105314544132
53. Van Puyvelde, M., Neyt, X., McGlone, F., & Pattyn, N. (2018). Voice stress analysis: A new framework for voice and effort in human performance. Frontiers in Psychology, 9, e1994. doi:10.3389/fpsyg.2018.01994
54. Zhou, G., Hansen, J.H., & Kaiser, J.F. (2001). Nonlinear feature based classification of speech under stress. IEEE Transactions on Speech and Audio Processing, 9, 201–216. doi:10.1109/89.905995