Self-Reported Symptoms of Depression and PTSD Are Associated with Reduced Vowel Space in Screening Interviews

IEEE Transactions on Affective Computing

Volumn 7, Issue 1, 2016, Pages 59-73

  Scherer, Stefan ^a   Lucas, Gale M ^a   Gratch, Jonathan ^a   Rizzo, Albert ^a   Morency, Louis Philippe ^b  

^a University of Southern California   (United States)

^b Carnegie Mellon University ^*  (United States)

Author keywords

Depression; post traumatic stress; psychomotor retardation; unsupervised learning; vowel space

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; LEARNING SYSTEMS; SPEECH; SURVEYS; UNSUPERVISED LEARNING;

DEPRESSION; POST-TRAUMATIC STRESS; POSTTRAUMATIC STRESS DISORDER; PSYCHOMOTOR RETARDATION; STATISTICAL ROBUSTNESS; SUBJECTIVE ASSESSMENTS; UNSUPERVISED MACHINE LEARNING; VOWEL SPACE;

LINGUISTICS;

EID: 84963805331     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAFFC.2015.2440264     Document Type: Article

References (94)

1. J. K. Darby, N. Simmons, and P. A. Berger, "Speech and voice parameters of depression: A pilot study," J. Commun. Disorders, vol. 17, no. 2, pp. 75-85, 1984.
2. N. Cummins, S. Vidhyasaharan, J. Epps, and J. Krajewski, "Probabilistic acoustic volume analysis for speech affected by depression," in Proc. Interspeech, 2014, pp. 1238-1242.
3. H.-M. Liu, F.-M. Tsao, and P. K. Kuhl, "The effect of reduced vowel working space on speech intelligibility in Mandarin-speaking young adults with cerebral palsy," J. Acoustical Soc. Am., vol. 117, no. 6, pp. 3879-3889, 2005.
4. G. S. Turner, K. Tjaden, and G. Weismer, "The influence of speaking rate on vowel space and speech intelligibility for individuals with amyotrophic lateral sclerosis," J. Speech Hearing Res., vol. 38, no. 5, pp. 1001-1013, 1995.
5. P. A. McRae, K. Tjaden, and B. Schoonings, "Acoustic and perceptual consequences of articulatory rate change in parkinson disease," J. Speech, Language, Hearing Res., vol. 45, no. 1, pp. 35-50, 2002.
6. M. Alpert, E. R. Pouget, and R. R. Silva, "Reflections of depression in acoustic measures of the patient's speech," J. Affective Disorders, vol. 66, no. 1, pp. 59-69, 2001.
7. J. Gratch, L.-P. Morency, S. Scherer, G. Stratou, J. Boberg, S. Koenig, T. Adamson, and A. Rizzo, "User-state sensing for virtual health agents and telehealth applications," Stud. Health Technol. Informat., vol. 184, pp. 151-157, 2012.
8. S. Scherer, G. Stratou, G. Lucas, M. Mahmoud, J. Boberg, J. Gratch, A. Rizzo, and L.-P. Morency, "Automatic audiovisual behavior descriptors for psychological disorder analysis," Image Vis. Comput. J., Special Issue Best Face Gesture, vol. 32, no. 10, pp. 648-658, 2014.
9. C. Sobin and H. A. Sackheim, "Psychomotor symptoms of depression," Am. J. Psychiatry, vol. 154, no. 1, pp. 4-17, 1997.
10. D. Schrijvers, W. Hulstijn, and B. G. Sabbe, "Psychomotor symptoms in depression: A diagnostic, pathophysiological and therapeutic tool," J. Affective Disorders, vol. 109, no. 1/2, pp. 1-20, 2008.
11. L. M. Bylsam, B. H. Morris, and J. Rottenberg, "A meta-analysis of emotional reactivity in major depressive disorder," Clinical Psychol. Rev., vol. 28, pp. 676-691, 2008.
12. N. Cummins, S. Scherer, J. Krajewski, S. Schnieder, J. Epps, and T. Quatieri, "A review of depression and suicide risk assessment using speech analysis," Speech Commun., vol. 17, pp. 10-49, 2015.
13. J. E. Perez and R. E. Riggio, Nonverbal Social Skills and Psychopathology, series Nonverbal behavior in clinical settings. London, U.K.: Oxford Univ. Press, 2003, pp. 17-44.
14. A. Nilsonne, "Speech characteristics as indicators of depressive illness," Acta Psychiatrica Scandinavica, vol. 77, no. 3, pp. 253-263, 1988.
15. J. Leff and E. Abberton, "Voice pitch measurements in schizophrenia and depression," Psychological Med., vol. 11, no. 4, pp. 849-852, 1981.
16. D. J. France, R. G. Shiavi, S. E. Silverman, M. K. Silverman, and D. M. Wilkes, "Acoustical properties of speech as indicators of depression and suicidal risk," IEEE Trans. Biomed. Eng., vol. 47, no. 7, pp. 829-837, Jul. 2000.
17. J. A. Hall, J. A. Harrigan, and R. Rosenthal, "Nonverbal behavior in clinician-patient interaction," Appl. Preventive Psychol., vol. 4, no. 1, pp. 21-37, 1995.
18. M. Cannizzaro, B. Harel, N. Reilly, P. Chappell, and P. J. Snyder, "Voice acoustical measurement of the severity of major depression," Brain Cognition, vol. 56, no. 1, pp. 30-35, 2004.
19. Y. Yang, C. Fairbairn, and J. F. Cohn, "Detecting depression severity from vocal prosody," IEEE Trans. Affective Comput., vol. 4, no. 2, pp. 142-150, Apr.-Jun. 2013.
20. S. Scherer, G. Stratou, J. Gratch, and L.-P. Morency, "Investigating voice quality as a speaker-independent indicator of depression and PTSD," in Proc. Interspeech, 2013, pp. 847-851.
21. D. Campbell, B. Felker, C.-F. Liu, E. Yano, J. Kirchner, D. Chan, L. Rubenstein, and E. Chaney, "Prevalence of depression-PTSD comorbidity: Implications for clinical practice guidelines and primary care-based interventions," J. General Internal Med., vol. 22, pp. 711-718, 2007.
22. G. N. Marshall, T. L. Schell, and J. N. V. Miles, "All PTSD symptoms are highly associated with general distress: Ramifications for the dysphoria symptom cluster," J. Abnormal Psychol., vol. 119, no. 1, pp. 126-135, 2010.
23. S. Scherer, G. Stratou, M. Mahmoud, J. Boberg, J. Gratch, A. Rizzo, and L.-P. Morency, "Automatic behavior descriptors for psychological disorder analysis," in Proc. IEEE Conf. Autom. Face Gesture Recogn., 2013, pp. 1-8.
24. F. L. Darley, A. E. Aronson, and J. R. Brown, "Differential diagnostic patterns of dysarthria," J. Speech Hearing Res., vol. 12, pp. 246-269, 1969.
25. F. L. Darley, A. E. Aronson, and J. R. Brown, "Clusters of deviant speech dimensions in the dysarthrias," J. Speech Hearing Res., vol. 12, pp. 462-496, 1969.
26. A. J. Flint, S. E. Black, I. Campbell-Taylor, G. F. G. Gailey, and C. Levinton, "Abnormal speech articulation, psychomotor retardation, and subcortical dysfunction in major depression," J. Psychiatric Research, vol. 27, no. 3, pp. 309-319, 1993.
27. J. R. Williamson, T. F. Quatieri, B. S. Helfer, R. Horwitz, B. Yu, and D. D. Mehta, "Vocal biomarkers of depression based on motor incoordination," in Proc. 3rd ACM Int. Workshop Audio/Visual Emotion Challenge, 2013, pp. 41-48.
28. J. Williamson, T. Quatieri, B. Helfer, G. Ciccarelli, and D. Mehta, "Vocal and facial biomarkers of depression based on motor incoordination and timing," in Proc. 4th ACM Int. Workshop Audio/Visual Emotion Challenge, 2014, pp. 65-72.
29. M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic, "Avec 2013-the continuous audio/visual emotion and depression recognition challenge," in Proc. ACM Int. Workshop Audio/Visual Emotion Challenge, 2013, pp. 3-10.
30. M. Valstar, B. Schuller, K. Smith, T. Almaev, F. Eyben, J. Krajewski, R. Cowie, and M. Pantic, "AVEC 2014: 3d dimensional affect and depression recognition challenge," in Proc. 4th ACM Int. Workshop Audio/Visual Emotion Challenge, 2014, pp. 3-10.
31. Y. Heman-Ackah, R. Heuer, D. Michael, R. Ostrowski, M. Horman, M. Baroody, J. Hillenbrand, and R. Sataloff, "Cepstral peak prominence: A more reliable measure of dysphonia," Ann. Otol., Rhinol. Laryngol., vol. 112, pp. 324-333, 2003.
32. S. Scherer, Z. Hammal, Y. Yang, L.-P. Morency, and J. F. Cohn, "Dyadic behavior analysis in depression severity assessment interviews," in Proc. Int. Conf. Multimodal Interaction, 2014, pp. 112-119.
33. C. Gobl and A. Ní Chasaide, "Acoustic characteristics of voice quality," Speech Commun., vol. 11, pp. 481-490, 1992.
34. S. Scherer, J. Kane, C. Gobl, and F. Schwenker, "Investigating fuzzy-input fuzzy-output support vector machines for robust voice quality classification," Comput. Speech Language, vol. 27, no. 1, pp. 263-287, 2013.
35. J. Kane, S. Scherer, L.-P. Morency, and C. Gobl, "A comparative study of glottal open quotient estimation techniques," in Proc. Interspeech, 2013, pp. 1658-1662.
36. R. D. Kent, R. Netsell, and J. H. Abbs, "Articulatory abnormalities in athetoid cerebral palsy," J. Speech Hearing Disorders, vol. 43, pp. 353-373, 1978.
37. L. J. Platt, G. Andrews, M. Young, and P. T. Quinn, "Dysarthria of adult cerebral palsy. i. intelligibility and articulatory impairment," J. Speech Hearing Res., vol. 23, pp. 28-40, 1980.
38. K. Yorkston, D. R. Beukelmann, and K. R. Bell, "Clinical management of dysarthric speakers," J. Neurol Neurosurg Psychiatry, vol. 51, pp. 1467-1468, 1988.
39. A. P. Simpson and C. Ericsdotter, "Sex-specific differences in f0 and vowel space," in Proc. 16th Int. Congr. Phonetic Sci., 2007, pp. 933-936.
40. J. Gratch, R. Artstein, G. Lucas, S. Scherer, A. Nazarian, R. Wood, J. Boberg, D. De Vault, S. Marsella, D. Traum, A. Rizzo, and L.-P. Morency, "The distress analysis interview corpus of human and computer interviews," in Proc. Language Resources Eval. Conf., 2014, pp. 3123-3128.
41. C. De Looze, S. Scherer, B. Vaughan, and N. Campbell, "Investigating automatic measurements of prosodic accommodation and its dynamics in social interaction," Speech Commun., vol. 58, pp. 11-34, 2014.
42. R. Levitan and J. Hirschberg, "Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions," in Proc. Interspeech, 2011, pp. 3081-3084.
43. C. A. Shepard, H. Giles, and B. A. Le Poired, Communication Accommodation Theory, P. Robinson and H. Giles, Eds. New York, NY, USA: Wiley, 2001.
44. S. Scherer, J. P. Pestian, and L.-P. Morency, "Investigating the speech characteristics of suicidal adolescents," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 709-713.
45. D. De Vault, R. Artstein, G. Benn, T. Dey, E. Fast, A. Gainer, K. Georgilia, J. Gratch, A. Hartholt, M. Lhommet, G. Lucas, S. Marsella, F. Morbini, A. Nazarian, S. Scherer, G. Stratou, A. Suri, D. Traum, R. Wood, Y. Xu, A. Rizzo, and L.-P. Morency, "SimSensei kiosk: A virtual human interviewer for healthcare decision support," in Proc. Int. Conf. Auton. Agents Multiagent Syst., 2014, pp. 1061-1068.
46. H. D. Daines, M. Kumar, A. Chan, A. W. Black, M. Ravishankar, and A. I. Rudnicky, "Pocketsphinx: A free, real-time continuous speech recognition system for hand-held devices," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2006, vol. 1, p. 1.
47. D. De Vault, K. Georgilia, R. Artstein, F. Morbini, D. Traum, S. Scherer, A. Rizzo, and L.-P. Morency, "Verbal indicators of psychological distress in interactive dialogue with a virtual human," in Proc. SIGDIAL Conf.: 14th Annu. Meeting Special Interest Group Discourse Dialogue, 2013, pp. 193-202.
48. F. Morbini, D. De Vault, K. Sagae, J. Gerten, A. Nazarian, and D. Traum, "FLoReS: A forward looking, reward seeking, dialogue manager," in Proc. 4th Int. Workshop Spoken Dialog Systems, 2012.
49. S. Scherer, S. Marsella, G. Stratou, Y. Xu, F. Morbini, A. Egan, A. Rizzo, and L.-P. Morency, "Perception markup language: Towards a standardized representation of perceived nonverbal behaviors," in Proc. 12th Int. Conf. Intell. Virtual Agents, 2012, pp. 313-325.
50. M. Thiebaux, S. Marsella, A. N. Marshall, and M. Kallmann, "Smartbody: Behavior realization for embodied conversational agents," in Proc. 7th Int. Joint Conf. Auton. Agents Multiagent Syst.- Vol. 1, 2008, pp. 151-158.
51. S. Marsella, A. Shapiro, A. Feng, Y. Xu, M. Lhommet, and S. Scherer, "Towards higher quality character performance in previz," in Proc. Digital Prod. Symp., 2013, pp. 31-35.
52. S. Marsella, Y. Xu, M. Lhommet, A. Feng, S. Scherer, and A. Shapiro, "Virtual character performance from speech," in Proc. Symp. Comput. Animation, 2013, pp. 25-35.
53. E. B. Blanchard, J. Jones-Alexander, T. Buckley, and C. Forneris, "Psychometric properties of the PTSD checklist (PCL)," Behav. Res. Therapy, vol. 34, no. 8, pp. 669-673, 1996.
54. E. E. Bolton, M. J. Gray, and B. T. Litz, "A cross-lagged analysis of the relationship between symptoms of PTSD and retrospective reports of exposure," J. Anxiety Disorders, vol. 20, no. 7, pp. 877-895, 2006.
55. C. W. Hoge, C. A. Castro, S. C. Messer, D. McGurk, D. I. Cotting, and R. L. Koffman, "Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care," New England J. Med., vol. 351, no. 1, pp. 13-22, 2004.
56. K. Kroenke and R. L. Spitzer, "The PHQ-9: A new depression and diagnostic severity measure," Psychiatric Ann., vol. 32, pp. 509-521, 2002.
57. K. Kroenke, R. L. Spitzer, and J. B. W. Williams, "The PHQ-9," J. General Internal Med., vol. 16, no. 9, pp. 606-613, 2001.
58. G. Degottex, J. Kane, T. Drugman, T. Raitio, and S. Scherer, "COVAREP-a collaborative voice analysis repository for speech technologies," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2014, pp. 960-964.
59. T. Drugman and A. Abeer, "Joint robust voicing detection and pitch estimation based on residual harmonics," in Proc. Interspeech, 2011, pp. 1973-1976.
60. D. D. Mehta, D. Rudoy, and P. J. Wolfe, "Karma: Kalman-based autoregressive moving average modeling and inference for formant and antiformant tracking," J. Acoustical Soc. Am., vol. 132, no. 3, pp. 1732-1746, 2012.
61. B. Bozkurt, B. Doval, C. d'Alessandro, and T. Dutoit, "Improved differential phase spectrum processing for formant tracking," in Proc. Interspeech, 2004, pp. 2421-2424.
62. P. Ladefoged, Elements of Acoustic Phonetics, 2nd ed. Chicago, IL, USA: Chicago Univ. Press, 1996.
63. R. B. Blackman and J. W. Tukey, "Particular pairs of windows," in The Measurement of Power Spectra, From the Point of View of Communications Engineering. New York, NY, USA: Dover, 1959, pp. 98-99.
64. K. Mustafa and I. C. Bruce, "Robust formant tracking for continuous speech with speaker variability," IEEE Trans. Audio, Speech, Language Process., vol. 14, no. 2, pp. 435-444, Mar. 2006.
65. N. H. De Jong and T. Wempe, "Praat script to detect syllable nuclei and measure speech rate automatically," Behav. Res. Methods, vol. 41, no. 2, pp. 385-390, 2009.
66. T. H. Crystal and A. S. House, "Articulation rate and the duration of syllables and stress groups in connected speech," J. Acoustical Soc. Am., vol. 88, no. 1, pp. 101-112, 1990.
67. B. Lindblom, "Explaining phonetic variation: A sketch of the H&H theory," in Speech Production and Speech Modelling. New York, NY, USA: Springer Science & Business Media 1990, pp. 403-439.
68. J. B. MacQueen, "Some methods for classification and analysis of multivariate observations," in Proc. 5th Berkeley Symp. Math. Statist. Probability, 1967, pp. 281-297.
69. S. Sandoval, V. Berisha, R. L. Utianski, J. M. Liss, and A. Spanias, "Automatic assessment of vowel space area," J. Acoustical Soc. Am., vol. 134, no. 5, pp. 477-483, 2013.
70. J. Hillenbrand, L. A. Getty, M. J. Clark, and K. Wheeler, "Acoustic characteristics of American English vowels," J. Acoustical Soc. Am., vol. 97, no. 5, pp. 3099-3111, 1995.
71. L. V. Hedges, "Distribution theory for glass's estimator of effect size and related estimators," J. Educational Statist., vol. 6, no. 2, pp. 107-128, 1981.
72. J. A. Durlak, "How to select, calculate, and interpret effect sizes," J. Pediatric Psychol., vol. 34, no. 9, pp. 917-928, 2009.
73. H. H. Stassen, G. Bomben, and E. Günther, "Speech characteristics in depression," Psychopathol., vol. 24, pp. 88-105, 1991.
74. S. Scherer, L.-P. Morency, J. Gratch, and J. P. Pestian, "Reduced vowel space is a robust indicator of psychological distress: A cross-corpus analysis," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2015, pp. 4789-4793
75. M. K. Nock, I. Hwang, N. A. Sampson, and R. C. Kessler, "Mental disorders, comorbidity and suicidal behavior: Results from the national comorbidity survey replication," Molecular Psychiatry, vol. 15, pp. 868-876, 2010.
76. Y. Yunusova, G. Weismer, R. D. Kent, and N. M. Rusche, "Breathgroup intelligibility in dysarthria characteristics and underlying correlates," J. Speech, Language, Hearing Res., vol. 48, pp. 1294-1310, 2005.
77. K. Tjaden and G. E. Wilding, "Rate and loudness manipulations in dysarthria: Acoustic and perceptual findings," J. Speech, Language, Hearing Res., vol. 47, pp. 766-783, 2004.
78. W. T. Fitch and J. Giedd, "Morphology and development of the human vocal tract: A study using magnetic resonance imaging," J. Acoustical Soc. Am., vol. 106, no. 3, pp. 1511-1522, 1999.
79. J. F. Cohn, T. S. Kruez, I. Matthews, Y. Ying, M. H. Nguyen, M. T. Padilla, F. Zhou, and F. De la Torre, "Detecting depression from facial actions and vocal prosody," in Proc. 3rd Int. Conf. Affective Comput. Intell. Interaction Workshops, 2009, pp. 1-7.
80. S. Scherer, G. Stratou, and L.-P. Morency, "Audiovisual behavior descriptors for depression assessment," in Proc. 15th ACM Int. Conf. Multimdoal Interaction, 2013, pp. 135-140.
81. Y. Zhou, S. Scherer, D. Devault, J. Gratch, G. Stratou, L.-P. Morency, and J. Cassell, "Multimodal prediction of psychological disorder: Learning nonverbal commonality in adjacency pairs," in Proc. 4th Int. Workshop Series Semantics Pragmatics Dialogue, 2014, pp. 33-40.
82. J. Joshi, R. Goecke, S. Alghowinem, A. Dhall, M. Wagner, J. R. Epps, G. Parker, and M. Breakspear, "Multimodal assistive technologies for depression diagnosis and monitoring," J. Multimodal User Interfaces, vol. 7, pp. 217-228, 2013.
83. V. Venek, S. Scherer, L.-P. Morency, A. Rizzo, and J. P. Pestian, "Adolescent suicidal risk assessment in clinician-patient interaction: A study of verbal and acoustic behaviors," in Proc. IEEE Spoken Language Technol. Workshop, 2014, pp. 277-282.
84. M. Chatterjee, G. Stratou, S. Scherer, and L.-P. Morency, "Contextbased signal descriptors of heart-rate variability for anxiety assessment," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2014, pp. 3659-3663.
85. J. M. Girard, J. F. Cohn, M. H. Mahoor, S. M. Mavadati, Z. Hammal, and D. P. Rosenwald, "Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses," Image Vis. Comput. J., vol. 32, pp. 641-647, 2014.
86. G. Stratou, S. Scherer, J. Gratch, and L.-P. Morency, "Automatic nonverbal behavior indicators of depression and PTSD: The effect of gender," J. Multimodal User Interfaces, vol. 9, no. 1, pp. 1-13, 2014.
87. M. Pätzold and A. P. Simpson, "Acoustic analysis of German vowels in the Kiel Corpus of Read Speech," in The Kiel Corpus of Read/Spontaneous Speech-Acoustic Data Base, Processing Tools and Analysis Results, series Arbeitsberichte des Instituts für Phonetik und digitale Sprachverarbeitung der Universität Kiel (AIPUK) 32, A. P. Simpson, K. J. Kohler, and T. Rettstadt, Eds. Kiel, Germany: IPDS, 1997 pp. 215-247.
88. J. Hart, J. Gratch, and S. Marsella, How Virtual Reality Training Can Win Friends and Influence People, series Human Factors in Defence. Farnham, U.K.: Ashgate, 2013, ch. 21, pp. 235-249.
89. J. Ku, H. J. Jang, K. U. Kim, S. H. Park, J. J. Kim, C. H. Kim, S. W. Nam, I. Y. Kim, and S. I. Kim, "Pilot study for assessing the behaviors of patients with schizophrenia towards a virtual avatar," Cyber Psychol. Behavior, vol. 9, no. 5, pp. 531-539, 2006.
90. B. Robins, K. Dautenhahn, R. Te Boekhorst, and A. Billard, "Robotic assistants in therapy and education of children with autism: Can a small humanoid robot help encourage social interaction skills?" Universal Access Inf. Soc., vol. 4, no. 2, pp. 105-120, 2005.
91. E. Kim, J. Ku, J.-J. Kim, H. Lee, K. Han, S. I. Kim, and H.-S. Cho, "Nonverbal social behaviors of patients with bipolar mania during interactions with virtual humans," J. Nervous Mental Disease, vol. 197, no. 6, pp. 412-418, 2009.
92. J. Gratch, G. M. Lucas, A. King, and L.-P. Morency, "It's only a computer: The impact of human-agent interaction in clinical interviews," in Proc. 13th Annu. Conf. Auton. Agents Multi-Agent Syst., 2014, pp. 85-92.
93. G. Lucas, J. Gratch, A. King, and L.-P. Morency, "It's only a computer: Virtual humans increase willingness to disclose," Comput. Human Behavior, vol. 37, pp. 94-100, 2014.
94. A. Cohen, M. Alpert, T. Nienow, T. Dinzeo, and N. Docherty, "Computerized measurement of negative symptoms in schizophrenia," J. Psychiatric Res., vol. 42, pp. 827-836, 2008.