Diagnosis of depression by behavioural signals: A multimodal approach

AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge

Volumn , Issue , 2013, Pages 11-20

  Cummins, Nicholas ^a   Joshi, Jyoti ^b   Dhall, Abhinav ^c   Sethu, Vidhyasaharan ^a   Goecke, Roland ^b,c   Epps, Julien ^a  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

^b UNIVERSITY OF CANBERRA   (Australia)

^c AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Acoustic speech features; Bag of words; Behavioural signals; Depression; Multimodal fusion; Multimodal technologies; Pyramid of histogram of gradients; Space time interest points; Support vector regression

Indexed keywords

ACOUSTIC SPEECH; BAG OF WORDS; DEPRESSION; HISTOGRAM OF GRADIENTS; MULTI-MODAL; MULTI-MODAL FUSION; SPACE-TIME INTEREST POINTS; SUPPORT VECTOR REGRESSION (SVR);

EID: 84887498641     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2512530.2512535     Document Type: Conference Paper

References (50)

1. C. D. Mathers and D. Loncar, "Projections of Global Mortality and Burden of Disease from 2002 to 2030," PLoS Med, vol. 3, no. 11, pp. 2011-2030, 2006.
2. D. Maust, M. Cristancho, L. Gray, S. Rushing, C. Tjoa, and M. E. Thase, "Chapter 13 - Psychiatric rating scales," in Handbook of Clinical Neurology, vol. Volume 106, F. B. Michael J. Aminoff and F. S. Dick, Eds. Elsevier, 2012, pp. 227-237.
3. A. T. Beck and B. A. Alford, Depression: Causes and Treatment. University of Pennsylvania Press, 2008.
4. J. C. Mundt, P. J. Snyder, M. S. Cannizzaro, K. Chappie, and D. S. Geralts, "Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology," Journal of Neurolinguistics, vol. 20, no. 1, pp. 50-64, 2007.
5. H. D. Schmidt, R. C. Shelton, and R. S. Duman, "Functional Biomarkers of Depression: Diagnosis, Treatment, and Pathophysiology," Neuropsychopharmacology, vol. 36, no. 12, pp. 2375-2394, 2011.
6. E. Moore, M. A. Clements, J. W. Peifer, and L. Weisser, "Critical Analysis of the Impact of Glottal Features in the Classification of Clinical Depression in Speech," IEEE Trans. on Biom. Eng., vol. 55, no. 1, pp. 96-107, 2008.
7. L. S. A. Low, N. C. Maddage, M. Lech, L. Sheeber, and N. Allen, "Influence of acoustic low-level descriptors in the detection of clinical depression in adolescents," in 2010 IEEE Int. Conf. on Acoustics Speech and Signal Processing (ICASSP), 2010, pp. 5154-5157.
8. N. Cummins, J. Epps, M. Breakspear, and R. Goecke, "An Investigation of Depressed Speech Detection: Features and Normalization," in Interspeech2011, 2011, pp. 2997-3000.
9. J. F. Cohn, T. S. Kruez, I. Matthews, Y. Ying, N. Minh Hoai, M. T. Padilla, Z. Feng, and F. De la Torre, "Detecting depression from facial actions and vocal prosody," in 3rd Int. Conf. on Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009., 2009, pp. 1-7.
10. S. Scherer, G. Stratou, M. Mahmoud, and J. Boberg, "Automatic Behavior Descriptors for Psychological Disorder Analysis," IEEE Conf. on Automatic Face and Gesture Recognition 2013, p. NA, 2013.
11. J. Joshi, R. Goecke, M. Breakspear, and G. Parker, "Can body expressions contribute to automatic depression analysis," Proceedings of the 10th IEEE Int. Conf. on Automatic Face and Gesture Recognition FG2013. Shanghai, China, 2013.
12. S. Alghowinem, R. Goecke, M. Wagner, G. Parker, and M. Breakspear, "Eye Movement Analysis for Depression Detection," in 2013 IEEE Int. Conf. on Image Processing ICIP2013, Melbourne, Australia, 15-18 Sep 2013, 2013.
13. M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic, "The Continuous Audio/Visual Emotion and Depression Recognition Challenge," in The 21st ACM Int. Conf. on Multimedia, 2013, p. NA.
14. M. J. H. Balsters, E. J. Krahmer, M. G. J. Swerts, and A. J. J. M. Vingerhoets, "Verbal and Nonverbal Correlates for Depression: A Review," Current Psychiatry Reviews, vol. 8, no. 3, pp. 227-234, 2012.
15. C. Sobin and H. A. Sackeim, "Psychomotor symptoms of depression," Am J Psychiatry, vol. 154, no. 1, pp. 4-17, 1997.
16. A. J. Flint, S. E. Black, I. Campbell-Taylor, G. F. Gailey, and C. Levinton, "Abnormal speech articulation, psychomotor retardation, and subcortical dysfunction in major depression," Journal of Psychiatric Research, vol. 27, no. 3, pp. 309-319, 1993.
17. T. F. Quatieri and N. Malyska, "Vocal-Source Biomarkers for Depression: A Link to Psychomotor Activity," in Interspeech2012, 2012, pp. 1059-1062.
18. J. C. Mundt, A. P. Vogel, D. E. Feltner, and W. R. Lenderking, "Vocal Acoustic Biomarkers of Depression Severity and Treatment Response," Biological Psychiatry, vol. 72, pp. 580 -587, 2012.
19. A. Trevino, T. Quatieri, and N. Malyska, "Phonologically-based biomarkers for major depressive disorder," EURASIP Journal on Advances in Signal Processing, vol. 2011, no. 1, pp. 1-18, 2011.
20. N. Cummins, J. Epps, and E. Ambikairajah, "Spectro-Temporal Analysis of Speech Affected by Depression and Psychomotor Retardation," in 2013 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2013, p. NA.
21. D. J. France, R. G. Shiavi, S. Silverman, M. Silverman, and M. Wilkes, "Acoustical properties of speech as indicators of depression and suicidal risk," IEEE Trans. on Bio-Eng, vol. 47, no. 7, pp. 829-837, 2000.
22. N. Cummins, J. Epps, V. Sethu, M. Breakspear, and R. Goecke, "Modeling Spectral Variability for the Classification of Depressed Speech," in 14th Annual Conference of the International Speech Communication Association Interspeech2013, Lyon, France, 25-29 Aug 2013, 2013.
23. D. Sturim, P. A. Torres-Carrasquillo, T. F. Quatieri, N. Malyska, and A. McCree, "Automatic Detection of Depression in Speech Using Gaussian Mixture Modeling with Factor Analysis," in Interspeech2011, 2011, pp. 2983-2986.
24. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Müller, and S. Narayanan, "Paralinguistics in speech and language; State-of-the-art and the challenge," Computer Speech & Language, p. NA, 2012.
25. B. Schuller, S. Steidl, A. Batliner, F. Schiel, J. Krajewski, F. Weninger, and F. Eyben, "Medium-term speaker states - A review on intoxication, sleepiness and the first challenge," Computer Speech & Language, p. NA, 2012.
26. H. Ellgring, Non-verbal communication in depression. Cambridge University Press, 1989.
27. G. McIntyre, R. Goecke, M. Hyett, M. Green, and M. Breakspear, "An approach for automatically measuring facial activity in depressed subjects," in 3rd Int. Conf. on Affective Computing and Intelligent Interaction and Workshops, 2009. (ACII '09), 2009, pp. 1-8.
28. J. Saragih and R. Goecke, "Learning AAM fitting through simulation," Pattern Recognition, vol. 42, no. 11, pp. 2628-2636, Nov. 2009.
29. H. Gunes and M. Piccardi, "Bi-modal emotion recognition from expressive face and body gestures," Journal of Network and Computer Applications, vol. 30, no. 4, pp. 1334-1345, Nov. 2007.
30. A. Kleinsmith and N. Bianchi-Berthouze, "Affective Body Expression Perception and Recognition: A Survey," in IEEE Transactions on Affective Computing, 2013, vol. 4, no. 1, pp. 15-33.
31. J. Joshi, A. Dhall, R. Goecke, and J. F. Cohn, "Relative Body Parts Movement for Automatic Depression Analysis," in Proceedings of Affective Computing and Intelligent Interaction ACII2013, Geneva, Switzerland, 2-5 Sep 2013, 2013.
32. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld, "Learning realistic human actions from movies," in IEEE Conf. on Computer Vision and Pattern Recognition, 2008. CVPR 2008., 2008, pp. 1-8.
33. A. Bosch, A. Zisserman, and X. Munoz, "Representing shape with a spatial pyramid kernel," in Proceedings of the 6th ACM Int. Conf. on Image and video retrieval (CIVR '07), 2007, pp. 401-408.
34. A. Dhall, A. Asthana, R. Goecke, and T. Gedeon, "Emotion recognition using PHOG and LPQ features," in 2011 IEEE Int. Conf. on Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011, pp. 878-883.
35. S. D'Mello and J. Kory, "Consistent but modest: a meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies," in 14th ACM Int. Conf. on Multimodal Interaction (ICMI '12), 2012, pp. 31-38.
36. R. J. Davidson, D. Pizzagalli, J. B. Nitschke, and K. Putnam, "Depression: Perspectives from Affective Neuroscience," Annual Review of Psychology, vol. 53, no. 1, pp. 545-574, 2002.
37. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, "Emotion recognition in human-computer interaction," Signal Processing Magazine, IEEE, vol. 18, no. 1, pp. 32-80, 2001.
38. M. A. Nicolaou, H. Gunes, and M. Pantic, "Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space," IEEE Trans. on Affective Computing, vol. 2, no. 2, pp. 92-105, 2011.
39. W. M. Campbell, D. E. Sturim, D. A. Reynolds, and A. Solomonoff, "SVM Based Speaker Verification using a GMM Supervector Kernel and NAP Variability Compensation," in 2006 IEEE Int. Conf. on Acoustics, Speech and Signal Processing, (ICASSP' 06)., 2006, vol. 1, pp. 97-100.
40. C. H. You, K. A. Lee, and H. Li, "GMM-SVM Kernel With a Bhattacharyya-Based Distance for Speaker Recognition," IEEE Trans.on Audio, Speech, and Lang. Proc., vol. 18, no. 6, pp. 1300-1312, Aug. 2010.
41. M. Li, K. J. Han, and S. Narayanan, "Automatic speaker age and gender recognition using acoustic and prosodic level information fusion," Computer Speech & Language, vol. 27, no. 1, pp. 151-167, Jan. 2013.
42. F. Eyben, M. Wollmer, and B. Schuller, "Opensmile: the munich versatile and fast open-source audio feature extractor," in Proceedings of the Int.Conf. on Multimedia, 2010, pp. 1459-1462.
43. C.-C. Chang and C.-J. Lin, "LIBSVM: A Library for Support Vector Machines," ACM Trans. on Intelligent Systems and Technology, vol. 2, no. 3, p. 1:27, 2011.
44. K. Sikka, A. Dhall, and M. Bartlett, "Weakly Supervised Pain Localization using Multiple Instance Learning," in Automatic Face and Gesture Recognition, FG 2013 IEEE Conference, 2013, p. NA.
45. J. Joshi, R. Goecke, S. Alghowinem, A. Dhall, M. Wagner, J. Epps, G. Parker, and M. Breakspear, "Multimodal Assistive Technologies for Depression Diagnosis and Monitoring," Journal on Multimodal User Interfaces, 2013.
46. X. Xiong and F. De la Torre, "Supervised Descent Method and its Applications to Face Alignment," in IEEE Transaction on Pattern Analysis and Machine Intelligence, 2013, p. NA.
47. J. Wu and J. M. Rehg, "Beyond the Euclidean distance: Creating effective visual codebooks using the Histogram Intersection Kernel," in 2009 IEEE 12th Int.Conf. on Computer Vision, 2009, pp. 630-637.
48. S. Maji, A. C. Berg, and J. Malik, "Classification using intersection kernel support vector machines is efficient," in IEEE Conf.on Computer Vision and Pattern Recognition, 2008. CVPR 2008., 2008, pp. 1-8.
49. X. Zhu, C. Vondrick, D. Ramanan, and C. Fowlkes, "Do We Need More Training Data or Better Models for Object Detection?," in BMVC, 2012, pp. 1-11.
50. N. Cummins, J. Epps, J. Min, K. Kua, and J. M. K. Kua, "A Comparison of Classification Paradigms for Speaker Likeability Determination," in Interspeech2012, 2012, pp. 282-285.