Depression recognition based on dynamic facial and vocal expression features using partial least square regression

AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge

Volumn , Issue , 2013, Pages 21-29

  Meng, Hongying ^a   Huang, Di ^b   Wang, Heng ^b   Yang, Hongyu ^b   Al Shuraifi, Mohammed ^a   Wang, Yunhong ^b  

^a BRUNEL UNIVERSITY   (United Kingdom)

^b BEIHANG UNIVERSITY   (China)

Author keywords

Affective computing; Challenge; Emotion recognition; Facial expression; Speech

Indexed keywords

SPEECH;

AFFECTIVE COMPUTING; BASELINE RESULTS; BETTER PERFORMANCE; CHALLENGE; EMOTION RECOGNITION; FACIAL EXPRESSIONS; MOTION HISTORY HISTOGRAMS (MHH); PARTIAL LEAST SQUARE REGRESSION;

FACE RECOGNITION;

EID: 84887425131     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2512530.2512532     Document Type: Conference Paper

References (51)

1. http://www.nimh.nih.gov/health/publications/depression/index.shtml, Retrieved 15 July 2013.
2. T. Ahonen, A. Hadid, and M. Pietikainen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):2037-2041, 2006.
3. N. C. Andreasen. Scale for the Assessment of Negative Symptoms. Department of Psychiatry, College of Medicine, the University of Iowa, 1984.
4. M. E. Ayadi, M. S. Kamel, and F. Karray. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition, 44(3):572-587, 2011.
5. I. Bloch. Information combination operators for data fusion: a comparative review with classification. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 26(1):52-67, 1996.
6. A. F. Bobick and J. W. Davis. The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(3):257-267, 2001.
7. C. Busso, Z. Deng, S. Yildirim, M. Bulut, C. M. Lee, A. Kazemzadeh, S. Lee, U. Neumann, and S. Narayanan. Analysis of emotion recognition using facial expressions, speech and multimodal information. In International Conference on Multimodal Interaction, pages 205-211, 2004.
8. C.Gobl and A. N. Chasaide. The role of voice quality in communicating emotion, mood and attitude. Speech Communication, 40(1-2):189-212, 2003.
9. I. Cohen, N. Sebe, A. Garg, L. S. Chen, and T. S. Huang. Facial expression recognition from video sequences: temporal and static modeling. Computer Vision and Image Understanding, 91(1-2):160-187, 2003. Special Issue on Face Recognition.
10. J. F. Cohn, T. S. Kruez, I. Matthews, Y. Yang, M. H. Nguyen, M. Padilla, F. Zhou, and F. De la Torre. Detecting depression from facial actions and vocal prosody. In International Conference on Affective Computing and Intelligent Interaction and Workshops, pages 1-7, 2009.
11. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 1, pages 886-893, 2005.
12. S. de Jong. Simpls: An alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems, 18(3):251-263, 1993.
13. G. J. Edwards, T. F. Cootes, and C. J. Taylor. Face recognition using active appearance models. In European Conference on Computer Vision, pages 581-595, London, UK, 1998. Springer-Verlag.
14. P. Ekman and W. V. Friesen. Facial Action Coding System. Consulting Psychologists Press, 1978.
15. D. A. Forsyth and J. Ponce. Computer Vision: A Modern Approach. Prentice Hall, us ed edition, August 2002.
16. W. T. Freeman, W. T. Freeman, M. Roth, and M. Roth. Orientation histograms for hand gesture recognition. In IEEE International Workshop on Automatic Face and Gesture Recognition, pages 296-301, 1994.
17. J. M. Girard, J. F. Cohn, M. H. Mahoor, S. Mavadati, and D. Rosenwald. Social risk and depression: Evidence from manual and automatic facial expression analysis. In IEEE International Conference on Automatic Face and Gesture Recognition, 2013.
18. J. Hamm, C. G. Kohler, R. C. Gur, and R. Verma. Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders. Journal of Neuroscience Methods, 200(2):237-256, 2011.
19. Health & Consumer Protection Directorate General. Improving the mental health of the population: Towards a strategy on mental health for the european union, 2005.
20. Health & Consumer Protection Directorate General. Mental health in the EU, 2008.
21. R. Kaliouby and P. Robinson. Real-time inference of complex mental states from facial expressions and head gestures. In B. Kisacanin, V. Pavlovic, and T. Huang, editors, Real-Time Vision for Human-Computer Interaction, pages 181-200. Springer US, 2005.
22. A. Kleinsmith, N. Bianchi-Berthouze, and A. Steed. Automatic recognition of non-acted affective postures. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, 41:1027-1038, 2011.
23. H. Meng and N. Bianchi-Berthouze. Affective state level recognition in naturalistic facial and vocal expressions. IEEE Transactions on Cybernetics, 2013.
24. H. Meng and N. Pears. Descriptive temporal template features for visual motion recognition. Pattern Recognition Letters, 30(12):1049-1058, 2009.
25. H. Meng, N. Pears, and C. Bailey. A human action recognition system for embedded computer vision application. In IEEE Conference on Computer Vision and Pattern Recognition Workshop on Embedded Computer Vision, pages 1-6, 2007.
26. H. Meng, N. Pears, M. Freeman, and C. Bailey. Motion history histograms for human action recognition. In B. Kisačanin, S. Bhattacharyya, and S. Chai, editors, Embedded computer vision, Advances in pattern recognition, pages 139-162. Springer, 2009.
27. H. Meng, B. Romera-Paredes, and N. Bianchi-Berthouze. Emotion recognition by two view SVM 2K classifier on dynamic facial expression features. In IEEE International Conference on Automatic Face and Gesture Recognition, pages 854-859, 2011.
28. T. Nwe, S. Foo, and L. De Silva. Speech emotion recognition using hidden markov models. Speech Communication, 41(1-2):603-623, 2003.
29. T. Ojala, M. Pietikäinen, and T. Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7):971-987, 2002.
30. M. Pantic and I. Patras. Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 36(2):433-449, 2006.
31. M. Pantic and L. J. Rothkrantz. Automatic analysis of facial expressions: The state of the art. IEEE Trans. Pattern Anal. Mach. Intell., 22:1424-1445, 2000.
32. R.Cowie and R. R. Cornelius. Describing the emotional states that are expressed in speech. Speech Communication, 40(1-2):5-32, 2003.
33. S. Salmans. Depression: questions you have - answers you need. People's Medical Society, 1995.
34. S. Scherer, G. Stratou, J. Gratch, J. Boberg, M. Mahmoud, A. S. Rizzo, and L.-P. Morency. Automatic behavior descriptors for psychological disorder analysis. In IEEE International Conference on Automatic Face and Gesture Recognition, 2013.
35. B. Schuller, M. Valstar, F. Eyben, R. Cowie, and M. Pantic. AVEC 2012: The continuous audio/visual emotion challenge. In ACM International Conference on Multimodal Interaction, pages 449-456, 2012.
36. B. Schuller, M. Valstar, F. Eyben, G. McKeown, R. Cowie, and M. Pantic. AVEC 2011: The first international audio/visual emotion challenge. In International Conference on Affective Computing and Intelligent Interaction (ACII 2011), 2011.
37. C. Shan, S. Gong, and P. W. McOwan. Facial expression recognition based on local binary patterns: A comprehensive study. Image Vision Comput., 27(6):803-816, May 2009.
38. A. Sinha, H. Chen, D. G. Danu, T. Kirubarajan, and M. Farooq. Estimation and decision fusion: A survey. In IEEE International Conference on Engineering of Intelligent Systems, pages 1-6, 2006.
39. H. Teager. Some observations on oral air flow during phonation. IEEE Transactions on Acoustics, Speech, and Signal Process, 28(5):599-601, 1990.
40. Y. L. Tian, T. Kanade, and J. F. Cohn. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97-115, 2001.
41. Y. Tong, W. Liao, and Q. Ji. Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(10):1683-1699, Oct. 2007.
42. N. Ueda. Optimal linear combination of neural networks for improving classification performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(2):207-215, 2000.
43. M. F. Valstar, B. Jiang, M. Méhu, M. Pantic, and K. Scherer. The first facial expression recognition and analysis challenge. In IEEE International Conference on Automatic Face and Gesture Recognition, 2011.
44. M. F. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic. AVEC 2013 - the continuous audio/visual emotion and depression recognition challenge. In International Conference on ACM Multimedia - Audio/Visual Emotion Challenge and Workshop, 2013.
45. P. Wang, F. Barrett, E. Martin, M. Milanova, R. E. Gur, R. C. Gur, C. Kohler, and R. Verma. Automated video-based facial expression analysis of neuropsychiatric disorders. Journal of Neuroscience Methods, 168(1):224-238, 2008.
46. Y. Yacoob and L. S. Davis. Recognizing human facial expressions from long image sequeces using optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6):636-642, 1996.
47. C. Yang, R. Duraiswami, and L. Davis. Fast multiple object tracking via a hierarchical particle filter. In IEEE International Conference on Computer Vision, pages 212-219, Washington, DC, USA, 2005. IEEE Computer Society.
48. Y. Yang, C. Fairbairn, and J. Cohn. Detecting depression severity from intra- and interpersonal vocal prosody. In IEEE International Conference on Automatic Face and Gesture Recognition, 2013.
49. Z. Zeng, J. Tu, B. Pianfetti, M. Liu, T. Zhang, Z. Zhang, T. Huang, and S. Levinson. Audio-visual affect recognition through multi-stream fused HMM for HCI. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2, pages 967-972, 2005.
50. Z. Zhang, M. Lyons, M. Schuster, and S. Akamatsu. Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron. In IEEE International Conference on Automatic Face and Gesture Recognition, pages 454-459, 1998.
51. G. Zhou, J. Hansen, and J. Kaiser. Nonlinear feature based classification of speech under stress. IEEE Transactions on Speech and Audio Processing, 9(3):599-601, 2001.