Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses

Image and Vision Computing

Volumn 32, Issue 10, 2014, Pages 641-647

  Girard, Jeffrey M ^a   Cohn, Jeffrey F ^a,b   Mahoor, Mohammad H ^c   Mavadati, S Mohammad ^c   Hammal, Zakia ^b   Rosenwald, Dean P ^a  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c University of Denver   (United States)

Author keywords

Depression; Facial expression; FACS; Head motion; Multimodal

Indexed keywords

BEHAVIORAL RESEARCH; FACE RECOGNITION; VIDEO RECORDING;

DEPRESSION; FACIAL EXPRESSIONS; FACS; HEAD MOTION; MULTI-MODAL;

COMPUTER KEYBOARDS;

EID: 84906781101     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2013.12.007     Document Type: Article

References (85)

1. N.B. Allen, and P.B.T. Badcock The social risk hypothesis of depressed mood: evolutionary, psychosocial, and neurobiological perspectives Psychol. Bull. 129 2003 887 913 (Pubitemid 37398634)
2. American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders 4th Ed. 1994 American Psychiatric Association Washington, DC
3. S. Baron-Cohen Mind Reading: The Interactive Guide to Emotion 2003 Jessica Kingsley Publishers London
4. M.S. Bartlett, G. Littlewort, M.G. Frank, C. Lainscsek, I.R. Fasel, and J.R. Movellan Automatic recognition of facial actions in spontaneous expressions J. Multimedia 1 2006 22 35
5. A.T. Beck Depression: Clinical, Experimental, and Theoretical Aspects 1967 Harper and Row New York
6. M. Belkin, and P. Niyogi Laplacian eigenmaps for dimensionality reduction and data representation Neural Comput. 15 2003 1373 1396 (Pubitemid 37049796)
7. H. Berenbaum, and T.F. Oltmanns Emotional experience and expression in schizophrenia and depression J. Abnorm. Psychol. 101 1992 37 44
8. G.H. Bower Mood and memory Am. Psychol. 36 1981 129
9. A.Z. Brozgold, J.C. Borod, C.C. Martin, L.H. Pick, M. Alpert, and J. Welkowitz Social functioning and facial emotional expression in neurological and psychiatric disorders Appl. Neuropsychol. 5 1998 15 23 (Pubitemid 28367171)
10. L.M. Bylsma, B.H. Morris, and J. Rottenberg A meta-analysis of emotional reactivity in major depressive disorder Clin. Psychol. Rev. 28 2008 676 691 (Pubitemid 351374605)
11. L. Cayton Algorithms for manifold learning Technical Report CS2008-0923 2005
12. C.C. Chang, and C.J. Lin LIBSVM: a library for support vector machines ACM Trans. Intell. Syst. Technol. 2 2011 27
13. Y.E. Chentsova-Dutton, J.L. Tsai, and I.H. Gotlib Further evidence for the cultural norm hypothesis: positive emotion in depressed and control European American and Asian American women Cult. Divers. Ethn. Minor. Psychol. 16 2010 284 295
14. S.W. Chew, P. Lucey, S. Lucey, J. Saragih, J.F. Cohn, I. Matthews, and S. Sridharan In the pursuit of effective affective computing: the relationship between features and registration IEEE Trans. Syst. Man Cybern. 42 2012 1006 1016
15. L.A. Clark, and D. Watson Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications J. Abnorm. Psychol. 100 1991 316
16. J. Cohen A coefficient of agreement for nominal scales Educ. Psychol. Meas. 20 1960 37 46
17. J.F. Cohn, T.S. Kruez, I. Matthews, Y. Ying, N. Minh Hoai, M.T. Padilla, Z. Feng, and F. De La Torre Detecting depression from facial actions and vocal prosody, in: Affective Computing and Intelligent Interaction 2009 Amsterdam, pp. 1-7
18. J.F. Cohn, and M.A. Sayette Spontaneous facial expression in a small group can be automatically measured: an initial demonstration Behav. Res. Methods 42 2010 1079 1086 10.3758/BRM.42.4.1079
19. T.F. Cootes, G.J. Edwards, and C.J. Taylor Active appearance models IEEE Trans. Pattern Anal. Mach. Intell. 23 2001 681 685 (Pubitemid 32585279)
20. M. Cox, J. Nuevo-Chiquero, J.M. Saragih, and S. Lucey CSIRO Face Analysis SDK 2013
21. J.C. Coyne Toward an interactional description of depression Psychiatry: Journal for the Study of Interpersonal Processes 39 1 1976 28 40
22. R. Depue, and W. Iacono Neurobehavioral aspects of affective disorders Annu. Rev. Psychol. 40 1989 457 492
23. A.K. Dixon, H.U. Fisch, C. Huber, and A. Walser Ethological studies in animals and man, their use in psychiatry Pharmacopsychiatry 22 1989 44 50 (Pubitemid 19071783)
24. P. Ekman, W.V. Friesen, and J. Hager Facial Action Coding System (FACS): A Technique for the Measurement of Facial Movement 2002 Research Nexus Salt Lake City, UT
25. P. Ekman, and D. Matsumoto Facial expression analysis Scholarpedia 3 2008
26. H. Ellgring Nonverbal Communication in Depression 1989 Cambridge University Press Cambridge
27. C.E. Fairbairn, M.a. Sayette, J.M. Levine, J.F. Cohn, and K.G. Creswell The effects of alcohol on the emotional displays of whites in interracial groups Emotion 12 2013
28. T. Fawcett An introduction to ROC analysis Pattern Recogn. Lett. 27 2006 861 874
29. M.B. First, M. Gibbon, R.L. Spitzer, and J.B. Williams User's Guide for the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I, Version 2.0, October 1995 Final Version) 1995 Biometric Research New York
30. H.U. Fisch, S. Frey, and H.P. Hirsbrunner Analyzing nonverbal behavior in depression J. Abnorm. Psychol. 92 1983 307 318
31. J.L. Fleiss Statistical Methods for Rates and Proportions 2nd ed. 1981 John Wiley & Sons New York
32. I.K. Fodor A survey of dimension reduction techniques Technical Report 2002
33. L. Fossi, C. Faravelli, and M. Paoli The ethological approach to the assessment of depressive disorders J. Nerv. Ment. Dis. 172 6 1984 332 341 (Pubitemid 14079972)
34. D.C. Fowles A motivational view of psychopathology W.D. Spaulding, H.A. Simon, Integrative Views of Motivation, Cognition, and Emotion 1994 U of Nebraska Press
35. N. Freedman The analysis of movement behavior during the clinical interview A.W. Siegman, B. Pope, Studies in Dyadic Communication 1972 Pergamon Press New York
36. A.J. Fridlund Human Facial Expression: An Evolutionary View 1994 Academic Press
37. N.H. Frijda Expression, emotion, neither, or both? Cogn. Emot. 9 1995 617 635
38. W. Gaebel, and W. Wolwer Facial expressivity in the course of schizophrenia and depression Eur. Arch. Psychiatry Clin. Neurosci. 254 2004 335 342 (Pubitemid 39312829)
39. J.G. Gehricke, and D. Shapiro Reduced facial expression and social context in major depression: discrepancies between facial muscle activity and self-reported emotion Psychiatry Res. 95 2000 157 167 (Pubitemid 30645889)
40. M. Hamilton Development of a rating scale for primary depressive illness Br. J. Soc. Clin. Psychol. 6 1967 278 296
41. Z. Hammal, T.E. Bailie, J.F. Cohn, D.T. George, S. Lucey, Temporal coordination of head motion in couples with history of interpersonal violence, in: International Conference on Automatic Face and Gesture Recognition, Shanghai, China.
42. J.B. Henriques, and R.J. Davidson Decreased responsiveness to reward in depression Cogn. Emot. 14 2000 711 724
43. U. Hess, R.B. Adams Jr., and R.E. Kleck Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger Cogn. Emot. 19 2005 515 536 (Pubitemid 40895208)
44. U. Hess, S. Blairy, and R.E. Kleck The influence of facial emotion displays, gender, and ethnicity on judgments of dominance and affiliation J. Nonverbal Behav. 24 2000 265 283
45. S.D. Hollon, M.E. Thase, and J.C. Markowitz Treatment and prevention of depression Psychological. Sci. Public Interes. 3 2002 39 77
46. C.W. Hsu, C.C. Chang, and C.J. Lin A practical guide to support vector classification Technical Report 2003
47. I.H. Jones, and M. Pansa Some nonverbal aspects of depression and schizophrenia occurring during the interview J. Nerv. Ment. Dis. 167 1979 402 409 (Pubitemid 9202700)
48. J. Joshi, R. Goecke, G. Parker, and M. Breakspear Can body expressions contribute to automatic depression analysis?, in: International Conference on Automatic Face and Gesture Recognition 2013
49. M. Katsikitis, and I. Pilowsky A controlled quantitative study of facial expression in Parkinson's disease and depression J. Nerv. Ment. Dis. 179 1991 683 688
50. D. Keltner, and B.N. Buswell Embarrassment: its distinct form and appeasement functions Psychol. Bull. 122 1997 250
51. G.L. Klerman, M.M. Weissman, B.J. Rounsaville, and E. Chevron Interpersonal Psychotherapy of Depression 1984 Basic Books New York
52. E. Klinger Consequences of commitment to and disengagement from incentives Psychol. Rev. 82 1975 1 25
53. B. Knutson Facial expressions of emotion influence interpersonal trait inferences J. Nonverbal Behav. 20 1996 165 182 (Pubitemid 126128257)
54. R. Kotov, W. Gamez, F. Schmidt, and D. Watson Linking big personality traits to anxiety, depressive, and substance use disorders: a meta-analysis Psychol. Bull. 136 2010 768
55. M.H. Mahoor, S. Cadavid, D.S. Messinger, and J.F. Cohn A framework for automated measurement of the intensity of non-posed Facial Action Units, in: Computer Vision and Pattern Recognition Workshops, Miami 2009 pp. 74-80
56. C.D. Mathers, and D. Loncar Projections of global mortality and burden of disease from 2002 to 2030 PLoS Med. 3 2006 e442
57. I. Matthews, and S. Baker Active appearance models revisited Int. J. Comput. Vis. 60 2004 135 164
58. I.B. Mauss, R.W. Levenson, L. McCarter, F.H. Wilhelm, and J.J. Gross The tie that binds? Coherence among emotion experience, behavior, and physiology Emotion 5 2005 175 190 (Pubitemid 40993307)
59. R. Mergl, P. Mavrogiorgou, U. Hegerl, and G. Juckel Kinematical analysis of emotionally induced facial expressions: a novel tool to investigate hypomimia in patients suffering from depression J. Neurol. Neurosurg. Psychiatry 76 2005 138 140 (Pubitemid 40013511)
60. R.M. Nesse Is depression an adaptation? Arch. Gen. Psychiatry 57 2000 14 20 (Pubitemid 30035654)
61. L.I. Reed, M.A. Sayette, and J.F. Cohn Impact of depression on response to comedy: a dynamic facial coding analysis J. Abnorm. Psychol. 116 2007 804 809
62. B. Renneberg, K. Heyn, R. Gebhard, and S. Bachmann Facial expression of emotions in borderline personality disorder and depression J. Behav. Ther. Exp. Psychiatry 36 2005 183 196 (Pubitemid 40961876)
63. J. Rottenberg, J.J. Gross, and I.H. Gotlib Emotion context insensitivity in major depressive disorder J. Abnorm. Psychol. 114 2005 627 639 (Pubitemid 41746999)
64. J. Rottenberg, K.L. Kasch, J.J. Gross, and I.H. Gotlib Sadness and amusement reactivity differentially predict concurrent and prospective functioning in major depressive disorder Emotion 2 2002 135 146
65. A.J.J. Rush, M.B. First, and D. Blacker Handbook of Psychiatric Measures 2nd ed. 2008 American Psychiatric Publishing, Inc. Washington, D.C.
66. S. Sakamoto, K. Nameta, T. Kawasaki, K. Yamashita, and A. Shimizu Polygraphic evaluation of laughing and smiling in schizophrenic and depressive patients Percept. Mot. Skills 85 1997 1291 1302 (Pubitemid 127727559)
67. J.T. Schelde Major depression: behavioral markers of depression and recovery J. Nerv. Ment. Dis. 186 1998 133 140 (Pubitemid 28124327)
68. S. Scherer, G. Stratou, M. Mahmoud, J. Boberg, J. Gratch, A.S. Rizzo, L.p. Morency, Automatic behavior descriptors for psychological disorder analysis, in: International Conference on Automatic Face and Gesture Recognition, 2013.
69. F. Schneider, H. Heimann, W. Himer, D. Huss, R. Mattes, and B. Adam Computer-based analysis of facial action in schizophrenic and depressed patients Eur. Arch. Psychiatry Clin. Neurosci. 240 1990 67 76
70. C. Segrin Social skills deficits associated with depression Clin. Psychol. Rev. 20 2000 379 403 (Pubitemid 30114377)
71. C. Segrin, and J.P. Dillard The interactional theory of depression: a meta-analysis of the research literature J. Soc. Clin. Psychol. 11 1992 43 70
72. S.S. Shapiro, and M.B. Wilk An analysis of variance test for normality (complete samples) Biometrika 52 1965 591 611
73. P.E. Shrout, and J.L. Fleiss Intraclass correlations: uses in assessing rater reliability Psychol. Bull. 86 1979 420
74. D.M. Sloan, M.E. Strauss, S.W. Quirk, and M. Sajatovic Subjective and expressive emotional responses in depression J. Affect. Disord. 46 1997 135 141 (Pubitemid 28037485)
75. D.M. Sloan, M.E. Strauss, and K.L. Wisner Diminished response to pleasant stimuli by depressed women J. Abnorm. Psychol. 110 2001 488 493 (Pubitemid 32703086)
76. L. Sloman Evolved mechanisms in depression: the role and interaction of attachment and social rank in depression J. Affect. Disord. 74 2003 107 121 (Pubitemid 36428692)
77. Y.l. Tian, T. Kanade, and J.F. Cohn Facial expression analysis Handbook of Face Recognition 2005 247 275
78. F. Tremeau, D. Malaspina, F. Duval, H. Correa, M. Hager-Budny, L. Coin-Bariou, J.P. Macher, and J.M. Gorman Facial expressiveness in patients with schizophrenia compared to depressed patients and nonpatient comparison subjects Am. J. Psychiatr. 162 2005 92 101 (Pubitemid 40095677)
79. A. Troisi, and A. Moles Gender differences in depression: an ethological study of nonverbal behavior during interviews J. Psychiatr. Res. 33 1999 243 250 (Pubitemid 29178667)
80. J.L. Tsai, N. Pole, R.W. Levenson, and R.F. Munoz The effects of depression on the emotional responses of Spanish-speaking Latinas Cult. Divers. Ethn. Minor. Psychol. 9 2003 49 63 (Pubitemid 36142951)
81. J. Whitehill, G. Littlewort, I.R. Fasel, M.S. Bartlett, and J.R. Movellan Toward practical smile detection IEEE Trans. Pattern Anal. Mach. Intell. 31 2009 2106 2111
82. F. Wilcoxon Individual comparisons by ranking methods Frank Wilcoxon Biom. Bull. 1 1945 80 83
83. M.A. Youngren, and P.M. Lewinsohn The functional relation between depression and problematic interpersonal behavior J. Abnorm. Psychol. 89 1980 333
84. Z. Zeng, M. Pantic, G.I. Roisman, and T.S. Huang A survey of affect recognition methods: audio, visual, and spontaneous expressions IEEE Trans. Pattern Anal. Mach. Intell. 31 2009 39 58
85. Y. Zhu, F. De La Torre, J.F. Cohn, Y.J. Zhang, Dynamic cascades with bidirectional bootstrapping for spontaneous facial action unit detection, in: International Conference on Affective Computing and Intelligent Interaction and Workshops, 2009, pp. 1-8