Automated video-based facial expression analysis of neuropsychiatric disorders

Journal of Neuroscience Methods

Volumn 168, Issue 1, 2008, Pages 224-238

  Wang, Peng ^a   Barrett, Frederick ^b   Martin, Elizabeth ^b   Milonova, Marina ^b   Gur, Raquel E ^b,c   Gur, Ruben C ^b   Kohler, Christian ^c   Verma, Ragini ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b HOSPITAL OF THE UNIVERSITY OF PENNSYLVANIA   (United States)

^c UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Affective deficits; Facial expression; Pattern classification; Schizophrenia; Video analysis

Indexed keywords

ARTICLE; ASPERGER SYNDROME; AUTOANALYSIS; AUTOMATED FACIAL EXPRESSION ANALYSIS; CALCULATION; CASE REPORT; CLASSIFICATION; CONTROLLED STUDY; DATA ANALYSIS; EMOTION; EVALUATION; FACIAL EXPRESSION; GEOMETRY; HUMAN; IMAGE ANALYSIS; MEASUREMENT; PRIORITY JOURNAL; PROBABILITY; QUANTITATIVE ANALYSIS; SCHIZOPHRENIA; VIDEORECORDING;

CASE-CONTROL STUDIES; CROSS-CULTURAL COMPARISON; EMOTIONS; FACIAL EXPRESSION; FEMALE; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MALE; MENTAL DISORDERS; PATTERN RECOGNITION, AUTOMATED; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; REPRODUCIBILITY OF RESULTS; VIDEO RECORDING;

EID: 38049169004     PISSN: 01650270     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jneumeth.2007.09.030     Document Type: Article

References (53)

1. Alvino C., Kohler C., Barrett F., Gur R.E., Gur R.C., and Verma R. Computerized measurement of facial expression of emotions in schizophrenia. J Neurosci Methods 163 2 (2007) 350-361
2. Andreasen N.C. Scale for the assessment of negative symptoms (SANS) (1984), University of Iowa, Iowa City
3. Bartlett M.S., Hager J.C., Ekman P., and Sejnowski T.J. Measuring facial expressions by computer image analysis. Psychophysiology 36 (1999) 253-263
4. Bartlett M.S., Littlewort G., Frank M.G., Lainscsek C., Fasel I., and Movellan J. Recognizing facial expression: machine learning and application to spontaneous behavior. CVPR (2005) 568-573
5. Berenbaum H., and Oltmann T.F. Emotional experience and expression in schizophrenia and depression. J Abnormal Psychiatry Res 101 1 (1992) 37-44
6. Chang Y., Hu C., and Turk M. Probabilistic expression analysis on manifolds. CVPR (2004)
7. Chang Y., Vieira M., Turk M., and Velho L. Automatic 3D facial expression analysis in videos. IEEE International Workshop on Analysis and Modeling of Faces and Gestures (2005)
8. Cohen I., Sebe N., Garg A., Chen L.S., and Huang T.S. Facial expression recognition from video sequences: temporal and static modeling. Comput Vis Image Understand 91 1-2 (2003) 160-187
9. Cohen I., Seve N., Cozman G.G., Cirelo M.C., and Huang T.S. Learning bayesian network classifier for facial expression recognition using both labeled and unlabeled data. CVPR (2003)
10. Cootes T.F., Edwards G.J., and Taylor C.J. Active appearance models. IEEE Trans PAMI 23 6 (2001) 681-685
11. Cortes C., and Vapnik V. Support-vector networks. Mach Learn 20 3 (1995) 273-297
12. Davatzikos C. Measuring biological shape using geometry-based shape transformations. Image Vis Comput 19 (2001) 63-74
13. Ekman P., and Friesen W.V. Facial action coding system: a technique for the measurement of facial movement (1978), Consulting Psychologists Press, Palo Alto, California
14. Essa I.A., and Pentland A.P. Facial expression recognition using a dynamic model and motion energy. ICCV (1995) 360-367
15. Essa I.A., and Pentland A.P. Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans PAMI 19 7 (1997) 757-763
16. Fasel B., and Luettin J. Recognition of asymmetric facial action unit activities and intensities. ICPR (2000) 1100-1103
17. Fasel B., and Luettin J. Automatic facial expression analysis: a survey. Pattern Recognit 36 1 (2003) 259-275
18. Gaebel W., and Wölwer W.C. Facial expression and emotional face recognition in schizophrenia and depression. Eur Arch Psychiatry Clin Neurosci 242 (1992) 46-52
19. Gur R., Sara R., Hagendoorn M., Marom O., Hughett P., Macy L., et al. A method for obtaining 3-dimensional facial expressions and its standardization for use in neurocognitive studies. J Neurosci Methods 115 2 (2002) 137-143
20. Gur R.E., Kohler C.G., Ragland J.D., Siegel S.J., Lesko K., Bilker W.B., et al. Flat affect in Schizophrenia: relation to emotion processing and neurocognitive measures. Schizophrenia Bull 32 2 (2006) 279-287
21. Hellewell J., Connell J., and Deakin J.F.W. Affect-judgment and facial recognition memory in schizophrenia. Psychopathology 27 (1994) 255-261
22. Hsu C-W, Lin C-J. A comparison of methods for multiclass support vector machines. IEEE Trans Neural Networks 2002;13(2).
23. Kanade T., Cohn J.F., and Tian Y. Comprehensive database for facial expression analysis. AFRG (2000) 46-53
24. Kimura S., and Yachida M. Facial expression recognition and its degree estimation. IEEE Conference on Computer Vision and Pattern Recognition (1997) 295-300
25. Kohler C.G., Turner T., Stolar N.M., Bilker W.B., Brensinger C.M., Gur R.E., and Gur R.C. Differences in facial expressions of four universal emotions. Psychiatry Res 128 3 (2004) 235-244
26. Kring A.M., and Sloan D.S. The facial expression coding system (FACES): development, validation, and utility. Psychol Assess 19 (2007) 210-224
27. Kring A.M., Neale M.A.J.M., and Harvey P.D. A multichannel, multimethod assessment of affective flattening in schizophrenia. Psychiatry Res 54 (1994) 211-222
28. Kwok J.T.-Y. The evidence framework applied to support vector machines. IEEE Trans Neural Networks 11 5 (2000) 1162-1173
29. Lanitis A., Taylor C.J., and Cootes T.F. Automatic interpretation and coding of face images using flexible models. IEEE Trans PAMI 19 7 (1997) 743-756
30. Li S.Z., and Zhang Z. FloatBoost learning and statistical face detection. IEEE Trans PAMI 26 9 (2004) 1112-1123
31. Lien J.J., Kanade T., Cohn J.F., and Li C.-C. Subtly different facial expression recognition and expression intensity estimation. IEEE Conference on Computer Vision and Pattern Recognition (1998) 853-859
32. Lien J.J., Kanade T., Cohn J.F., and Li C.-C. Detection, tracking, and classification of action units in facial expression. Robtics Autonomous Syst 31 (2000) 131-146
33. Littlewort G., Bartlett M., Fasel I., Susskind J., and Movellan J. Dynamics of facial expression extracted automatically from video. Image Vis Comput 24 6 (2006) 615-625
34. Lyons M.J., Budynek J., and Akamatsu S. Automatic classification of single facial images. IEEE Trans PAMI 21 12 (1999) 1357-1362
35. Mandal M., Pandey R., and Prasad A. Facial expressions of emotions and schizophrenia: a review. Schizophrenia Bull 24 3 (1998) 399-412
36. Morrison R.L., Bellack A.S., and Mueser K.T. Deficits in facial-affect recognition and schizophrenia. Schizophrenia Bull 14 1 (1988) 67-83
37. Ohta H., Saji H., and Nakatani H. Recognition of facial expressions using muscle-based feature models. IEEE International Conference on Pattern Recognition (1998)
38. Pantic M., and Rothkrantz L.J.M. Automatic analysis of facial expressions: the state of the art. IEEE Trans PAMI 22 12 (2000) 1424-1445
39. Platt J. Probabilistic outputs for support vector machines and comparison to regularized likelihood methods (2000), MIT Press, Cambridge, MA
40. Schneider F.H.H., Himer W., Huss D., Mattes R., and Adam B. Computer-based analysis of facial action in schizophrenic and depressed patients. Eur Arch Psychiatry Clin Neurosci 240 2 (1990) 67-76
41. Stegmann M.B., Ersbol B.K., and Larsen R. FAME-a flexible appearance modeling environment. IEEE Trans Med Imaging 22 10 (2003) 1319-1331
42. Tian Y.-L., Kanade T., and Cohn J.F. Recognizing action units for facial expression analysis. IEEE Trans PAMI 23 2 (2001) 97-115
43. Tian Y.-L., Kanade T., and Cohn J.F. Handbook of face recognition (2005), Springer
44. Verma R., Davatzikos C., Loughead J., Indersmitten T., Hu R., Kohler C., et al. Quantification of facial expressions using high dimensional shape transformations. J Neurosci Methods 141 (2005) 61-73
45. Viola P., and Jones M. Robust real-time object detection. Int J Comput Vis 57 2 (2004) 137-154
46. Wang P., and Ji Q. Learning discriminant features for multi-view face and eye detection. Comput Vis Image Understand 105 2 (2007) 99-111
47. Wang Y., Ai H., Wu B., and Huang C. Real time facial expression recognition with AdaBoost. ICPR (2004)
48. Wen Z., and Huang T.S. Capturing subtle facial motions in 3D face tracking. IEEE International Conference on Computer Vision and Pattern Recognition (2003) 1343-1350
49. Wiskott L., Fellous J.-M., Kruger N., and v.d. Malsburg C. Face recognition by elastic bunch graph matching. IEEE Trans PAMI 19 7 (1997) 775-779
50. Yacoob Y., and Davis L.S. Recognizing human facial expressions from long image sequeces using optical flow. IEEE Trans PAMI 18 6 (1996) 636-642
51. Yang M.-H., Kriegman D.J., and Ahuja N. Detecting faces in images: a survey. IEEE Trans PAMI 24 1 (2002) 34-58
52. Yeasin M., Bullot B., and Sharma R. From facial expression to level of interest: a spatio-temporal approach. IEEE Conference on Computer Vision and Pattern Recognition (2004)
53. Zhang Z., Lyons M., Schuster M., and Akamatsu S. Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron. AFRG (1998) 454-459