Automated Facial Action Coding System for dynamic analysis of facial expressions in neuropsychiatric disorders

Journal of Neuroscience Methods

Volumn 200, Issue 2, 2011, Pages 237-256

  Hamm, Jihun ^a   Kohler, Christian G ^b   Gur, Ruben C ^b,c   Verma, Ragini ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c VETERANS AFFAIRS MEDICAL CENTER   (United States)

Author keywords

Action Units; Computerized method; Facial Action Coding System; Facial expressions

Indexed keywords

ARTICLE; AUTOMATION; CASE REPORT; COMPUTER SYSTEM; CONTROLLED STUDY; EMOTION; FACE; FACE MUSCLE; FACIAL ACTION CODING SYSTEM; FACIAL EXPRESSION; FEMALE; GEOMETRY; HUMAN; INFORMATION PROCESSING; MALE; MENTAL DISEASE; PILOT STUDY; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; RATING SCALE; SCHIZOPHRENIA; TEMPORAL SUMMATION; VIDEORECORDING; VOLUNTARY MOVEMENT;

AUTOMATIC DATA PROCESSING; EMOTIONS; FACIAL EXPRESSION; FACIAL MUSCLES; FEMALE; HUMANS; MALE; MENTAL DISORDERS; NONLINEAR DYNAMICS; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; VIDEO RECORDING;

EID: 80051583841     PISSN: 01650270     EISSN: 1872678X     Source Type: Journal    
DOI: 10.1016/j.jneumeth.2011.06.023     Document Type: Article

References (65)

1. Alvino C., Kohler C., Barrett F., Gur R., Gur R., Verma R. Computerized measurement of facial expression of emotions in schizophrenia. J Neurosci Methods 2007, 163(2):350-361.
2. Andreasen N. The scale for the assessment of negative symptoms (SANS) 1984.
3. Andreasen N. The scale for the assessment of positive symptoms (SAPS) 1984.
4. Bartlett M.S., Littlewort G., Frank M.G., Lainscsek C., Fasel I.R., Movellan J.R. Automatic recognition of facial actions in spontaneous expressions. J Multimedia 2006, 1(6):22-35.
5. Bleuler E. Dementia praecox oder die gruppe der schizophrenien 1911.
6. Cohen I., Sebe N., Cozman G., Cirelo M., Huang T. Learning Bayesian network classifier for facial expression recognition using both labeled and unlabeled data. CVPR 2003.
7. Cohen I., Sebe N., Garg A., Chen L., Huang T. Facial expression recognition video sequences: temporal and static modeling. Comput Vis Image Underst 2003, 91(1-2):160-187.
8. Cootes T.F., Edwards G.J., Taylor C.J. Active appearance models. IEEE Trans Pattern Anal Mach Intell January 2001, 23:681-685.
9. Cootes T.F., Taylor C.J., Cooper D.H., Graham J. Active shape models-their training and application. Comput Vis Image Underst 1995, 61(1):38-59.
10. Craig K., Hyde S., Patrick J. Genuine, suppressed and faked facial behavior during exacerbation of chronic low back pain. Pain 1991, 46:161-171.
11. Cristinacce D., Cootes T. Automatic feature localisation with constrained local models. Pattern Recogn October 2008, 41(10):3054-3067.
12. Del Giudice M., Colle L. Differences between children and adults in the recognition of enjoyment smiles. Dev Psychol 2007, 43(3):796-803.
13. Eibl-Eibesfeldt I. Ethology, the biology of behavior 1970, Holt, Rinehart and Winston, New York.
14. Ekman P., Friesen W. Unmasking the face 1975, Prentice-Hall, Englewood Cliffs.
15. Ekman P., Friesen W. Facial Action Coding System: investigator's guide 1978, Consulting Psychologists Press, Palo Alto, CA.
16. Ekman P., Friesen W. Manual of the Facial Action Coding System (FACS) 1978, Consulting Psychologists Press, Palo Alto, CA.
17. Fasel B., Luettin J. Recognition of asymmetric facial action unit activities and intensities. ICPR 2000, 1100-1103.
18. Fasel B., Luettin J. Automatic facial expression analysis: a survey. Pattern Recogn 2003, 36(1):259-275.
19. Felzenszwalb P.F., Huttenlocher D.P. Pictorial structures for object recognition. Int J Comput Vis 2005, 61(1):55-79.
20. Friedman J., Hastie T., Tibshirani R. Additive logistic regression: a statistical view of boosting. Ann Stat 1998, 28:2000.
21. Gelber E., Kohler C., Bilker W., Gur R., Brensinger C., Siegel S., et al. Symptom and demographic profiles in first-episode schizophrenia. Schizophr Res 2004, 67(2-3):185-194.
22. Gosselin P., Kirouac G., Dor F. Components and recognition of facial expression in the communication of emotion by actors 1995, Oxford University Press, Oxford, p. 243-267.
23. Gross R., Matthews I., Baker S. Generic vs. person specific active appearance models. Image Vis Comput November 2005, 23(1):1080-1093.
24. Gur R., Kohler C., Ragland J.D., Siegel S.J., Lesko K., Bilker W., Gur R. Flat affect in schizophrenia: relation to emotion processing and neurocognitive measures. Schizophr Bull 2006, 32(2):279-287.
25. Gur R., Sara R., Hagendoorn M., Marom O., Hughett P., Macy L., et al. A method for obtaining 3-dimensional facial expressions and its standardization for use in neurocognitive studies. J Neurosci Methods 2002, 115(2):137-143.
26. Izard C. Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychol Bull 1994, 115(2):288-299.
27. Kanade T., Tian Y., Cohn J.F. Comprehensive database for facial expression analysis. Automatic Face Gesture Recogn 2000, 0:46.
28. Koelstra S., Pantic M., Patras I.Y. A dynamic texture-based approach to recognition of facial actions and their temporal models. IEEE Trans Pattern Anal Mach Intell 2010, 32:1940-1954.
29. Kohler C., Gur R., Swanson C., Petty R., Gur R. Depression in schizophrenia: I. association with neuropsychological deficits. Biol Psychiatry 1998, 43(3):165-172.
30. Kohler C., Turner T., Stolar N., Bilker W., Brensinger C., Gur R., et al. Differences in facial expressions of four universal emotions. Psychiatry Res 2004, 128(3):235-244.
31. Kohler C.G., Martin E.A., Stolar N., Barrett F.S., Verma R., Brensinger C., et al. Static posed and evoked facial expressions of emotions in schizophrenia. Schizophr Res 2008, 105.
32. Kotsia I., Pitas I. Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 2007, 16(1):172-187.
33. Kring A., Kerr S., Smith D., Neale J. Flat affect in schizophrenia does not reflect diminished subjective experience of emotion. J Abnorm Psychol 1993, 102(4):507-517.
34. Kring A., Sloan D. The facial expression coding system (faces): development, validation, and utility. Psychol Assess 2007, 19(2):210-224.
35. Larochette A., Chambers C., Craig K. Genuine, suppressed and faked facial expressions of pain in children. Pain 2006, 126:64-71.
36. Lien J., Kanade T., Cohn J., Li C.-C. Detection, tracking, and classification of action units in facial expression. Robot Auton Syst 2000, 31:131-146.
37. Lienhart R., Maydt J. An extended set of haar-like features for rapid object detection. IEEE Conf. ICIP 2002, 900-903.
38. Lints-Martindale A., Hadjistavropoulos T., Barber B., Gibson S. A psychophysical investigation of the facial action coding system as an index of pain variability among older adults with and without Alzheimer's disease. Pain Med 2007, 8(8).
39. Littlewort G., Bartlett M.S., Fasel I.R., Susskind J., Movellan J.R. Dynamics of facial expression extracted automatically from video. Image Vis Comput 2006, 24(6):615-625.
40. Lucey S., Ashraf A.B., Cohn J. Investigating spontaneous facial action recognition through AAM representations of the face. Face recognition book April 2007, Pro Literatur Verlag, Mammendorf, Germany. K. Kurihara (Ed.).
41. Mardia K.V., Dryden I.L. Statistical shape analysis 1998, Wiley, Chichester.
42. Pantic M., Bartlett M. Machine analysis of facial expressions 2007, I-Tech Education and Publishing, Vienna, Austria, p. 377-416.
43. Pantic M., Rothkrantz L. Expert system for automatic analysis of facial expressions. Image Vis Comput August 2000, 18(11):881-905.
44. Pantic M., Rothkrantz L.J. Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 2000, 22:1424-1445.
45. Pantic M., Valstar M., Rademaker R., Maat L. Web-based database for facial expression analysis. IEEE Conf. Multimedia and Expo 2005.
46. Prkachin K. The consistency of facial expressions of pain: a comparison across modalities. Pain 1992, 58:297-306.
47. Prkachin K., Mercer S. Pain expression in patients with shoulder pathology: validity, properties and relationship to sickness impact. Pain 1989, 39:257-265.
48. Reed L., Sayette M., Cohn J. Impact of depression on response to comedy: a dynamic facial coding analysis. J Abnorm Psychol 2007, 116(4):804-809.
49. Rocha E., Prkachin K., Beaumont S., Hardy C., Zumbo B. Pain reactivity and somatization in kindergarten-age children. J Pediatr Psychol 2003, 28:47-57.
50. Schoelkopf B., Smola A.J. Learning with kernels: support vector machines, regularization, optimization, and beyond 2001, MIT Press, Cambridge, MA, USA.
51. Shtasel D., Gur R., Gallacher F., Heimberg C., Gur R. Gender differences in the clinical expression of schizophrenia. Schizophr Res 1992, 7(3):225-231.
52. Stegmann M., Ersbol B., Larsen R. Famea flexible appearance modeling environment. IEEE Trans Med Imaging 2003, 22(10):1319-1431.
53. Tian Y.-L., Kanade T., Cohn J. Handbook of face recognition 2005, Springer.
54. Tian Y.-l., Kanade T., Cohn J.F. Recognizing action units for facial expression analysis. IEEE Trans Pattern Anal Mach Intell 2001, 23(2):97-115.
55. Tong Y., Chen J., Ji Q. A unified probabilistic framework for spontaneous facial action modeling and understanding. IEEE Trans Pattern Anal Mach Intell 2010, 32:258-273.
56. Tong Y., Liao W., Ji Q. Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Trans Pattern Anal Mach Intell 2007, 29(10):1683-1699.
57. Valstar M., Pantic M. Fully automatic facial action unit detection and temporal analysis. IEEE Conf. CVPR 2006, 149.
58. Verma R., Davatzikos C., Loughead J., Indersmitten T., Hu R., Kohler C., et al. Quantification of facial expressions using high-dimensional shape transformations. J Neurosci Methods 2005, 141(1):61-73.
59. Viola P., Jones M. Rapid object detection using a boosted cascade of simple features. IEEE Conf. CVPR 2001, 511-518.
60. Walker E., Grimes K., Davis D., Smith A. Childhood precursors of schizophrenia: facial expressions of emotion. Am J Psychiatry 1993, 150(11):1654-1660.
61. Wang P., Barrett F., Martin E., Milonova M., Gur R.E., Gur R.C., et al. Automated video-based facial expression analysis of neuropsychiatric disorders. J Neurosci Methods 2008, 168:224-238.
62. Wikle C., Berliner L. A Bayesian tutorial for data assimilation. Physica D 2007, 230(1-2):1-16.
63. Yang M.-H., Kriegman D., Ahuja N. Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 2002, 24(1):34-58.
64. Yang P., Liu Q., Metaxas D.N. Boosting coded dynamic features for facial action units and facial expression recognition. IEEE Conf. CVPR 2007, 1-6.
65. Zhang Y., Ji Q. Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans Pattern Anal Mach Intell May 2005, 27:699-714.