Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding

Psychophysiology

Volumn 36, Issue 1, 1999, Pages 35-43

  Cohn, Jeffrey F ^a,c   Zlochower, Adena J ^a   Lien, James ^a   Kanade, Takeo ^b  

^a UNIVERSITY OF PITTSBURGH   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c Clinical Psychology Program   (United States)

Author keywords

Computer vision; Facial expression; FACS; Optical flow

Indexed keywords

ARTICLE; COMPUTER ANALYSIS; FACE PROFILE; FACIAL EXPRESSION; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; IMAGE DISPLAY; NORMAL HUMAN; PRIORITY JOURNAL; VIDEOTAPE;

EID: 0032924381     PISSN: 00485772     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0048577299971184     Document Type: Article

References (64)

1. Bakeman, R., & Gottman, J. M. (1986). Observing behavior: An introduction to sequential analysis. Cambridge: Cambridge University Press.
2. Bartlett, M., Stewart, S., Viola, P. A., Sejnowski, T. J., Golomb, B.A., Larsen, J., Hager, J. C., & Ekman, P. (1996). Classifying facial action. Advances in Neural Information Processing Systems, 8, 823-829.
3. Basu, S., Essa, I., & Pentland, A. (1996). Motion regularization for model-based head tracking. Proceedings of the International Conference on Pattern Recognition (pp. 611-616).
4. Black, M. J., & Yacoob, Y. (1995). Recognizing facial expressions under rigid and non-rigid facial motions. Proceedings of the IEEE International Workshop on Automatic Face and Gesture Recognition (pp. 12-17).
5. Black, M. J., Yacoob, Y., Jepson, A. D., & Fleet, D. J. (1997). Learning parameterized models of image motion. Proceedings of the International Conference on Computer Vision and Pattern Recognition (pp. 561-567).
6. Campos, J. J., Bertenthal, B. I., & Kermoian, R. (1992). Early experience and emotional development: The emergence of wariness of heights. Psychological Science, 3, 61-64.
7. Camras, L. (1992). Expressive development and basic emotions. Cognition and Emotion, 6, 269-283.
8. Carroll, J. M., & Russell, J. A. (1997). Facial expressions in Hollywood's portrayal of emotion. Journal of Personality and Social Psychology, 72, 164-176.
9. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
10. Cohn, J. F., & Elmore, M. (1988). Effect of contingent changes in mothers' affective expression on the organization of behavior in 3-month-old infants. Infant Behavior and Development, 11, 493-505.
11. Cohn, J. F., & Tronick, E. Z. (1988). Mother-infant interaction: Influence is bidirectional and unrelated to periodic cycles in either partner's behavior. Developmental Psychology, 24, 386-392.
12. Cohn, J. F., Zlochower, A., Lien, J., Wu, Y. T., & Kanade, T. (1996). In N. H. Frijda (Ed.), Proceedings of the 9th Conference of the International Society for Research on Emotions (pp. 329-333). Toronto, Canada.
13. Cottrell, G. W., & Metcalfe (1991). EMPATH: Face, emotion, and gender recognition using hologons. In R. P. Lippmann, J. E. Moody, & D. S. Touretzky (Eds.), Neural information processing systems (Vol. 3, pp. 564-571). San Mateo, CA: Morgan Kaufmann.
14. Craig, K. D., Hyde, S. A., & Patrick, C. J. (1991). Genuine, suppressed and faked facial behavior during exacerbation of chronic low back pain. Pain, 46, 161-171.
15. DeCarlo, D., & Metaxas, D. (1995). Deformable model-based face shape and motion estimation. Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (pp. 146-150).
16. DePaulo, B. M. (1992). Nonverbal behavior and self-presentation. Psychological Bulletin, 111, 203-243.
17. Duda, R. O., & Hart, P. E. (1973). Pattern classification and analysis. New York: Wiley.
18. Ekman, P. (1982). Methods for measuring facial action. In K.R. Scherer & P. Ekman (Eds.), Handbook of methods in nonverbal behavior research (pp. 45-90). Cambridge: Cambridge University Press.
19. Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48, 384-392.
20. Ekman, P., Davidson, R. J., & Friesen, W. V. (1990). The Duchenne smile: Emotional expression and brain physiology II. Journal of Personality and Social Psychology, 58, 342-353.
21. Ekman, P., & Friesen, W. (1976). Pictures of facial affect. Palo Alto, CA: Consulting Psychologist Press.
22. Ekman, P., & Friesen, W. V. (1978a). Facial Action Coding System. Palo Alto, CA: Consulting Psychologist Press.
23. Ekman, P., & Friesen, W. V. (1978b). Facial Action Coding System: Investigator's guide Part I. Palo Alto. CA: Consulting Psychologist Press.
24. Ekman, P., & Friesen, W. V. (1982). Felt, false, and miserable smiles. Journal of Nonverbal Behavior, 6, 238-252.
25. Ekman, P., & Rosenberg, E. (1997). What the face reveals. New York: Oxford University Press.
26. Emde, R. N., Gaensbauer, T. J., & Harmon, R. J. (1976). Emotional expression in infancy: A biobehavioral study. Psychological Issues: Monograph Series, No. 37. New York: International Universities.
27. Essa, I. A., & Pentland, A. (1994). A vision system for observing and extracting facial action parameters. Proceedings of the International Conference on Computer Vision and Pattern Recognition (pp. 76-83).
28. Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.
29. Fox, N., & Davidson, R. J. (1988). Patterns of brain electrical activity during facial signs of emotion in ten-month-old infants. Developmental Psychology, 24, 230-236.
30. Frank, M. G., Ekman, P., & Friesen, W. V. (1993). Behavioral markers and recognizability of the smile of enjoyment. Journal of Personality and Social Psychology, 64, 83-93.
31. Fridlund, A. J. (1994). Human facial expression: An evolutionary view. New York: Academic Press.
32. Friesen, W. V., & Ekman, P. (1984). EMFACS-7: Emotional Facial Action Coding System. Unpublished manuscript, University of California at San Francisco.
33. Gosselin, P., Kirouac, G., & Dore, F. Y. (1995). Components and recognition of facial expression in the communication of emotion by actors. Journal of Personality and Social Psychology, 68, 83-96.
34. Izard, C. E. (1983). The Maximally Discriminative Facial Movement Coding System. Unpublished manuscript, University of Delaware.
35. Izard, C. E., Dougherty, L. M., & Hembree, E. A. (1983). A system for identifying affect expressions by holistic judgments. Unpublished manuscript, University of Delaware.
36. Kanade, T. (1973). Picture processing system by computer complex and recognition of human faces. Unpublished doctoral dissertation, Kyoto University, Kyoto.
37. Kanade, T. (1977). Computer recognition of human faces. Stuttgart: Birkhauser Verlag.
38. Katsikitis, M., & Pilowsky, I. (1988). A study of facial expression in Parkinson's disease using a novel microcomputer-based method. Journal of Neurology, Neurosurgery, and Psychiatry, 51, 362-366.
39. Lien, J. J., Kanade, T., Cohn, J. F., & Li, C. C. (1998a). Subtly different facial expression recognition and expression intensity estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 853-859).
40. Lien, J. J., Zlochower, A., Cohn, J. F., & Kanade, T. (1998b). Automated facial expression recognition. Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (pp. 390-395).
41. Lucas, B. D., & Kanade, T. (1981). An iterative image registration technique with an application in stereo vision. Proceedings of the International Joint Conference on Artificial Intelligence, 7, 674-679.
42. Malatesta, C. Z., Culver, C., Tesman, J. R., & Shephard, B. (1989). The development of emotion expression during the first two years of life. Monographs of the Society for Research in Child Development, 54 (Serial No. 219).
43. Martin, P., & Bateson, P. (1986). Measuring behavior: An introductory guide. Cambridge: Cambridge University Press.
44. Mase, K. (1991). Recognition of facial expression from optical flow. IEICE of Japan Transactions, 10, 3474-3483.
45. Mase, K., & Pentland, A. (1990). Lip reading by optical flow. IEICE of Japan Transactions, 6, 796-803.
46. Matias, R., Cohn, J. F., & Ross, S. (1989). A comparison of two systems to code infants' affective expression. Developmental Psychology, 25, 483-489.
47. Moore, G., Cohn, J. F., & Campbell, S. B. (1997). Mothers' affective behavior with infant siblings: Stability and change. Developmental Psychology, 33, 856-860.
48. Narayanan, P.J., Rander, P., & Kanade, T. (1998). Constructing virtual worlds using dense flow. Proceedings of the International Conference on Computer Vision (pp. 3-10).
49. Oster, H., Hegley, D., & Nagel, L. (1992). Adult judgments and fine-grained analysis of infant facial expressions: Testing the validity of a priori coding formulas. Developmental Psychology, 28, 1115-1131.
50. Oster, H., & Rosenstein, D. (1993). Baby FACS: Analyzing facial movement in infants. Unpublished manuscript, New York University.
51. Padgett, C., Cottrell, G. W., & Adolphs, B. (1996). Categorical perception in facial emotion classification. Proceedings of the Cognitive Science Conference, 18, 249-253.
52. Parke, F. I., & Waters, K. (1996). Computer facial animation. Wellesley, MA: A. K. Peters.
53. Phillips, J. (1996, October). Face recognition: Where are we now and where are we going. Panel session conducted at the International Conference on Automatic Face and Gesture Recognition, Killington, VT.
54. Poelman, C. J. (1995). The paraperspective and projective factorization methods for recovering shape and motion (Technical Report CMU-CS-95-173). Pittsburgh, PA: Carnegie Mellon University, Robotics Institute.
55. Rinn, W. E. (1984). The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions. Psychological Bulletin, 95, 52-77.
56. Rinn, W. E. (1991). Neuropsychology of facial expression. In R.S. Feldman & B. Rime (Eds.), Fundamentals of nonverbal behavior (pp. 3-30). New York: Cambridge University Press.
57. Rosenblum, M., Yacoob, Y., & Davis, L. S. (1994). Human emotion recognition from motion using a radial basis function network architecture. Proceedings of the Workshop on Motion of Non-rigid and Articulated Objects (pp. 43-49).
58. Russell, J. A. (1994). Is there universal recognition of emotion from facial expression? Psychological Bulletin, 115, 102-141.
59. Tronick, E., Als, H., & Brazelton, T. B. (1980). Monadic phases: A structural descriptive analysis of infant-mother face-to-face interaction. Merrill-Palmer Quarterly of Behavior and Development, 26, 3-24.
60. Turk, M. A., & Pentland, A. P. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3, 71-86.
61. Vetter, T. (1995). Learning novels views to a single face image. Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (pp. 22-29).
62. Wu, Y. T., Kanade, T., Cohn, J. F., & Li, C. C. (1998). Optical flow estimation using wavelet motion model. Proceedings of the International Conference on Computer Vision (pp. 992-998).
63. Yacoob, Y., & Davis, L. (1994). Computing spatio-temporal representations of human faces. Proceedings in Computer Vision and Pattern Recognition (pp. 70-75).
64. Zlochower, A. J., Cohn, J. F., Lien, J. J., & Kanade, T. (April, 1998). Automated face coding: A computer vision based method of facial expression analysis in parent-infant interaction. Paper presented at the International Conference on Infant Studies, Atlanta, GA.