Measuring facial expressions by computer image analysis

Psychophysiology

Volumn 36, Issue 2, 1999, Pages 253-263

  Bartlett, Marian Stewart ^a,b   Hager, Joseph C ^c   Ekman, Paul ^d   Sejnowski, Terrence J ^a,e  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b SALK INSTITUTE FOR BIOLOGICAL STUDIES   (United States)

^c Network Information Research Corp   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e HOWARD HUGHES MEDICAL INSTITUTE   (United States)

Author keywords

Computer image analysis; Emotion; Facial Action Coding System (FACS); Facial expression

Indexed keywords

ARTICLE; AUTOMATION; COGNITION; COMPUTER SIMULATION; DATA BASE; DIAGNOSTIC ACCURACY; DIAGNOSTIC ERROR; DIAGNOSTIC IMAGING; EMOTION; FACIAL EXPRESSION; IMAGE ANALYSIS; PRIORITY JOURNAL; SOCIAL INTERACTION; SPATIAL DISCRIMINATION;

EID: 0033057705     PISSN: 00485772     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0048577299971664     Document Type: Article

References (56)

1. Adolphs, R., Damasio, H., Tranel, D., & Damasio, A. (1996). Cortical systems for the recognition of emotion in facial expressions. Journal of Neuroscience, 16, 7678-7687.
2. Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1995). Fear and the human amygdala. Journal of Neuroscience, 15, 5879-5891.
3. Bartlett, M. S., Viola, P. A., Sejnowski, T. J., Golomh, B. A., Larsen, J., Hager, J. C., & Ekman, P. (1996). Classifying facial action. In D. Touretzki, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems (Vol. 8, pp. 823-829). Cambridge. MA: MIT Press.
4. Bassili, J. (1979). Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. Journal of Personality and Social Psychology, 37, 2049-2059.
5. Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Patient Analysis and Machine Intelligence, 15, 711-720.
6. Beymer, D. (1994). Face recognition under varying pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 756-761). Los Alamitos, CA: IEEE Computer Society Press.
7. Beymer, D., Shashua, A., & Poggio, T. (1993). Example based image analysis and synthesis (AI Memo No. 1431). Cambridge, MA: MIT Press.
8. Breiter, H. C., Etcoff, N. L., Whalen, P. J., Kennedy, W. A., Rauch, S. L., Buckner, R. L., Strauss, M. M., Hyman, S. E., & Rosen, B. R. (1996). Response and habituation of the human amygdala during visual processing of facial expression. Neuron, 17, 875-887.
9. Bruce, V. (1988). Recognising faces. London: Erlbaum.
10. Bruce, V., Hancock, P., & Burton, M. (1998). Human face perception and identification. In H. Wechsler, V. Bruce, J. Phillips, S. Fogelman-Soulie, & T. Huang (Eds.), Face recognition, from theory to applications. (NATO ASI Series F). New York: Springer-Verlag.
11. Brunelli, R., & Poggio, T. (1993). Face recognition: Features versus templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15, 1042-1052.
12. Camras, L. A. (1977). Facial expressions used by children in conflict situations. Child Development, 48, 1431-1435.
13. Cohn, J. F., Zlochower, A. J., Lien, J. J., Wu, Y.-T., & Kanade, T. (in press). Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. Psychophysiology.
14. Cottrell, G. W., & Fleming, M. K. (1990). Face recognition using unsupervised feature extraction. Proceedings of the International Neural Network Conference (pp. 322-325). Paris: Kluwer.
15. Cottrell, G., & Metcalfe, J. (1991). EMPATH: Face, gender and emotion recognition using holons. In R. P. Lippman, J. Moody, & D. Touretzky (Eds.), Advances in neural information processing systems (Vol. 3, pp. 564-571). San Mateo, CA: Morgan & Kaufman.
16. Davidson, R. J., Ekman, P., Saron, C., Senulis, J., & Friesen, W. V. (1990). Emotional expression and brain physiology: I. Approach/withdrawal and cerebral asymmetry. Journal of Personality and Social Psychology, 58, 330-341.
17. Ekman, P. (Ed.). (1982a). Emotion in the human face (2nd ed.). New York: Cambridge University Press.
18. Ekman, P. (1982b). Methods for measuring facial action. In K. R. Scherer and P. Ekman (Eds.), Handbook of methods in nonverbal behavior research (pp. 45-135). New York: Cambridge University Press.
19. Ekman, P. (1984). Expression and the nature of emotion. In K. Scherer and P. Ekman (Eds.), Approaches to emotion (pp. 319-343). Hillsdale, NJ: Erlbaum.
20. Ekman, P. (1989). The argument and evidence about universals in facial expressions of emotion. In H. Wagner & A. Manstead (Eds.). Handbook of social psychophysiology, 58, 342-353.
21. Ekman, P., & Friesen, W. (1978). Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.
22. Ekman, P., Huang, T., Sejnowski, T., & Hager. J. (1992). Final Report to NSF of the Planning Workshop on Facial Expression Understanding [on-line]. Available: http://alt.xmission.com/nirc1/Research/ research.html
23. Ekman, P., Levenson, R. W., & Friesen, W. V. (1983). Autonomic nervous system activity distinguishes between emotions. Science, 221, 1208-1210.
24. Ekman, P., & Oster, H. (1979). Facial expressions of emotion. Annual Review of Psychology, 30, 527-554.
25. Ekman, P., & Rosenberg, E. L. (1997). What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). New York: Oxford University Press.
26. Essa, I., & Pentland, A. (1997). Coding, analysis, interpretation, and recognition of facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 757-763.
27. Hager, J., & Ekman, P. (1995). The essential behavioral science of the face and gesture that computer scientists need to know. In M. Bichsel (Ed.), Proceedings of the International Workshop on Automatic Face and Gesture Recognition (pp. 7-11). Available: http://alt.xmission.com/ nirc1/Research/research.html
28. Hancock, P. J. B., Burton, A. M., & Bruce, V. (1996). Face processing: Human perception and principal components analysis. Memory and Cognition, 24, 26-40.
29. Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989). The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey. Behavioural Brain Research, 32, 203-218.
30. Haykin, S. (1994). Neural networks: A comprehensive foundation. Riverside, NJ: Macmillan.
31. Himer, W., Schneider, F., Kost, G., & Heimann, H. (1991). Computer-based analysis of facial action: A new approach. Journal of Psychophysiology, 5, 189-195.
32. Horn, B., & Schunk, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185-203.
33. Jain, R., Kasturi, R., & Schunk, B. G. (1995). Machine vision. New York: McGraw-Hill.
34. Kaiser, S., & Wherle, T. (1992). Automated coding of facial behavior in human-computer interactions with FACS. Journal of Nonverbal Behavior, 16, 65-140.
35. Kanade, T. (1977). Computer recognition of human faces. Basel: Birkhäuser Verlag.
36. Kinomura, S., Kawashima, R., Yamada, K., Ono, S., Itoh, M., Yoshioka, S., Yamaguchi, T., Matsui, H., Miyazawa, H., & Itoh, H. (1994). Functional anatomy of taste perception in the human brain studied with positron emission tomography. Brain Research, 659, 263-266.
37. Lanitis, A., Taylor, C. J., & Cooles, T. F. (1997). Automatic interpretation and coding of face images usrig flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 743-756.
38. Li, H., Roivainen, P., & Forcheirner, R. (1993). 3-D motion estimation in model-based facial image coding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 545-555.
39. Lien, J.-J. (1998). Automatic recognition of facial expressions using hidden Markov models and estimation of expression intensity. Unpublished doctoral dissertation, School of Engineering, University of Pittsburgh, Pittsburgh, PA.
40. Mase, K. (1991). Recognition of facial expression from optical flow. IEICE Transactions E, 74, 3474-3483.
41. Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W., Calder, A. J., & Dolan, R. J. (1996). A differential neural response in the human amygdala to fearful and happy facial expressions. Nature, 383, 812-815.
42. Movellan, J. R. (1995). Visual speech recognition with stochastic networks. In G. Tesauro, D. Touretzky, & T. Leen (Eds.), Advances in neural information processing systems ( Vol. 7, pp. 851 -858). Cambridge, MA: MIT Press.
43. O'Toole, A. J., Deffenbacher, K. A., Valentin, D., & Abdi. H. (1994). Structural aspects of face recognition and the other race effect. Memory and Cognition, 22, 208-224.
44. Padgett, C., & Cottrell, G. (1997). Representing face images for emotion classification. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems (Vol. 9, pp. 894-900). Cambridge, MA: MIT Press.
45. Phillips, M. L., Young, A. W., Senior, C., Brammer, M., Andrews, C., Calder, A. J., Bullmore, E. T., Perrett, D. I., Rowland, D., Williams, S. C. R., Gray, A. J., & David, A. S. (1997). A specific neural substrate for perceiving facial expressions of disgust. Nature, 389, 495-498.
46. Rosenblum, M., Yacoob, Y. & Davis, L. (1996). Human expression recognition from motion using a radial basis function network architecture. IEEE Transactions on Neural Networks, 7, 1121-1138.
47. Terzopoulos, D., & Waters, K. (1993). Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15, 569-579.
48. Tranel, D., Damasio, A., & Damasio, H. (1988). Intact recognition of facial expression, gender, and age in patients with impaired recognition of face identity. Neurology, 38, 690-696.
49. Tukey, J. W. (1958). Bias and confidence in not-quite large samples. Annals of Mathematical Statistics, 29, 614.
50. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3, 71-86.
51. Valentin, D., Abdi, H., O'Toole, A. J., & Coltrell, G. W. (1994). Connectionist models of face processing: A survey. Pattern Recognition, 27, 1209-1230.
52. Vetter, T., & Poggio, T. (1997). Linear object classes and image synthesis from a single example image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15, 733-741.
53. Wallbott, H. (1992). Effects of distortion of spatial and temporal resolution of video stimuli on emotion attributions. Journal of Nonverbal Behavior, 16, 5-20.
54. Yacoob, Y., & Davis, L. (1994). Recognizing human facial expressions from long image sequences using optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18, 636-642.
55. Yaxley, S., Rolls, E. T., & Seinkiewicz, Z. J. (1988). The responsiveness of neurons in the insular gustatory cortex of the macaque monkey is independent of hunger. Physiology and Behavior, 42, 223-229.
56. Zajonc, R. B. (1984). The interaction of affect and cognition. In K. Scherer & P. Ekman (Eds.), Approaches to emotion (pp. 239-246). Hillsdale, NJ: Erlbaum.