Human and computer recognition of facial expressions of emotion

Neuropsychologia

Volumn 45, Issue 1, 2007, Pages 152-162

  Susskind, J M ^a   Littlewort, G ^b   Bartlett, M S ^b   Movellan, J ^b   Anderson, A K ^a,c  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c ROTMAN RESEARCH INSTITUTE   (Canada)

Author keywords

Computational model; Discrimination; Expression similarity; Specialized experts; Support vector machines; Visual features

Indexed keywords

ACCURACY; ADULT; ARTICLE; AUTOMATION; COMPUTER ANALYSIS; COMPUTER MODEL; CONTROLLED STUDY; EMOTION; FACIAL EXPRESSION; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; NERVE CELL; RECOGNITION;

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DISCRIMINATION (PSYCHOLOGY); EMOTIONS; FACIAL EXPRESSION; HUMANS; MODELS, NEUROLOGICAL; PHOTIC STIMULATION; RECOGNITION (PSYCHOLOGY);

EID: 33751439293     PISSN: 00283932     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuropsychologia.2006.05.001     Document Type: Article

References (67)

1. Adelman P., and Zajonc R. Facial efference and the experience of emotion. Annual Review of Psychology 40 (1989) 249-280
2. Adolphs R., Tranel D., Damasio H., and Damasio A.R. Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372 (1994) 669-672
3. Adolphs R., Tranel D., Hamann S., Young A.W., Calder A.J., Phelps E.A., et al. Recognition of facial emotion in nine individuals with bilateral amygdala damage. Neuropsychologia 37 10 (1999) 1111-1117
4. Adolphs R., Damasio H., Tranel D., Cooper G., and Damasio A.R. A role for somatosensory cortices in the visual recognition of emotion as revealed by three-dimensional lesion mapping. The Journal of Neuroscience 20 7 (2000) 2683-2690
5. Adolphs R., Gosselin F., Buchanan T.W., Tranel D., Schyns P., and Damasio A.R. A mechanism for impaired fear recognition after amygdala damage. Nature 433 7021 (2005) 68-72
6. Anderson A.K., and Phelps E.A. Perceiving emotion: There's more than meets the eye. Current Biology 10 15 (2000) R551-R554
7. Anderson A.K., and Sobel N. Dissociating intensity from valence as sensory inputs to emotion. Neuron 39 4 (2003) 581-583
8. Anderson A.K., Spencer D.D., Fulbright R.K., and Phelps E.A. Contribution of the anteromedial temporal lobes to the evaluation of facial emotion. Neuropsychology 14 4 (2000) 526-536
9. Anderson A.K., Christoff K., Panitz D., De Rosa E., and Gabrieli J.D. Neural correlates of the automatic processing of threat facial signals. The Journal of Neuroscience 23 13 (2003) 5627-5633
10. Ashwin, C., Baron-Cohen, S., Wheelwright, S., O'Riordan, M., & Bullmore, E. T. (this issue). Differential activation of the amygdala and the 'social brain' during fearful face processing in Asperger Syndrome. Neuropsychologia.
11. Bartlett M., Littlewort G., Lainscsek C., Fasel I., and Movellan J. Machine learning methods for fully automatic recognition of facial expressions and facial actions. IEEE International Conference on Systems, Man and Cybernetics (2004) 592-597
12. Biehl M., Matsumoto D., Ekman P., Hearn V., Heider K., Kudoh T., et al. Matsumoto and Ekman's Japanese and Caucasian facial expressions of emotion (JACFEE): Reliability data and cross-national differences. Journal of Nonverbal Behavior 21 (1997) 2-21
13. Blair R.J., Morris J.S., Frith C.D., Perrett D.I., and Dolan R.J. Dissociable neural responses to facial expressions of sadness and anger. Brain; a Journal of Neurology 122 Pt 5 (1999) 883-893
14. Burges C.J.C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2 2 (1998) 121-167
15. Calder A.J., Young A.W., Perrett D.I., Etcoff N.L., and Rowland D. Categorical perception of morphed facial expressions. Visual Cognition 3 2 (1996) 81-117
16. Calder A.J., Keane J., Lawrence A.D., and Manes F. Impaired recognition of anger following damage to the ventral striatum. Brain 127 9 (2004) 1958-1969
17. Canli T., Sivers H., Whitfield S.L., Gotlib I.H., and Gabrieli J.D. Amygdala response to happy faces as a function of extraversion. Science 296 5576 (2002) 2191
18. Dailey M.N., Cottrell G.W., and Adolphs R. A six-unit network is all you need to discover happiness. Proceedings of the 22nd Annual Cognitive Science Conference, vol. 22 (2000) 101-106
19. Dailey M.N., Cottrell G.W., Padgett C., and Adolphs R. EMPATH: A neural network that categorizes facial expressions. Journal of Cognitive Neuroscience 14 8 (2002) 1158-1173
20. Damasio A.R. Descartes' error (1994), Putnam, New York
21. Damasio A.R. The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences 351 1346 (1996) 1413-1420
22. Darwin C. The expression of the emotions in man and animals (1872), D. Appleton and Company, New York
23. Davidson R.J. Asymmetric brain function, affective style and psychopathology: The role of early experience and plasticity. Development and Psychopathology 6 (1994) 741-758
24. Downing P.E., Jiang Y., Shuman M., and Kanwisher N. A cortical area selective for visual processing of the human body. Science 293 5539 (2001) 2470-2473
25. Ekman P. Universals and cultural differences in facial expressions of emotion. In: Cole J. (Ed). Nebraska symposium on motivation, 1971 (1972), University of Nebraska Press, Lincoln 207-283
26. Ekman P. An argument for basic emotions. Cognition and Emotion 6 3-4 (1992) 169-200
27. Ekman P., and Friesen W.V. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 17 2 (1971) 124-129
28. Ekman P., and Friesen W. In: Ekman P. (Ed). Pictures of facial affect (1976), Human Interaction Laboratory, UCSF, HIL-0984, San Francisco, CA 94143
29. Elfenbein H.A., Marsh A., and Ambady N. Emotional Intelligence and the recognition of emotion from the face. In: Barrett L.F., and Salovey P. (Eds). The wisdom of feelings: Processes underlying emotional intelligence (2002), Guilford Press, New York 37-59.
30. Etcoff N.L., and Magee J.J. Categorical perception of facial expressions. Cognition 44 3 (1992) 227-240
31. Fasel, I., Dahl, R., Hershey, J., Fortenberry, B., Susskind, J., & Movellan, J. (2004). The machine perception toolbox. Retrieved 10/07, 2004 from http://www.mplab.ucsd.edu/projects-home/project1/free-software/MPTWebSite/API/index.html.
32. Fasel I., Fortenberry B., and Movellan J. A generative framework for real time object detection and classification. Computer Vision and Image Understanding 98 1 (2005) 182-210
33. Fernandez-Dols J.M., and Carroll J.M. Is the meaning perceived in facial expression independent of its context?. In: Russell J.A., and Fernandez-Dols J.M. (Eds). The psychology of facial expression (1997), Cambridge University Press 275-294
34. Freeman P.R. Optimal Bayesian sequential estimation of the median effective dose. Biometrika 57 (1970) 79-89
35. Friedman J., Hastie T., and Tibshirani R. Additive logistic regression: A statistical view of boosting. Annals of Statistics 28 2 (2000) 337-407
36. Hamann S.B., and Adolphs R. Normal recognition of emotional similarity between facial expressions following bilateral amygdala damage. Neuropsychologia 37 10 (1999) 1135-1141
37. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten J.L., and Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293 5539 (2001) 2425-2430
38. Hennenlotter A., Schroeder U., Erhard P., Haslinger B., Stahl R., Weindl A., et al. Neural correlates associated with impaired disgust processing in pre-symptomatic Huntington's disease. Brain; a Journal of Neurology 127 Pt 6 (2004) 1446-1453
39. Kanade T., Cohn J.F., and Tian Y.L. Comprehensive database for facial expression analysis. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG'00) (2000) 46-53
40. Kanwisher N., McDermott J., and Chun M.M. The fusiform face area: A module in human extrastriate cortex specialized for face perception. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 17 11 (1997) 4302-4311
41. Lades M., Vorbruggen J.C., Buhmann J., Lange J., Malsburg C.v.d., Wurtz R.P., et al. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers 42 3 (1993) 300-311
42. Lawrence, A. D., Goerendt, I. K., & Brooks, D. J. (this issue). Impaired recognition of facial expressions of anger in Parkinson's disease patients acutely withdrawn from dopamine replacement therapy. Neuropsychologia.
43. Linkser R. Self-organization in a percentual network. IEEE Computer 21 (1988) 105-117
44. Littlewort G., Bartlett M.S., Fasel I., Susskind J., and Movellan J. Dynamics of facial expression extracted automatically from video. IEEE Conference on Computer Vision and Pattern Recognition, (Workshop on Face Processing in Video) (2004)
45. Morris J.S., Friston K.J., Buchel C., Frith C.D., Young A.W., Calder A.J., et al. A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain; a Journal of Neurology 121 Pt 1 (1998) 47-57
46. Movellan, J. (2002). Tutorial on gabor filters (http://www.mplab.ucsd.edu/tutorials/pdfs/Gabor.pdf).
47. Neville H.J., and Bavelier D. Specificity and plasticity in neurocognitive development in humans. In: Gazzaniga M.S. (Ed). The cognitive neurosciences (2000), MIT Press, Cambridge, MA 83-98
48. Phillips M.L., Young A.W., Senior C., Brammer M., Andrew C., Calder A.J., et al. A specific neural substrate for perceiving facial expressions of disgust. Nature 389 6650 (1997) 495-498
49. Phillips M.L., Young A.W., Scott S.K., Calder A.J., Andrew C., Giampietro V., et al. Neural responses to facial and vocal expressions of fear and disgust. Proceedings of the Royal Society of London. Series B: Biological Sciences 265 1408 (1998) 1809-1817
50. Plutchik R. A general psychoevolutionary theory of emotion. In: Plutchik R., and Kellerman H. (Eds). Emotion: Theory, research, and experience: Theories of emotion vol. 1 (1980), Academic, New York 3-33
51. Rapcsak S.Z., Galper S.R., Comer J.F., Reminger S.L., Nielsen L., Kaszniak A.W., et al. Fear recognition deficits after focal brain damage: A cautionary note. Neurology 54 3 (2000) 575-581
52. Russell J.A. A circumplex model of affect. Journal of Personality and Social Psychology 39 (1980) 1161-1178
53. Russell J.A. Culture and the categorization of emotions. Psychological Bulletin 110 3 (1991) 426-450
54. Russell J.A., and Bullock M. Fuzzy concepts and the perception of emotion in facial expressions. Social Cognition 4 3 (1986) 309-341
55. Russell J.A., and Mehrabian A. Evidence for a three-factor theory of emotions. Journal of Research in Personality 11 (1977) 273-294
56. Russell, T. A., Reynaud, E., Pietura, K., Ecker, C., Benson, P. J., Zelaya, F., et al. (this issue). Neural responses to dynamic expressions of fear in schizophrenia. Neuropsychologia.
57. Scherer K.R. On the nature and function of emotion: A component process approach. In: Scherer K.R., and Ekman P. (Eds). Approaches to emotion (1984), Erlbaum, Hillsdale, NJ 293-317
58. Scherer K.R. Criteria for emotion-antecedent appraisal: A review. In: Hamilton V., Bower G.H., and Frijda N.H. (Eds). Cognitive perspectives on emotion and motivation (1988), Kluwer, Dordrecht 89-126
59. Scherer K.R. Appraisal considered as a process of multi-level sequential checking. In: Scherer K.R., Schorr A., and Johnstone T. (Eds). Appraisal processes in emotion: Theory, methods, research (2001), Oxford University Press, New York and Oxford 92-120
60. Schlosberg H. The description of facial expressions in terms of two dimensions. Journal of Experimental Psychology 44 (1952) 229-237
61. Shaver P., Schwartz J., Kirson D., and O'Connor C. Emotion knowledge: Further exploration of a prototype approach. Journal of Personality and Social Psychology 52 6 (1987) 1061-1086
62. Suzuki A., Hoshino T., Shigemasu K., and Kawamura M. Disgust-specific impairment of facial expression recognition in Parkinson's disease. Brain; a Journal of Neurology 129 Pt 3 (2006) 707-717
63. Vapnik V. Statistical learning theory (1998), John Wiley and Sons Inc., New York
64. Vuilleumier P., Armony J.L., Driver J., and Dolan R.J. Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nature Neuroscience 6 6 (2003) 624-631
65. Whalen P.J., Kagan J., Cook R.G., Davis F.C., Kim H., Polis S., et al. Human amygdala responsivity to masked fearful eye whites. Science 306 5704 (2004) 2061
66. Wright C.I., Martis B., Shin L.M., Fischer H., and Rauch S.L. Enhanced amygdala responses to emotional versus neutral schematic facial expressions. Neuroreport 13 6 (2002) 785-790
67. Young A.W., Rowland D., Calder A.J., Etcoff N.L., Seth A., and Perrett D.I. Facial expression megamix: Tests of dimensional and category accounts of emotion recognition. Cognition 63 3 (1997) 271-313