Understanding the recognition of facial identity and facial expression

Nature Reviews Neuroscience

Volumn 6, Issue 8, 2005, Pages 641-651

  Calder, Andrew J ^a   Young, Andrew W ^b  

^a MEDICAL RESEARCH COUNCIL   (United Kingdom)

^b UNIVERSITY OF YORK   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN REGION; FACIAL EXPRESSION; HUMAN; IDENTITY; INFORMATION PROCESSING; NEUROPHYSIOLOGY; NEUROPSYCHOLOGY; NONHUMAN; PERCEPTION; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; RECOGNITION; REVIEW; VISION; VISUAL SYSTEM;

FACE; HUMANS; PATTERN RECOGNITION, VISUAL; RECOGNITION (PSYCHOLOGY); VISUAL PATHWAYS;

EID: 23044455404     PISSN: 1471003X     EISSN: None     Source Type: Journal    
DOI: 10.1038/nrn1724     Document Type: Review

References (125)

1. Bruce, V. & Young, A. W. Understanding face recognition. Br. J. Psychol. 77, 305-327 (1986). A theoretical paper that proposes a multiroute model of face perception in which different facial attributes are processed by distinct functional routes.
2. Haxby, J. V., Hoffman, E. A. & Gobbini, M. I. The distributed human neural system for face perception. Trends Cogn. Sci. 4, 223-233 (2000). Presents a neural model of face perception that comprises 'core' and 'extended' systems. The core system differentiates mechanisms for coding changeable facial properties and mechanisms coding invariant facial properties. The extended system includes neural regions that are involved in semantics, language, emotion and attention, which support the recognition of different facial characteristics.
3. Bruce, V. Influences of familiarity on the processing of faces. Perception 15, 387-397 (1986).
4. Campbell, R., Brooks, B., de Haan, E. & Roberts, T. Dissociating face processing skills: decisions about lipread speech, expression and identity. Q. J. Exp. Psychol. A 49A, 295-314 (1996).
5. Young, A. W., McWeeny, K. H., Hay, D. C. & Ellis, A. W. Matching familiar and unfamiliar faces on identity and expression. Psychol. Res. 48, 63-68 (1986).
6. Calder, A. J., Young, A. W., Keane, J. & Dean, M. Configural information in facial expression perception. J. Exp. Psychol. Hum. Percept. Perform. 26, 527-551 (2000).
7. Etcoff, N. L. Selective attention to facial identity and facial emotion. Neuropsychologia 22, 281-295 (1984).
8. Young, A. W., Newcombe, F., de Haan, E. H. F., Small, M. & Hay, D. C. Race perception after brain injury: selective impairments affecting identity and expression. Brain 116, 941-959 (1993).
9. Bruyer, R. et al. A case of prosopagnosia with some preserved covert remembrance of familiar faces. Brain Cogn. 2, 257-284 (1983).
10. Tranel, D., Damasio, A. R. & Damasio, H. Intact recognition of facial expression, gender, and age in patients with impaired recognition of face identity. Neurology 38, 690-696 (1988).
11. Hornak, J., Rolls, E. T. & Wade, D. Face and voice expression identification in patients with emotion and behavioural changes following ventral frontal lobe damage. Neuropsychologia 34, 247-261 (1996).
12. Hasselmo, M. E., Rolls, E. T. & Baylis, G. C. The role of expression and identity in face-selective responses of neurons in the temporal visual cortex of the monkey. Behav. Brain Res. 32, 203-218 (1989).
13. George, M. S. et al. Brain-regions involved in recognizing facial emotion or identity: an oxygen-15 PET study. J. Neuropsychiatry Clin. Neurosci. 5, 384-394 (1993).
14. Sergent, J., Ohta, S., MacDonald, B. & Zuck, E. Segregated processing of identity and emotion in the human brain: a PET study. Vis. Cogn. 1, 349-369 1994).
15. Winston, J. S., Henson, R. N. A., Fine-Goulden, M. R. & Dolan, R. J. fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. J. Neurophysiol. 92, 1830-1839 (2004).
16. Parry, F. M., Young, A. W., Saul, J. S. M. & Moss, A. Dissociable face processing impairments after brain injury. J. Clin. Exp. Neuropsychol. 13, 545-558 (1990).
17. Shuttleworth, E. C. J., Syring, V. & Allen, N. Further observations on the nature of prosopagnosia. Brain Cogn. 1, 307-322 (1982).
18. Hanley, J. R., Pearson, N. & Young, A. W. Impaired memory for new visual forms. Brain 113, 1131-1148 (1990). Reports the first detailed case study of a patient who showed impaired learning of faces (prosopamnesia) and of visual forms in general after a brain injury.
19. Tippet, L. J., Milller, L. A. & Farah, M. J. Prosopamnesia: a selective impairment in face learning. Cogn. Neuropsychol. 17, 241-255 (2000).
20. Hanley, J. R., Young, A. W. & Pearson, N. Defective recognition of familiar people. Cogn. Neuropsychol. 6, 179-210 (1989).
21. Evans, J. J., Heggs, A. J., Antoun, N. & Hodges, J. R. Progressive prosopagnosia associated with selective right temporal lobe atrophy: a new syndrome? Brain 118, 1-13 (1995).
22. Ellis, A. W., Young, A. W. & Critchley, E. M. R. Loss of memory for people following temporal lobe damage. Brain 112, 1469-1483 (1989). Reports the first detailed case study of a patient who showed impaired access to person-specific semantic information from several modalities.
23. Malone, D. R., Morris, H. H., Kay, M. C. & Levin, H. S. Prosopagnosia: a double dissociation between the recognition of familiar and unfamiliar faces. J. Neurol. Neurosurg. Psychiatry 45, 820-822 (1982).
24. Ekman, P. & Friesen, W. V. Pictures of Facial Affect (Consulting Psychologists, Palo Alto, California, 1976).
25. Sprengelmeyer, R. et al. Loss of disgust: perception of faces and emotions in Huntington's disease. Brain 119, 1647-1665 (1996).
26. Calder, A. J., Keane, J., Manes, F., Antoun, N. & Young, A. W. Impaired recognition and experience of disgust following brain injury. Nature Neurosci. 3, 1077-1078 (2000).
27. Rapcsak, S. Z. et al. Fear recognition deficits after focal brain damage: a cautionary note. Neurology 54, 575-581 (2000).
28. Bowers, D., Bauer, R. M., Coslett, H. B. & Heilman, K. M. Processing of faces by patients with unilateral hemisphere lesions. 1. Dissociation between judgements of facial affect and facial identity. Brain Cogn. 4, 258-272 (1985).
29. Shallice, T. From Neuropsychology to Mental Structure (Cambridge Univ. Press, Cambridge, 1988).
30. Duchaine, B. C., Paerker, H. & Nakayama, K. Normal recognition of emotion in a prosopagnosic patient. Perception 32, 827-839 (2003).
31. Bishop, D. V. M. Cognitive neuropsychology and developmental disorders: uncomfortable bedfellows. Q. J. Exp. Psychol. A 50A, 899-923 (1997).
32. Karmiloff-Smith, A., Scerif, G. & Ansari, D. Double dissociations in developmental disorders? Theoretically misconceived, empirically dubious. Cortex 39, 161-163 (2003).
33. Turk, M. & Pentland, A. Eigenfaces for recognition. J. Cogn. Neurosci. 3, 71-86 (1991).
34. Burton, A. M., Bruce, V. & Hancock, P. J. B. From pixels to people: a model of familiar face recognition. Cogn. Sci. 23, 1-31 (1999). Presents a model of facial identity recognition in which the front end is implemented as a PCA, and the subsequent cognitive processes are implemented as an interactive activation and competition architecture.
35. Hancock, P. J. B., Burton, A. M. & Bruce, V. Face processing: human perception and principal components analysis. Mem. Cognit. 24, 26-40 (1996).
36. Hancock, P. J. B., Bruce, V. & Burton, A. M. A comparison of two computer-based face identification systems with human perception of faces. Vision Res. 38, 2277-2288 (1998).
37. O'Toole, A. J. et al. The perception of face gender: the role of stimulus structure in recognition and classification. Mem. Cognit. 26, 146-160 (1998).
38. O'Toole, A. J., Abdi, H., Deffenbacher, K. A. & Valentin, D. Low-dimensional representation of faces in higher dimensions of the face space. J. Opt. Soc. Am. 10, 405-411 (1993).
39. O'Toole, A. J., Deffenbacher, K. A., Valentin, D. & Abdi, H. Structural aspects of face recognition and the other race effect. Mem. Cognit. 22, 208-224 (1994).
40. Padgett, C. & Cottrell, G. In Proceedings of the 2nd Joint Symposium on Neural Computation Vol. 5, 91-101 (La Jolla, Univ. California, San Diego, 1995).
41. Cottrell, G. W., Branson, K. M. & Calder, A. J. in 24th Annual Meeting of the Cognitive Science Society (Fairfax, Virginia, 2002).
42. Calder, A. J., Burton, A. M., Miller, P., Young, A. W. & Akamatsu, S. A principal component analysis of facial expressions. Vision Res. 41, 1179-1208 (2001). Provides evidence that a PCA of the Ekman and Friesen facial expressions (reference 24) generates statistical dissociation between PCs that code identity and expression.
43. Costen, N., Craw, I. & Akamatsu, S. in British Machine Vision Association (Scottish Chapter) Meeting (British Machine Vision Association, Glasgow, 1995).
44. Deffenbacher, K. A., Vetter, T., Johanson, J. & O'Toole, A. J. Facial aging, attractiveness, and distinctiveness. Perception 27, 1233-1243 (1998).
45. Schweinberger, S. R. & Soukup, G. R. Asymmetric relationships among perceptions of facial identity, emotion, and facial speech. J. Exp. Psychol. Hum. Percept. Perform. 24, 1748-1765 (1998).
46. Narumoto, J., Okada, T., Sadato, N., Fukui, K. & Yonekura, Y. Attention to emotion modulates fMRI activity in human right superior temporal sulcus. Cogn. Brain Res. 12, 225-231 (2001).
47. Sugase, Y., Yamane, S., Ueno, S. & Kawano, K. Global and fine information coded by single neurons in the temporal visual cortex. Nature 400, 869-873 (1999).
48. Phillips, M. L. et al. Neural responses to facial and vocal expressions of fear and disgust. Proc. R. Soc. Lond. B 265, 1809-1817 (1998).
49. Morris, J. S. et al. A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain 121, 47-57 (1998).
50. Lewis, S. et al. Visual processing of facial affect. Neuroreport 14, 1841-1845 (2003).
51. Streit, M. et al. Neurophysiological correlates of the recognition of facial expressions of emotion as revealed by magnetoencephalography. Cogn. Brain Res. 7, 481-491 (1999).
52. Kanwisher, N., McDermott, J. & Chun, M. M. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 17, 4302-4311 (1997).
53. McCarthy, G., Puce, A., Core, J. C. & Allison, T. Face-specific processing in the human fusiform gyrus. J. Cogn. Neurosci. 9, 605-610 (1997).
54. Henson, R., Shallice, T. & Dolan, R. Neuroimaging evidence for dissociable forms of repetition priming. Science 287, 1269-1272 (2000).
55. Perrett, D. I. et al. Neurones responsive to faces in the temporal cortex: studies of functional organization, sensitivity to identity and relation to perception. Hum. Neurobiol. 3, 197-208 (1984).
56. Perrett, D. I., Hietanen, J. K., Oram, M. W. & Benson, P. J. Organization and functions of cells responsive to faces in the temporal cortex. Phil. Trans. R. Soc. Lond. B B335, 23-30 (1992).
57. Young, M. P. & Yamane, S. Sparse population coding of faces in the inferotemporal cortex. Science 256, 1327-1331 (1992). Shows that face-responsive cells in the inferotemporal cortex of monkeys code visual properties, whereas face-responsive cells in the STS code nonvisual properties, such as familiarity or social status.
58. Yamane, S., Kaji, S. & Kawano, K. What facial features activate face neurones in the inferotemporal cortex in the monkey? Exp. Brain Res. 73, 209-214 (1988).
59. Rolls, E. T. & Tovee, M. J. Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. J. Neurophysiol. 73, 713-726 (1995).
60. Seltzer, B. & Pandya, D. N. Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Res. 149, 1-24 (1978).
61. Bruce, C., Desimone, R. & Gross, C. G. Visual properties of neurons in a polysensory area in superior temporal cortex. J. Neurophysiol. 46, 369-384 (1981).
62. Adolphs, R., Tranel, D., Damasio, H. & Damasio, A. Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372, 669-672 (1994).
63. Sprengelmeyer, R. et al. Knowing no fear. Proc. R. Soc. Lond. B 266, 2451-2456 (1999).
64. Calder, A. J. et al. Facial emotion recognition after bilateral amygdala damage: differentially severe impairment of fear. Cogn. Neuropsychol. 13, 699-745 (1996).
65. Calder, A. J., Lawrence, A. D. & Young, A. W. The neuropsychology of fear and loathing. Nature Rev. Neurosci. 2, 352-363 (2001).
66. Sprengelmeyer, R. et al. Disgust implicated in obsessive-compulsive disorder. Proc. R. Soc. Lond. B 264, 1767-1773 (1997).
67. Calder, A. J., Keane, J., Lawrence, A. D. & Manes, F. Impaired recognition of anger following damage to the ventral striatum. Brain 127, 1958-1969 (2004).
68. Lawrence, A. D., Calder, A. J., McGowan, S. W. & Grasby, P. M. Selective disruption of the recognition of facial expressions of anger. Neuroreport 13, 881-884 (2002).
69. Krolak-Salmon, P. et al. A specific response to disgust modulated by attention in human ventral anterior insula. Ann. Neurol. 53, 446-453 (2003).
70. Morris, J. S. et al. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature 383, 812-815 (1996).
71. Breiter, H. C. et al. Response and habituation of the human amygdala during visual processing of facial expression. Neuron 17, 875-887 (1996).
72. Phillips, M. L. et al. A specific neural substrate for perceiving facial expressions of disgust. Nature 389, 495-498 (1997).
73. Lawrence, A. D. & Calder, A. J. in Emotion, Evolution, and Rationality (eds Evans, D. & Cruse, P.) 15-47 (Oxford Univ. Press, Oxford, 2003).
74. Scott, S. K. et al. Impaired auditory recognition of fear and anger following bilateral amygdala lesions. Nature 385, 254-257 (1997).
75. Adolphs, R. & Tranel, D. Intact recognition of emotional prosody following amygdala damage. Neuropsychologia 37, 1285-1292 (1999).
76. Anderson, A. K. & Phelps, E. A. Intact recognition of vocal expressions of fear following bilateral lesions of the human amygdala. Neuroreport 9, 3607-3613 (1998).
77. Wicker, B. et al. Both of us disgusted in my insula: the common neural basis of seeing and feeling disgust. Neuron 40, 655-664 (2003).
78. Gallese, V., Keysers, C. & Rizzolatti, G. A unifying view of the basis of social cognition. Trends Cogn. Sci. 8, 396-403 (2004).
79. Goldman, A. I. & Sekhar Sripada, C. Simulationist models of face-based emotion recognition. Cognition 94, 193-213 (2005).
80. Keane, J., Calder, A. J., Hodges, J. R. & Young, A. W. Face and emotion processing in frontal variant frontotemporal dementia. Neuropsychologia 40, 655-665 (2002).
81. Adolphs, R., Tranel, D. & Damasio, H. Neural systems for recognizing emotion from prosody. Emotion 2, 23-51 (2002).
82. Adolphs, R., Damasio, H., Tranel, D., Cooper, G. & Damasio, A. R. A role for somatosensory cortices in the visual recognition of emotion as revealed by three-dimensional lesion mapping. J. Neurosci. 20, 2683-2690 (2000).
83. Anderson, A. K. & Phelps, E. A. Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature 411, 305-309 (2001).
84. Anderson, A. K., Spencer, D. D., Fulbright, R. K. & Phelps, E. A. Contribution of the anteromedial temporal lobes to the evaluation of facial emotion. Neuropsychology 14, 526-536 (2000).
85. Labar, K. S., LeDoux, J. E., Spencer, D. D. & Phelps, E. A. Impaired fear conditioning following unilateral temporal lobectomy in humans. J. Neurosci. 15, 6846-6855 (1995).
86. LaBar, K. S. & Phelps, E. A. Arousal-mediated memory consolidation: role of the medial temporal lobe in humans. Psychol. Sci. 9, 490-493 (1998).
87. Phelps, E. A. et al. Specifying the contributions of the human amygdala to emotional memory: a case study. Neurocase 4, 527-540 (1998).
88. Adolphs, R., Cahill, L., Schul, R. & Babinsky, R. Impaired declarative memory for emotional material following bilateral amygdala damage in humans. Learn. Mem. 4, 291-300 (1997).
89. Bechara, A. et al. Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans. Science 269, 1115-1118 (1995).
90. Barraclough, N. E., Xiao, D., Baker, C. I., Oram, M. W. & Perrett, D. I. Integration of visual and auditory information by STS neurons responsive to the sight of actions. J. Cogn. Neurosci. 7, 377-391 (2005).
91. Dolan, R. J., Morris, J. S. & de Gelder, B. Crossmodal binding of fear in voice and face. Proc. Natl Acad. Sci. USA 98, 10006-10010 (2001).
92. Calvert, G. A., Campbell, R. & Brammer, M. J. Evidence from functional magnetic resonance imaging of crossmodal binding in human heteromodal cortex. Curr. Biol. 10, 649-657 (2000). Provides evidence from fMRI that the STS is involved in the integration of seen and heard speech.
93. Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H. & Martin, A. Unravelling multisensory integration: patchy organization within human STS multisensory cortex. Nature Neurosci. 7, 1190-1192 (2004).
94. Campbell, R. et al. Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). Cogn. Brain Res. 12, 233-243 (2001).
95. Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P. & Pike, B. Voice-selective areas in human auditory cortex. Nature 403, 309-312 (2000).
96. Desimone, R. & Gross, C. G. Visual areas in the temporal cortex of the macaque. Brain Res. 178, 363-380 (1979).
97. Oram, M. W. & Perrett, D. I. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J. Neurophysiol. 76, 109-129 (1996). Reports evidence from cell recording in monkeys that the STS is involved in the integration of visual form and motion components of biological motion.
98. McGurk, H. & MacDonald, J. Hearing lips and seeing voices. Nature 264, 746-748 (1976).
99. de Gelder, B. & Vroomen, J. The perception of emotions by ear and by eye. Cogn. Emotion 14, 289-311 (2000).
100. Rosenblum, L. D., Johnson, J. A. & Saldana, H. M. Point-light facial displays enhance comprehension of speech in noise. J. Speech Hear. Res. 39, 1159-1170 (1996).
101. Tartter, V. C. & Knowlton, K. C. Perception of sign language from an array of 27 moving spots. Nature 289, 676-678 (1981).
102. Puce, A., Allison, T., Bentin, S., Gore, J. C. & McCarthy, G. Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci. 18, 2188-2199 (1998).
103. Hoffman, E. A. & Haxby, J. V. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nature Neurosci. 3, 80-84 (2000).
104. Langton, S. R. H. The mutual influence of gaze and head orientation in the analysis of social attention direction. Q. J. Exp. Psychol. A 53, 825-845 (2000).
105. Langton, S. R. H., Honeyman, H. & Tessler, E. The influence of head contour and nose angle on the perception of eye-gaze direction. Percept. Psychophys. 66, 752-771 (2004).
106. Langton, S. R. H. & Bruce, V. You must see the point: automatic processing of cues to the direction of social attention. J. Exp. Psychol. Hum. Percept. Perform. 26, 747-757 (2000).
107. Adams, R. B. & Kleck, R. E. Perceived gaze direction and the processing of facial displays of emotion. Psychol. Sci. 14, 644-647 (2003). Presents evidence that gaze direction affects the perception of emotion from faces.
108. Hanley, J. R., Smith, S. T. & Hadfield, J. I recognise you but I can't place you: an investigation of familiar-only experiences during tests of voice and face recognition. Q. J. Exp. Psychol. A 51A, 179-195 (1998).
109. Bruce, V. & Valentine, T. in Practical Aspects of Memory: Current Research and Issues Vol. 1 (eds Gruneberg, M. M., Morris, P. E. & Sykes, R. N.) 169-174 (Wiley, Chichester, 1988).
110. Bonda, E., Petrides, M., Ostry, D. & Evans, A. Specific involvement of human parietal systems and the amygdala in the perception of biological motion. J. Neurosci. 16, 3737-3744 (1996).
111. Grossman, E. et al. Brain areas involved in perception of biological motion. J. Cogn. Neurosci. 12, 711-720 (2000).
112. Castelli, F., Happe, F., Frith, U. & Frith, C. Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns. Neuroimage 12, 314-325 (2000).
113. Allison, T., Puce, A. & McCarthy, G. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267-278 (2000).
114. Beauchamp, M. S., Lee, K. E., Haxby, J. V. & Martin, A. fMRI responses to video and point-light displays of moving and manipulable objects. J. Cogn. Neurosci. 15, 991-1001 (2003).
115. Calvert, G. A. et al. Activation of auditory cortex during silent lipreading. Science 276, 593-596 (1997).
116. Campbell, R., Landis, T. & Regard, M. Face recognition and lipreading: a neurological dissociation. Brain 109, 509-521 (1986).
117. Adams, R. B. & Kleck, R. E. The effects of direct and averted gaze on the perception of facial communicated emotion. Emotion 5, 3-11 (2005).
118. Adams, R. B., Gordon, H. L., Baird, A. A., Ambady, N. & Kleck, R. E. Gaze differentially modulates amygdala sensitivity to anger and fear faces. Science 300, 1536 (2003).
119. O'Toole, A. J., Roark, D. A. & Abdi, H. Recognizing moving faces: a psychological and neural synthesis. Trends Cogn. Sci. 6, 261-266 (2002).
120. Simons, D. J. & Levin, D. T. Failure to detect changes to people during a real-world interaction. Psychol. Bull. Rev. 5, 644-649 (1998).
121. Dimberg, U. Facial reactions for facial expressions. J. Psychophys. 19, 643-647 (1982).
122. Watkins, K. E., Strafella, A. P. & Paus, T. Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia 41, 989-994 (2003).
123. Langton, S. R. H. & Bruce, V. Reflexive visual orienting in response to the social attention of others. Vis. Cogn. 6, 541-567 (1999).
124. Driver, J. et al. Gaze perception triggers reflexive visuospatial orienting. Vis. Cogn. 6, 509-540 (1999).
125. Friesen, C. K. & Kingstone, A. The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychol. Bull. Rev. 5, 490-495 (1998).