Hearing facial identities: Brain correlates of face-voice integration in person identification

Cortex

Volumn 47, Issue 9, 2011, Pages 1026-1037

  Schweinberger, Stefan R ^a   Kloth, Nadine ^a   Robertson, David M C ^a  

^a FRIEDRICH SCHILLER UNIVERSITY JENA   (Germany)

Author keywords

Audiovisual integration; ERPs; Face recognition; Identity; Voice recognition

Indexed keywords

ADULT; ARTICLE; BRAIN FUNCTION; ELECTRODE; EVENT RELATED POTENTIAL; FACE VOICE INTEGRATION; FACIES; FEMALE; HEARING; HUMAN; HUMAN EXPERIMENT; IDENTITY; MALE; NORMAL HUMAN; PERSONAL APPEARANCE; SPEECH; STIMULUS RESPONSE; VOICE;

ACOUSTIC STIMULATION; AUDITORY PERCEPTION; BRAIN; EVOKED POTENTIALS; FACE; FEMALE; HUMANS; MALE; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; REACTION TIME; RECOGNITION (PSYCHOLOGY); VOICE; YOUNG ADULT;

EID: 79960939488     PISSN: 00109452     EISSN: 19738102     Source Type: Journal    
DOI: 10.1016/j.cortex.2010.11.011     Document Type: Article

References (60)

1. Belin P., Zatorre R.J. Adaptation to speaker's voice in right anterior temporal lobe. NeuroReport 2003, 14(16):2105-2109.
2. Belin P., Zatorre R.J., Lafaille P., Ahad P., Pike B. Voice-selective areas in human auditory cortex. Nature 2000, 403(6767):309-312.
3. Bentin S., Allison T., Puce A., Perez E., Mccarthy G. Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience 1996, 8(6):551-565.
4. Bötzel K., Schulze S., Stodieck S.R.G. Scalp topography and analysis of intracranial sources of face-evoked potentials. Experimental Brain Research 1995, 104(1):135-143.
5. Calvert G.A., Brammer M.J., Iversen S.D. Crossmodal identification. Trends in Cognitive Sciences 1998, 2(7):247-253.
6. Campanella S., Belin P. Integrating face and voice in person perception. Trends in Cognitive Sciences 2007, 11(12):535-543.
7. Chandrasekaran C., Trubanova A., Stillittano S., Caplier A., Ghazanfar A.A. The natural statistics of audiovisual speech. PLos Computational Biology 2009, 5(7):1-18.
8. Charest I., Pernet C.R., Rousselet G.A., Quinones I., Latinus M., Fillion-Bilodeau S., et al. Electrophysiological evidence for an early processing of human voices. BMC Neuroscience 2009, 10:127. art.no.1471.
9. Chartrand J.P., Peretz I., Belin P. Auditory recognition expertise and domain specificity. Brain Research 2008, 1220(C):191-198.
10. Cooper T.J., Harvey M., Lavidor M., Schweinberger S.R. Hemispheric asymmetries in image-specific and abstractive priming of famous faces: Evidence from reaction times and event-related brain potentials. Neuropsychologia 2007, 45(13):2910-2921.
11. Dodson C.S., Johnson M.K., Schooler J.W. The verbal overshadowing effect: Why descriptions impair face recognition. Memory & Cognition 1997, 25(2):129-139.
12. Eimer M. The face-specific N170 component reflects late stages in the structural encoding of faces. NeuroReport 2000, 11(10):2319-2324.
13. Ellis H.D., Jones D.M., Mosdell N. Intra- and inter-modal repetition priming of familiar faces and voices. British Journal of Psychology 1997, 88:143-156.
14. Farah M. Patterns of co-occurrence among the associative agnosias: Implications for visual object representation. Cognitive Neuropsychology 1991, 8(1):1-19.
15. Garrido L., Eisner F., Mcgettigan C., Stewart L., Sauter D., Hanley J.R., et al. Developmental phonagnosia: A selective deficit of vocal identity recognition. Neuropsychologia 2009, 47(1):123-131.
16. Ghazanfar A.A., Schroeder C.E. Is neocortex essentially multisensory?. Trends in Cognitive Sciences 2006, 10(6):278-285.
17. Green K.P., Kuhl P.K., Meltzoff A.N., Stevens E.B. Integration speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect. Perception & Psychophysics 1991, 50(6):524-536.
18. Hagan C.C., Woods W., Johnson S., Calder A.J., Green G.G.R., Young A.W. MEG demonstrates a supra-additive response to facial and vocal emotion in the right superior temporal sulcus. Proceedings of the National Academy of Sciences of the United States of America 2009, 106(47):20010-20015.
19. Herrmann M.J., Ehlis A.C., Ellgring H., Fallgatter A.J. Early stages (P100) of face perception in humans as measured with event-related potentials (ERPs). Journal of Neural Transmission 2005, 112(8):1073-1081.
20. Hillger L.A., Koenig O. Separable mechanisms in face processing - Evidence from hemispheric-specialization. Journal of Cognitive Neuroscience 1991, 3(1):42-58.
21. Huynh H., Feldt L.S. Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics 1976, 1:69-82.
22. Joassin F., Maurage P., Bruyer R., Crommelinck M., Campanella S. When audition alters vision: An event-related potential study of the cross-modal interactions between faces and voices. Neuroscience Letters 2004, 369:132-137.
23. Joassin F., Pesenti M., Maurage P., Verreckt E., Bruyer R., Campanella S. Cross-modal interactions between human faces and voices involved in person recognition. Cortex 2011, 47(3):367-376.
24. Kamachi M., Hill H., Lander K., Vatikiotis-Bateson E. 'Putting the face to the voice': Matching identity across modality. Current Biology 2003, 13(19):1709-1714.
25. Kanwisher N. Domain specificity in face perception. Nature Neuroscience 2000, 3(8):759-763.
26. Kanwisher N., Yovel G. The fusiform face area: A cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society B-Biological Sciences 2006, 361(1476):2109-2128.
27. Latinus M., Vanrullen R., Taylor M.J. Top-down and bottom-up modulation in processing bimodal face/voice stimuli. BMC Neuroscience 2010, 11. art.no. 36.
28. Lehmann D. Principles of spatial analysis. Handbook of Electroencephalography and Clinical Neurophysiology. Vol. 1. Methods of Analysis of Brain Electrical and Magnetic Signals 1987, 309-354. Elsevier, Amsterdam. A.S. Gevins, A. Remond (Eds.).
29. Lewis J.W., Wightman F.L., Brefczynski J.A., Phinney R.E., Binder J.R., Deyoe E.A. Human brain regions involved in recognizing environmental sounds. Cerebral Cortex 2004, 14(9):1008-1021.
30. Liu J., Harris A., Kanwisher N. Stages of processing in face perception: An MEG study. Nature Neuroscience 2002, 5(9):910-916.
31. Mangun G.R. Neural mechanisms of visual selective attention. Psychophysiology 1995, 32(1):4-18.
32. McCandliss B.D., Cohen L., Dehaene S. The visual word form area: Expertise for reading in the fusiform gyrus. Trends in Cognitive Sciences 2003, 7(7):293-299.
33. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264(5588):746-748.
34. Mohamed T.N., Neumann M.F., Schweinberger S.R. Perceptual load manipulation reveals sensitivity of the face-selective N170 to attention. NeuroReport 2009, 20(8):782-787.
35. Neuner F., Schweinberger S.R. Neuropsychological impairments in the recognition of faces, voices, and personal names. Brain and Cognition 2000, 44(3):342-366.
36. Nygaard L.C., Sommers M.S., Pisoni D.B. Speech-perception as a talker-contingent process. Psychological Science 1994, 5:42-46.
37. Oldfield R.C. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 1971, 9:97-113.
38. Paulmann S., Kotz S.A. Early emotional prosody perception based on different speaker voices. NeuroReport 2008, 19(2):209-213.
39. Remez R.E., Fellowes J.M., Nagel D.S. On the perception of similarity among talkers. Journal of the Acoustical Society of America 2007, 122(6):3688-3696.
40. Robertson D.M.C., Schweinberger S.R. The role of audiovisual asynchrony in person recognition. Quarterly Journal of Experimental Psychology 2010, 63(1):23-30.
41. Saint-Amour D., De Sanctis P., Molholm S., Ritter W., Foxe J.J. Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion. Neuropsychologia 2007, 45(3):587-597.
42. Sams M., Aulanko R., Hämalainen M., Hari R., Lounasmaa O.V., Lu S.-T., et al. Seeing speech: Visual information from lip movements modifies activity in the human auditory cortex. Neuroscience Letters 1991, 127(1):141-145.
43. Schweinberger S.R. Human brain potential correlates of voice priming and voice recognition. Neuropsychologia 2001, 39:921-936.
44. Schweinberger S.R., Herholz A., Sommer W. Recognizing famous voices: Influence of stimulus duration and different types of retrieval cues. Journal of Speech, Language, and Hearing Research 1997, 40(2):453-463.
45. Schweinberger S.R., Herholz A., Stief V. Auditory long-term memory: Repetition priming of voice recognition. Quarterly Journal of Experimental Psychology 1997, 50A:498-517.
46. Schweinberger S.R., Huddy V., Burton A.M. N250r - A face-selective brain response to stimulus repetitions. NeuroReport 2004, 15(9):1501-1505.
47. Schweinberger S.R., Robertson D., Kaufmann J.M. Hearing facial identities. Quarterly Journal of Experimental Psychology 2007, 60(10):1446-1456.
48. Soto-Faraco S., Alsius A. Deconstructing the McGurk-MacDonald illusion. Journal of Experimental Psychology: Human Perception and Performance 2009, 35(2):580-587.
49. Spreckelmeyer KN, kutas M., kutas M., Urbach T.P., Altenmüller E., Münte T.F. Neural processing of vocal emotion and identity. Brain and Cognition 2008, 69:121-126.
50. Stein B.E., Meredith M.A., Wallace M.T. Development and neural basis of multisensory integration. The Development of Intersensory Perception: Comparative Perspectives 1994, 81-105. Lawrence Erlbaum Associates, Hillsdale. D.J. Lewkowicz, R. Lickliter (Eds.).
51. Stevens A.A., Schwartzreich E. Dissociating message and messenger: fMRI of word and voice recognition. Journal of Cognitive Neuroscience 2000, 12:S49.
52. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America 1954, 26:212-215.
53. Summerfield Q., Macleod A., Mcgrath M., Brooke M. Lips, teeth, and the benefits of lipreading. Handbook of Research on Face Processing 1989, 223-233. North-Holland, Amsterdam. A.W. Young, H.D. Ellis (Eds.).
54. van Wassenhove V., Grant K.W., Poeppel D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia 2007, 45(3):598-607.
55. van Wassenhove V., Grant K.W., Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences 2005, 102(4):1181-1186.
56. VanLancker D.R., Canter G.J. Impairment of voice and face recognition in patients with hemispheric damage. Brain & Cognition 1982, 1:185-195.
57. von Kriegstein K., Dogan O., Gruter M., Giraud A.L., Kell C.A., Gruter T., et al. Simulation of talking faces in the human brain improves auditory speech recognition. Proceedings of the National Academy of Sciences of the United States of America 2008, 105(18):6747-6752.
58. von Kriegstein K., Kleinschmidt A., Sterzer P., Giraud A.L. Interaction of face and voice areas during speaker recognition. Journal of Cognitive Neuroscience 2005, 17(3):367-376.
59. Welch R.B., Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 1980, 88(3):638-667.
60. Zäske R., Schweinberger S.R., Kaufmann J.M., Kawahara H. In the ear of the beholder: Neural correlates of adaptation to voice gender. European Journal of Neuroscience 2009, 30(3):527-534.