Crossmodal interaction of facial and vocal person identity information: An event-related potential study

Brain Research

Volumn 1385, Issue , 2011, Pages 229-245

  Föcker, Julia ^a,b   Hölig, Cordula ^a   Best, Anna ^a   Röder, Brigitte ^a  

^a UNIVERSITY OF HAMBURG   (Germany)

^b UNIVERSITY OF ROCHESTER   (United States)

Author keywords

Crossmodal; Event related potentials; Faces; Multisensory; Priming; Voices

Indexed keywords

ADULT; ARTICLE; AUDITORY RESPONSE; AUDITORY STIMULATION; BEHAVIORAL SCIENCE; ELECTROENCEPHALOGRAPHY; EVENT RELATED POTENTIAL; FACIAL EXPRESSION; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; MEDICAL EDUCATION; PRIORITY JOURNAL; SOCIAL INTERACTION; VISION;

ACOUSTIC STIMULATION; ADULT; AGED; AUDITORY PERCEPTION; EVOKED POTENTIALS; EVOKED POTENTIALS, AUDITORY; FACE; FEMALE; HUMANS; MALE; MIDDLE AGED; PHOTIC STIMULATION; REACTION TIME; RECOGNITION (PSYCHOLOGY); VOICE; YOUNG ADULT;

EID: 79953196075     PISSN: 00068993     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainres.2011.02.021     Document Type: Article

References (84)

1. A. Amedi, K. von Kriegstein, N.M. van Atteveldt, M.S. Beauchamp, and M.J. Naumer Functional imaging of human crossmodal identification and object recognition Exp. Brain Res. 166 2005 559 571 (Pubitemid 41502099)
2. S.E. Barrett, and M.D. Rugg Event-related potentials and the semantic matching of faces Neuropsychologia 27 1989 913 922 (Pubitemid 19206922)
3. M.S. Beauchamp, B.D. Argall, J. Bodurka, J.H. Duyn, and A. Martin Unraveling multisensory integration: patchy organization within human STS multisensory cortex Nat. Neurosci. 11 2004 1190 1192 (Pubitemid 39426237)
4. M. Beauchemin, L. de Beaumont, P. Vannasing, A. Turcotte, C. Arcand, P. Belin, and M. Lassonde Electrophysiological markers of voice familiarity Eur. J. Neurosci. 23 2006 3081 3086 (Pubitemid 43882888)
5. P. Belin, S. Fecteau, and C. Bédard Thinking the voice: neural correlates of voice perception Trends Cogn. Sci. 8 2004 129 135 (Pubitemid 38299323)
6. P. Belin, and R.J. Zatorre Adaptation to speaker's voice in right anterior temporal lobe NeuroReport 14 2003 2105 2109
7. P. Belin, R.J. Zatorre, and P. Ahad Human temporal-lobe response to vocal sounds Brain Res. Cogn. Brain Res. 13 2002 17 26 (Pubitemid 34183360)
8. P. Belin, R.J. Zatorre, P. Lafaille, P. Ahad, and B. Pike Voice-selective areas in human auditory cortex Nature 403 2000 309 312 (Pubitemid 30062386)
9. L.A. Benevento, J. Fallon, B.J. Davis, and M. Rezak Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey Exp. Neurol. 57 1977 849 872 (Pubitemid 8242630)
10. J. Besle, O. Bertrand, and M.H. Giard Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex Hear. Res. 258 2009 143 151
11. J. Besle, C. Fischer, A. Bidet-Caulet, F. Lecaignard, O. Bertrand, and M.H. Giard Visual activation and audiovisual interactions in the auditory cortex during speech perception: intracranial recordings in humans J. Neurosci. 28 2008 14301 14320
12. V. Bruce, and A. Young Understanding face recognition Br. J. Psychol. 77 1986 305 327
13. T.W. Budd, R.J. Barry, E. Gordon, C. Rennie, and P.T. Michie Decrement of the N1 auditory event-related potential with stimulus repetition: habituation vs. refractoriness Int. J. Psychophysiol. 31 1998 51 68 (Pubitemid 29045662)
14. G.A. Calvert, E.T. Bullmore, M.J. Brammer, R. Campbell, S.C.R. Williams, P.K. McGuire, P.W.R. Woodruff, S.D. Iversen, and A.S. David Activation of auditory cortex during silent lipreading Science 276 1997 593 596 (Pubitemid 27193683)
15. G.A. Calvert, P.C. Hansen, S.D. Iversen, and M.J. Brammer Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect Neuroimage 14 2001 427 438 (Pubitemid 32709557)
16. S. Campanella, and P. Belin Integrating face and voice in person perception Trends Cogn. Sci. 11 2007 535 543 (Pubitemid 350181565)
17. S. Campanella, C. Hanoteau, D. Dépy, B. Rossion, R. Bruyer, M. Crommelinck, and J.M. Guérit Right N170 modulation in a face discrimination task: an account for categorical perception of familiar faces Psychophysiology 37 2000 796 806
18. R. Campbell, C.A. Heywood, A. Cowey, M. Regard, and T. Landis Sensitivity to eye gaze in prosopagnosic patients and monkeys with superior temporal sulcus ablation Neuropsychologia 28 1990 1123 1142
19. I. Charest, C.R. Pernet, G.A. Rousselet, I. Quinones, M. Latinus, S. Fillion-Bilodeau, J.P. Chartrand, and P. Belin Electrophysiological evidence for an early processing of human voices BMC Neurosci. 20 2009 1 11
20. B. de Gelder, and J.J. Stekelenburg Naso-temporal asymmetry of the N170 for processing faces in normal viewers but not in developmental prosopagnosia Neurosci. Let. 376 2005 40 45 (Pubitemid 40202934)
21. M. Eimer The face-specific N170 component reflects late stages in the structural encoding of faces NeuroReport 10 2000 2319 2324 (Pubitemid 30485584)
22. H.D. Ellis, D.M. Jones, and N. Mosdell Intra- and inter-modal repetition priming of familiar faces and voices Br. J. Psychol. 88 1997 143 156 (Pubitemid 127454312)
23. F.M. Engst, M. Martín-Loeches, and W. Sommer Memory systems for structural and semantic knowledge of faces and buildings Brain Res. 1124 2006 70 80 (Pubitemid 44779549)
24. F. Fang, D. Kersten, and S.O. Murray Perceptual grouping and inverse fMRI activity patterns in human visual cortex J. Vis. 8 2008 1 9
25. G. Ganis, M. Kutas, and M.I. Sereno The search for "common sense": an electrophysiological study of the comprehension of words and pictures in reading J. Cogn. Neurosci. 8 1996 89 106
26. A.A. Ghazanfar, C. Chandrasekaran, and N.K. Logothetis Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys J. Cogn. Neurosci. 28 2008 4457 4469
27. A.A. Ghazanfar, J.X. Maier, K.L. Hoffman, and N.K. Logothetis Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex J. Neurosci. 25 2005 5004 5012 (Pubitemid 40712843)
28. M. Gondan, B. Niederhaus, F. Rösler, and B. Röder Multisensory processing in the redundant-target effect: a behavioral and event-related potential study Percept. Psychophys. 67 2005 713 726 (Pubitemid 41328715)
29. J. Grigor, S. Van Toller, J. Behan, and A. Richardson The effect of odour priming on long latency visual evoked potentials of matching and mismatching objects Chem. Senses 24 1999 137 144 (Pubitemid 29176548)
30. K. Grill-Spector, R. Henson, and A. Martin Repetition and the brain: neural models of stimulus-specific effects Trends Cogn. Sci. 10 2006 14 23 (Pubitemid 43049768)
31. J.V. Haxby, E.A. Hoffman, and M.I. Gobbini The distributed human neural system for face perception Trends Cogn. Sci. 4 2000 223 233 (Pubitemid 30332148)
32. G. Hein, O. Doehrmann, N.G. Müller, J. Kaiser, L. Muckli, and M.J. Naumer Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas J. Neurosci. 27 2007 7881 7887 (Pubitemid 47173312)
33. C. Heywood, and A. Cowey The role of the 'face-cell' area in the discrimination and recognition of faces by monkeys Philos. Trans. R. Soc. Lond. B Biol. Sci. 335 1992 31 38
34. E.A. Hoffman, and J.V. Haxby Distinct representations of eye gaze and identity in the distributed human neural system for face perception Nat. Neurosci. 3 2000 80 84 (Pubitemid 30126352)
35. P.J. Holcomb, J. Anderson, and J. Grainger An electrophysiological study of cross-modal repetition priming Psychophysiology 42 2005 493 507 (Pubitemid 43942830)
36. A. Holmes, P. Vuilleumier, and M. Eimer The processing of emotional facial expression is gated by spatial attention: evidence from event-related potentials Cogn. Brain. Res. 16 2003 174 184 (Pubitemid 36372058)
37. H. Huynh, and L.S. Feldt Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs J. Educ. Stat. 1 1976 69 82
38. Joassin, F.; Maurage, P.; Bruyer, R.; Crommelinck, M.; Campanella, S. When audition alters vision: an event related-potential study of the cross-modal interactions between faces and voices Neurosci. Lett. 369 2004 132 137 (Pubitemid 39265414)
39. F. Joassin, P. Maurage, and S. Campanella The neural network sustaining the crossmodal processing of human gender from faces and voices: an fMRI study NeuroImage 54 2011 1654 1661
40. C. Kayser, and N.K. Logothetis Do early sensory cortices integrate cross-modal information? Brain Struct. Funct. 212 2007 121 132 (Pubitemid 47394498)
41. M. Kutas, and K.D. Federmeier Electrophysiology reveals semantic memory use in language comprehension Trends Cogn. Sci. 14 2000 463 470
42. M. Kutas, and S.A. Hillyard Reading senseless sentences: brain potentials reflect semantic incongruity Science 11 1980 203 205 (Pubitemid 10141432)
43. S. Lehmann, and M.M. Murray The role of multisensory memories in unisensory object discrimination Brain Res. Cogn. Brain Res. 24 2005 326 334 (Pubitemid 40922640)
44. D.A. Levy, R. Granot, and S. Bentin Processing specificity for human voice stimuli: electrophysiological evidence NeuroReport 12 2001 2653 2657 (Pubitemid 32803194)
45. S.J. Luck An Introduction to the Event-Related Potential Technique 2005 The MIT Press Cambridge
46. D. Maurer, K.M. O'Craven, R. Le Grand, C.J. Mondloch, M.V. Springer, T.L. Lewis, and C.L. Grady Neural correlates of processing facial identity based on features versus their spacing Neuropsychologia 45 2007 1438 1451 (Pubitemid 46097007)
47. J.J. McDonald, W.A. Teder-Sälejärvi, and L.M. Ward Multisensory integration and crossmodal attention effects in the human brain Science 292 2001 1791
48. W.B. McPherson, and P.J. Holcomb An electrophysiological investigation of semantic priming with pictures of real objects Psychophysiology 36 1999 53 65 (Pubitemid 29044456)
49. S. Molholm, W. Ritter, D.C. Javitt, and J.J. Foxe multisensory visual-auditory object recognition in humans: a high-density electrical mapping study Cereb. Cortex 14 2004 452 465 (Pubitemid 38506976)
50. T.F. Münte, M. Brack, O. Grootheer, B.M. Wieringa, M. Matzke, and S. Johannes Brain potentials reveal the timing of face identity and expression judgments Neurosci. Res. 30 1998 25 34 (Pubitemid 28119146)
51. M.M. Murray, C.M. Michel, R. Grave de Peralta, S. Ortigue, D. Brunet, S. Gonzalez Andino, and A. Schnider Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging Neuroimage 21 2004 125 135 (Pubitemid 38115003)
52. R. Näätänen Attention and Brain Function 1992 Lawrence Erbaum Associates Hillsdale, New Jersey
53. F. Neuner, and S.R. Schweinberger Neuropsychological impairments in the recognition of faces, voices, and personal names Brain Cogn. 44 2000 342 366 (Pubitemid 32001408)
54. U. Noppeney, O. Josephs, J. Hocking, C.J. Price, and K.J. Friston The effect of prior visual information on recognition of speech and sounds Cereb. Cortex 18 2008 598 609 (Pubitemid 351264229)
55. R. Oostenveld, and P. Praamstra The five percent electrode system for high-resolution EEG and ERP measurements Clin. Neurophysiol. 112 2001 713 719 (Pubitemid 32245646)
56. J. Pekkola, V. Ojanen, T. Autti, I.P. Jääskeläinen, R. Möttönen, A. Tarkiainen, and M. Sams Primary auditory cortex activation by visual speech: an fMRI study at 3T NeuroReport 16 2005 125 128 (Pubitemid 40250383)
57. A. Puce, T. Allison, S. Bentin, J.C. Gore, and G. McCarthy Temporal cortex activation in humans viewing eye and mouth movements J. Neurosci. 18 1998 2188 2199 (Pubitemid 28141423)
58. R.P.N. Rao, and D.H. Ballard Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects Nat. Neurosci. 2 1999 79 87 (Pubitemid 30488944)
59. K. Relander, and P. Rämä Separate neural processes for retrieval of voice identity and word content in working memory Brain Res. 1252 2009 143 151
60. B. Rossion, I. Gauthier, V. Goffaux, M.J. Tarr, and M. Crommelinck Expertise training with novel objects leads to face-like electrophysiological responses Psychol. Sci. 13 2002 250 257 (Pubitemid 41692297)
61. M. Sams, R. Aulanko, M. Hämäläinen, R. Hari, O.V. Lounasmaa, S.T. Lu, and J. Simola Seeing speech: visual information from lip movements modifies activity in the human auditory cortex Neurosci. Lett. 127 1991 141 145
62. D.L. Schacter, and R.L. Buckner Priming and the brain Neuron 20 1998 185 195 (Pubitemid 28125064)
63. T.R. Schneider, S. Debener, R. Oostenveld, and A.K. Engel Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming Neuroimage 42 2008 1244 1254
64. S.R. Schweinberger Human brain potential correlates of voice priming and voice recognition Neuropsychologia 39 2001 921 936
65. S.R. Schweinberger, E.M. Pfütze, and W. Sommer Repetition and associative priming of face recognition: evidence from event-related potentials J. Exp. Psychol. Learn. Mem. Cogn. 21 1995 722 736
66. S.R. Schweinberger, E.C. Pickering, A.M. Burton, and J.M. Kaufmann Human brain potential correlates of repetition priming in face and name recognition Neuropsychologia 40 2002 2057 2073
67. N.J. Shah, J.C. Marshall, O. Zafiris, A. Schwab, K. Zilles, H.J. Markowitsch, and G.R. Fink The neural correlates of person familiarity: a functional magnetic resonance imaging study with clinical implications Brain 124 2001 804 815 (Pubitemid 32303838)
68. K.N. Spreckelmeyer, M. Kutas, T. Urbach, E. Altenmüller, and T.F. Münte Neural processing of vocal emotion and identity Brain Cogn. 69 2009 121 126
69. J.J. Stekelenburg, and J. Vroomen Neural correlates of multisensory integration of ecologically valid audiovisual events J. Cogn. Neurosci. 19 2007 1964 1973 (Pubitemid 350220508)
70. B.E. Stein, and T.R. Stanford Multisensory integration: current issues from the perspective of the single neuron Nat. Rev. Neurosci. 9 2008 255 267
71. M. Toivonen, and P. Rämä N400 during recognition of voice identity and vocal affect NeuroReport 20 2009 1245 1249
72. D. Tranel, A.R. Damasio, and H. Damasio Intact recognition of facial expression, gender, and age in patients with impaired recognition of face identity Neurology 38 1988 690 696
73. T. Tsukiura, and R. Cabeza Orbitofrontal and hippocampal contributions to memory for face-name associations: the rewarding power of a smile Neuropsychologia 46 2008 2310 2319
74. T. Tsukiura, Y. Mano, A. Sekiguchi, Y. Yomogida, K. Hoshi, T. Kambara, H. Takeuchi, M. Sugiura, and R. Kawashima Dissociable roles of the anterior temporal regions in successful encoding of memory for person identity information J. Cogn. Neurosci. 22 2010 2226 22237
75. T. Tsukiura, H. Mochizuki-Kawai, and T. Fujii Dissociable roles of the bilateral anterior temporal lobe in face-name associations: an event-related fMRI study Neuroimage 30 2006 617 620
76. C. Van Petten, and H. Rheinfelder Conceptual relationships between spoken words and environmental sounds: event-related brain potential measures Neuropsychologia 33 1995 485 508
77. V. van Wassenhove, K.W. Grant, and D. Poeppel Visual speech speeds up the neural processing of auditory speech Proc. Natl Acad. Sci. USA 102 2005 1181 1186 (Pubitemid 40170746)
78. K. von Kriegstein, E. Eger, A. Kleinschmidt, and A.L. Giraud Modulation of neural responses to speech by directing attention to voices or verbal content Brain Res. Cogn. Brain Res. 17 2003 48 55 (Pubitemid 36577541)
79. K. von Kriegstein, and A.L. Giraud Distinct functional substrates along the right superior temporal sulcus for the processing of voices Neuroimage 22 2004 948 955
80. K. von Kriegstein, and A.L. Giraud Implicit multisensory associations influence voice recognition PLoS Biol. 4 2006 1809 1820 (Pubitemid 44610099)
81. K. von Kriegstein, A. Kleinschmidt, P. Sterzer, and A.L. Giraud Interaction of face and voice areas during speaker recognition J. Cogn. Neurosci. 17 2005 367 376 (Pubitemid 40364526)
82. K. von Kriegstein, D.R. Smith, R.D. Patterson, S.J. Kiebel, and T.D. Griffiths How the human brain recognizes speech in the context of changing speakers J. Neurosci. 30 2010 629 638
83. A.W. Young, D.J. Hellawell, C. van de Wal, and M. Johnson Facial expression processing after amygdalotomy Neuropsychologia 34 1996 31 39 (Pubitemid 26059596)
84. R. Zäske, S.R. Schweinberger, J.M. Kaufmann, and H. Kawahara In the ear of the beholder: neural correlates of adaptation to voice gender Eur. J. Neurosci. 30 2009 527 534