Early auditory sensory processing of voices is facilitated by visual mechanisms

NeuroImage

Volumn 77, Issue , 2013, Pages 237-245

  Schall, Sonja ^a   Kiebel, Stefan J ^a,b   Maess, Burkhard ^a   Von Kriegstein, Katharina ^a,c  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^b University Clinic Jena   (Germany)

^c HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Audiovisual; FFA; Human; Multisensory; Speaker recognition

Indexed keywords

ACCURACY; ADULT; ARTICLE; AUDITORY DISCRIMINATION; AUDITORY STIMULATION; BRAIN FUNCTION; BRAIN REGION; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; FEMALE; FUSIFORM GYRUS; HUMAN; HUMAN EXPERIMENT; LATENT PERIOD; MAGNETOENCEPHALOGRAPHY; MALE; PRIORITY JOURNAL; SENSITIVITY ANALYSIS; SIGNAL DETECTION; SPEECH DISCRIMINATION; STIMULUS RESPONSE; VISUAL STIMULATION;

AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; FACE; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MAGNETOENCEPHALOGRAPHY; MALE; RECOGNITION (PSYCHOLOGY); VISUAL PERCEPTION; VOICE; YOUNG ADULT;

EID: 84876720401     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2013.03.043     Document Type: Article

References (56)

1. Altmann C.F., Nakata H., Noguchi Y., Inui K., Hoshiyama M., Kaneoke Y., Kakigi R. Temporal dynamics of adaptation to natural sounds in the human auditory cortex. Cereb. Cortex 2008, 18:1350-1360.
2. Barbeau E.J., Taylor M.J., Regis J., Marquis P., Chauvel P., Liegeois-Chauvel C. Spatio temporal dynamics of face recognition. Cereb. Cortex 2008, 18:997-1009.
3. Besle J., Fischer C., Bidet-Caulet A., Lecaignard F., Bertrand O., Giard M.H. Visual activation and audiovisual interactions in the auditory cortex during speech perception: intracranial recordings in humans. J. Neurosci. 2008, 28:14301-14310.
4. Bizley J.K., Nodal F.R., Bajo V.M., Nelken I., King A.J. Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cereb. Cortex 2007, 17:2172-2189.
5. Blank H., Anwander A., von Kriegstein K. Direct structural connections between voice- and face-recognition areas. J. Neurosci. 2011, 31:12906-12915.
6. Bricker P.D., Pruzansky S. Effects of stimulus content and duration on talker identification. J. Acoust. Soc. Am. 1966, 40:1441-1449.
7. Calvert G.A., Bullmore E.T., Brammer M.J., Campbell R., Williams S.C., McGuire P.K., Woodruff P.W., Iversen S.D., David A.S. Activation of auditory cortex during silent lipreading. Science 1997, 276:593-596.
8. Capilla A., Belin P., Gross J. The early spatio-temporal correlates and task independence of cerebral voice processing studied with MEG. Cereb. Cortex 2012, 10.1093/cercor/bhs119.
9. Chandrasekaran C., Trubanova A., Stillittano S., Caplier A., Ghazanfar A.A. The natural statistics of audiovisual speech. PLoS Comput. Biol. 2009, 5:e1000436.
10. Charest I., Pernet C.R., Rousselet G.A., Quinones I., Latinus M., Fillion-Bilodeau S., Chartrand J.P., Belin P. Electrophysiological evidence for an early processing of human voices. BMC Neurosci. 2009, 10:127.
11. De Lucia M., Clarke S., Murray M.M. A temporal hierarchy for conspecific vocalization discrimination in humans. J. Neurosci. 2010, 30:11210-11221.
12. Deffke I., Sander T., Heidenreich J., Sommer W., Curio G., Trahms L., Lueschow A. MEG/EEG sources of the 170-ms response to faces are co-localized in the fusiform gyrus. Neuroimage 2007, 35:1495-1501.
13. Fischl B., Sereno M.I., Dale A.M. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. Neuroimage 1999, 9:195-207.
14. Fischl B., Sereno M.I., Tootell R.B., Dale A.M. High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 1999, 8:272-284.
15. Ghazanfar A.A., Schroeder C.E. Is neocortex essentially multisensory?. Trends Cogn. Sci. 2006, 10:278-285.
16. Ghazanfar A.A., Maier J.X., Hoffman K.L., Logothetis N.K. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J. Neurosci. 2005, 25:5004-5012.
17. Gross J., Baillet S., Barnes G.R., Henson R.N., Hillebrand A., Jensen O., Jerbi K., Litvak V., Maess B., Oostenveld R., Parkkonen L., Taylor J.R., van Wassenhove V., Wibral M., Schoffelen J.M. Good practice for conducting and reporting MEG research. Neuroimage 2012, 65C:349-363.
18. Haxby J.V., Hoffman E.A., Gobbini M.I. The distributed human neural system for face perception. Trends Cogn. Sci. 2000, 4:223-233.
19. Henson R.N., Mouchlianitis E., Friston K.J. MEG and EEG data fusion: simultaneous localisation of face-evoked responses. Neuroimage 2009, 47:581-589.
20. Hoshino O. Neuronal responses below firing threshold for subthreshold cross-modal enhancement. Neural Comput. 2011, 23:958-983.
21. Ishai A., Haxby J.V., Ungerleider L.G. Visual imagery of famous faces: effects of memory and attention revealed by fMRI. Neuroimage 2002, 17:1729-1741.
22. Joassin F., Maurage P., Bruyer R., Crommelinck M., Campanella S. When audition alters vision: an event-related potential study of the cross-modal interactions between faces and voices. Neurosci. Lett. 2004, 369:132-137.
23. Kanwisher N., McDermott J., Chun M.M. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 1997, 17:4302-4311.
24. Kayser C., Logothetis N.K. Do early sensory cortices integrate cross-modal information?. Brain Struct. Funct. 2007, 212:121-132.
25. Kayser C., Logothetis N.K., Panzeri S. Visual enhancement of the information representation in auditory cortex. Curr. Biol. 2010, 20:19-24.
26. Keil J., Muller N., Ihssen N., Weisz N. On the variability of the McGurk effect: audiovisual integration depends on prestimulus brain states. Cereb. Cortex 2012, 22:221-231.
27. Kiebel S.J., Poline J.B., Friston K.J., Holmes A.P., Worsley K.J. Robust smoothness estimation in statistical parametric maps using standardized residuals from the general linear model. Neuroimage 1999, 10:756-766.
28. Kilner J.M., Friston K.J. Topological inference for EEG and MEG. Ann. Appl. Stat. 2010, 4:1272-1290.
29. Lakatos P., Chen C.M., O'Connell M.N., Mills A., Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 2007, 53:279-292.
30. Lemus L., Hernandez A., Luna R., Zainos A., Romo R. Do sensory cortices process more than one sensory modality during perceptual judgments?. Neuron 2010, 67:335-348.
31. Meyer K., Kaplan J.T., Essex R., Webber C., Damasio H., Damasio A. Predicting visual stimuli on the basis of activity in auditory cortices. Nat. Neurosci. 2010, 13:667-668.
32. Nieuwenhuys R. The insular cortex: a review. Prog. Brain Res. 2012, 195:123-163.
33. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.J., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 2007, 27:11431-11441.
34. Oldfield R.C. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 1971, 9:97-113.
35. Oostenveld R., Fries P., Maris E., Schoffelen J.M. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 2011:156869.
36. Pascual-Marqui R.D. Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find. Exp. Clin. Pharmacol. 2002, 24(Suppl. D):5-12.
37. Pekkola J., Ojanen V., Autti T., Jaaskelainen I.P., Mottonen R., Tarkiainen A., Sams M. Primary auditory cortex activation by visual speech: an fMRI study at 3T. Neuroreport 2005, 16:125-128.
38. Poirier C., Collignon O., Devolder A.G., Renier L., Vanlierde A., Tranduy D., Scheiber C. Specific activation of the V5 brain area by auditory motion processing: an fMRI study. Brain Res. Cogn. Brain Res. 2005, 25:650-658.
39. Roberts T.P., Ferrari P., Stufflebeam S.M., Poeppel D. Latency of the auditory evoked neuromagnetic field components: stimulus dependence and insights toward perception. J. Clin. Neurophysiol. 2000, 17:114-129.
40. Rotshtein P., Henson R.N., Treves A., Driver J., Dolan R.J. Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain. Nat. Neurosci. 2005, 8:107-113.
41. Schweinberger S.R. Human brain potential correlates of voice priming and voice recognition. Neuropsychologia 2001, 39:921-936.
42. Sergent J., Ohta S., MacDonald B. Functional neuroanatomy of face and object processing. A positron emission tomography study. Brain 1992, 115(Pt 1):15-36.
43. Shams L., Seitz A.R. Benefits of multisensory learning. Trends Cogn. Sci. 2008, 12:411-417.
44. Sheffert S.M., Olson E. Audiovisual speech facilitates voice learning. Percept. Psychophys. 2004, 66:352-362.
45. Stekelenburg J.J., Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. J. Cogn. Neurosci. 2007, 19:1964-1973.
46. Taulu S., Kajola M., Simola J. Suppression of interference and artifacts by the Signal Space Separation Method. Brain Topogr. 2004, 16:269-275.
47. Thelen A., Cappe C., Murray M.M. Electrical neuroimaging of memory discrimination based on single-trial multisensory learning. Neuroimage 2012, 62:1478-1488.
48. Tiitinen H., Sivonen P., Alku P., Virtanen J., Naatanen R. Electromagnetic recordings reveal latency differences in speech and tone processing in humans. Brain Res. Cogn. Brain Res. 1999, 8:355-363.
49. Tottenham N., Tanaka J.W., Leon A.C., McCarry T., Nurse M., Hare T.A., Marcus D.J., Westerlund A., Casey B.J., Nelson C. The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Res. 2009, 168:242-249.
50. van Wassenhove V., Grant K.W., Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. U. S. A. 2005, 102:1181-1186.
51. von Kriegstein K., Giraud A.L. Implicit multisensory associations influence voice recognition. PLoS Biol. 2006, 4:e326.
52. von Kriegstein K., Kleinschmidt A., Sterzer P., Giraud A.L. Interaction of face and voice areas during speaker recognition. J. Cogn. Neurosci. 2005, 17:367-376.
53. von Kriegstein K., Dogan O., Gruter M., Giraud A.L., Kell C.A., Gruter T., Kleinschmidt A., Kiebel S.J. Simulation of talking faces in the human brain improves auditory speech recognition. Proc. Natl. Acad. Sci. U. S. A. 2008, 105:6747-6752.
54. Vroomen J., Stekelenburg J.J. Visual anticipatory information modulates multisensory interactions of artificial audiovisual stimuli. J. Cogn. Neurosci. 2010, 22:1583-1596.
55. Yildirim I., Jacobs R.A. A rational analysis of the acquisition of multisensory representations. Cogn. Sci. 2012, 36:305-332.
56. Zhou Y.D., Fuster J.M. Visuo-tactile cross-modal associations in cortical somatosensory cells. Proc. Natl. Acad. Sci. U. S. A. 2000, 97:9777-9782.