Temporal characteristics of audiovisual information processing

Journal of Neuroscience

Volumn 28, Issue 20, 2008, Pages 5344-5349

  Alpert, Galit Fuhrmann ^a   Hein, Grit ^a   Tsai, Nancy ^a   Naumer, Marcus J ^b   Knight, Robert T ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b GOETHE UNIVERSITY   (Germany)

Author keywords

Auditory; fMRI; Information theory; Multisensory; Prefrontal cortex; Visual

Indexed keywords

ADULT; ARTICLE; BRAIN REGION; FEMALE; FRONTAL CORTEX; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEARING ACUITY; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; LATENT PERIOD; MALE; OCCIPITAL CORTEX; PRIORITY JOURNAL; PROVOCATION TEST; TEMPORAL CORTEX; TEMPORAL GYRUS; VISUAL ACUITY; VISUAL CORTEX; AUDITORY CORTEX; AUDITORY NERVOUS SYSTEM; AUDITORY STIMULATION; BRAIN; BRAIN MAPPING; FRONTAL LOBE; HEARING; HEMISPHERIC DOMINANCE; HISTOLOGY; IMAGE PROCESSING; NERVE CELL NETWORK; NUCLEAR MAGNETIC RESONANCE IMAGING; PHOTOSTIMULATION; PHYSIOLOGY; REACTION TIME; TIME; VISION; VISUAL SYSTEM;

ACOUSTIC STIMULATION; ADULT; AUDITORY CORTEX; AUDITORY PATHWAYS; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; FEMALE; FRONTAL LOBE; FUNCTIONAL LATERALITY; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; NERVE NET; PHOTIC STIMULATION; REACTION TIME; TIME FACTORS; VISUAL CORTEX; VISUAL PATHWAYS; VISUAL PERCEPTION;

EID: 45549090495     PISSN: 02706474     EISSN: 02706474     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.5039-07.2008     Document Type: Article

References (33)

1. Altmann CF, Bledowski C, Wibral M, Kaiser J (2007) Processing of location and pattern changes of natural sounds in the human auditory cortex. NeuroImage 35:1192-1200.
2. Beauchamp MS, Lee KE, Argall BD, Martin A (2004) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 4:809-823.
3. Belardinelli MO, Sestieri C, Di Matteo R, Delogu F, Del Gratta C, Ferretti A, Caulo M, Tartaro A, Romani GL (2004) Audio-visual crossmodal interactions in environmental perception: an fMRI investigation. Cogn Process 5:167-174.
4. Boynton GM, Engel SA, Glover GH, Heeger DJ (1996) Linear systems analysis of functional magnetic resonance imaging in human V1. J Neurosci 16:4207-4221.
5. Calvert GA, Campbell R (2003) Reading speech from still and moving faces: the neural substrates of visible speech. J Cogn Neurosci 15:57-70.
6. Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649-657.
7. Campbell R, Zihl J, Massaro DW, Munhall K, Cohen MM (1997) Speechreading in the akinetopsic patient, L.M. Brain 120:1793-1803.
8. Celesia GG (1976) Organization of auditory cortical areas in man. Brain 99:403-414.
9. Di Russo F, Martinez A, Hillyard SA (2003) Source analysis of event-related cortical activity during visuo-spatial attention. Cereb Cortex 13:486-499.
10. Fort A, Delpuech C, Pernier J, Girard MH (2002) Dynamics of corticsubcortical cross-modal operations involved in audio-visual object detection in humans. Cereb Cortex 12:1031-1039.
11. Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory convergence during early cortical processing. NeuroReport 16:419-423.
12. Fuhrmann Alpert G, Sun FT, Handwerker D, D'Esposito M, Knight RT (2007) Spatio-temporal information analysis of event-related BOLD response. NeuroImage 34:1545-1561.
13. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278-285.
14. Girard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473-490.
15. Hein G, Doehrmann O, Müller NG, Kaiser J, Muckli L, Naumer MJ (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881-7887.
16. Karns CM, Knight RT (2007) Electrophysiological evidence that intermodal auditory, visual, and tactile attention modulates early stages of neural processing. Soc Neurosci Abstr 33:423.24.
17. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 18:279-292.
18. Martuzzi R, Murray MM, Michel CM, Thiran JP, Maeder PP, Clarke S, Meuli RA (2007) Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cereb Cortex 17:1672-1679.
19. Meienbrock A, Naumer MJ, Doehrmann O, Singer W, Muckli L (2007) Retinotopic effects during spatial audio-visual integration. Neuropsychologia 45:531-539.
20. Molholm S, Ritter W, Javitt DC, Foxe JJ (2004) Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 14:452-465.
21. Munhall KG, Servos P, Santi A, Goodale MA (2002) Dynamic visual speech perception in a patient with visual form agnosia. NeuroReport 13:1793-1796.
22. Picton TW, Hillyard SA, Krausz HI, Galambos R (1974) Human auditory evoked potentials. I. Evaluation of components. Electroencephalogr Clin Neurophysiol 36:179-190.
23. Schroeder CE, Foxe J (2005) Multisensory contributions to low-level "unisensory" processing. Curr Opin Neurobiol 15:454-458.
24. Schroeder CE, Mehta AD, Givre SJ (1998) A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cereb Cortex 8:575-592.
25. Schroeder CE, Mehta AD, Foxe JJ (2001) Determinants and mechanisms of attentional modulation of neural processing. Front Biosci 6:672-684.
26. Senkowski D, Saint-Amour D, Kelly SP, Foxe JJ (2007) Multisensory processing of naturalistic objects in motion: a high-density electrical mapping and source estimation study. NeuroImage 36:877-888.
27. Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Exp Brain Res 179:85-95.
28. Talsma D, Doty TJ, Woldorff MG (2007) Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cereb Cortex 17:679-690.
29. Taylor KI, Moss HE, Stamatakis EA, Tyler LK (2006) Binding crossmodal object features in perirhinal cortex. Proc Natl Acad Sci USA 21:8239-8244.
30. Teder-Sälejärvi WA, Di Russo F, McDonald JJ, Hillyard SA (2005) Effects of spatial congruity on audio-visual multimodal integration. J Cogn Neurosci 17:1396-1409.
31. van Atteveldt NM, Formisano E, Goebel R, Blomert L (2004) Integration of letters and speech sounds in the human brain. Neuron 43:271-282.
32. van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962-974.
33. van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci USA 4:1181-1186.