Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices

Journal of Neuroscience

Volumn 27, Issue 42, 2007, Pages 11431-11441

  Noesselt, Toemme ^a,b,c   Rieger, Jochem W ^b   Schoenfeld, Mircea Ariel ^b   Kanowski, Martin ^b   Hinrichs, Hermann ^b   Heinze, Hans Jochen ^b   Driver, Jon ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b OTTO VON GUERICKE UNIVERSITY   (Germany)

^c Center for Advanced Imaging   (Germany)

Author keywords

Audiovisual; Connectivity; fMRI; Human multisensory; Temporal integration

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; BLOOD OXYGENATION; BRAIN REGION; CONTROLLED STUDY; CORRESPONDENCE ANALYSIS; ELECTROENCEPHALOGRAM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; PRIORITY JOURNAL; SENSORY CORTEX; TEMPORAL CORTEX; VISUAL CORTEX;

ACOUSTIC STIMULATION; ADULT; AUDITORY PERCEPTION; FEMALE; HUMANS; MALE; PHOTIC STIMULATION; PSYCHOMOTOR PERFORMANCE; SOMATOSENSORY CORTEX; TEMPORAL LOBE; VISUAL PERCEPTION;

EID: 35448976795     PISSN: 02706474     EISSN: 02706474     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.2252-07.2007     Document Type: Article

References (70)

1. Avillac M, Deneve S, Olivier E, Pouget A, Duhamel JR (2005) Reference frames for representing visual and tactile locations in parietal cortex. Nat Neurosci 8:941-949.
2. Baier B, Kleinschmidt A, Müller NG (2006) Cross-modal processing in early visual and auditory cortices depends on expected statistical relationship of multisensory information. J Neurosci 26:12260-12265.
3. Barraclough NE, Xiao DK, Baker CI, Oram MW, Perrett DI (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J Cogn Neurosci 17:377-391.
4. Beauchamp MS (2005a) See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol 15:145-153.
5. Beauchamp MS (2005b) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3:93-113.
6. Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004a) Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci 7:1190-1192.
7. Beauchamp MS, Lee KE, Argall BD, Martin A (2004b) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809-823.
8. Benevento LA, Fallon J, Davis BJ, Rezak M (1977) Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp Neurol 57:849-872.
9. Bischoff M, Walter B, Blecker CR, Morgen K, Vaitl D, Sammer G (2007) Utilizing the ventriloquism-effect to investigate audio-visual binding. Neuropsychologia 45:578-586.
10. Brosch M, Scheich H (2005) Non-acoustic influence on neural activity in auditory cortex. In: Auditory cortex: a synthesis of human and animal research (König P, Heil P, Budinger E, Scheich H, eds), pp 127-143. Hillsdale, NJ: Erlbaum.
11. Brosch M, Selezneva E, Scheich H (2005) Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys. J Neurosci 25:6797-6806.
12. Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46:369-384.
13. Bushara KO, Grafman J, Hallett M (2001) Neural correlates of auditoryvisual stimulus onset asynchrony detection. J Neurosci 21:300-304.
14. Calvert GA, Bullmore ET, Brammer MJ, Campbell R, Williams SC, McGuire PK, Woodruff PW, Iversen SD, David AS (1997) Activation of auditory cortex during silent lipreading. Science 276:593-596.
15. Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. NeuroImage 14:427-438.
16. Calvert GA, Stein BE, Spence C (2004) The handbook of multisensory processing. Cambridge, MA: MIT.
17. Corbetta M, Shulman GL (2002) Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci 3:201-215.
18. Cusick CG (1997) The superior temporal polysensory region in monkeys. In: Cerebral cortex: extrastriate cortex in primates (Rockland KS, Kaas JH, Peters A, eds), pp 435-463. New York: Plenum.
19. Dhamala M, Assisi CG, Jirsa VK, Steinberg FL, Kelso JA (2007) Multisensory integration for timing engages different brain networks. NeuroImage 34:764-773.
20. Falchier A, Clavagnier S, Barone P, Kennedy H (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749-5759.
21. Fort A, Delpuech C, Pernier J, Giard MH (2002) Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Res Cogn Brain Res 14:20-30.
22. Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory convergence during early cortical processing. NeuroReport 16:419-423.
23. Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res Cogn Brain Res 10:77-83.
24. Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ (1997) Psychophysiological and modulatory interactions in neuroimaging. NeuroImage 6:218-229.
25. Fu KM, Shah AS, O'Connell MN, McGinnis T, Eckholdt H, Lakatos P, Smiley J, Schroeder CE (2004) Timing and laminar profile of eye-position effects on auditory responses in primate auditory cortex. J Neurophysiol 92:3522-3531.
26. Fujisaki W, Koene A, Arnold D, Johnston A, Nishida S (2006) Visual search for a target changing in synchrony with an auditory signal. Proc Biol Sci 273:865-874.
27. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278-285.
28. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25:5004-5012.
29. Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473-490.
30. Hinrichs H, Heinze HJ, Schoenfeld MA (2006) Causal visual interactions as revealed by an information theoretic measure and fMRI. NeuroImage 31:1051-1060.
31. Jones EG (1998) Viewpoint: the core and matrix of thalamic organization. Neuroscience 85:331.
32. Kaas JH, Collins CE (2004) The resurrection of multisensory cortex in primates. In: The handbook of multisensory processes (Calvert GA, Spence C, Stein BE, eds), pp 285-294. Cambridge, MA: Bradford.
33. Kanowski M, Rieger JW, Noesselt T, Tempelmann C, Hinrichs H (2007) Endoscopic eye tracking system for fMRI. J Neurosci Methods 160:10-15.
34. Kayser C, Petkov CI, Augath M, Logothetis NK (2007) Functional imaging reveals visual modulation of specific fields in auditory cortex. J Neurosci 27:1824-1835.
35. Lakatos P, Shah AS, Knuth KH, Ulbert I, Karmos G, Schroeder CE (2005) An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J Neurophysiol 94:1904-1911.
36. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279-292.
37. Lavie N (2005) Distracted and confused?: selective attention under load. Trends Cogn Sci 9:75-82.
38. Lipschutz B, Kolinsky R, Damhaut P, Wikler D, Goldman S (2002) Attention-dependent changes of activation and connectivity in dichotic listening. NeuroImage 17:643-656.
39. Macaluso E, Driver J (2005) Multisensory spatial interactions: a window onto functional integration in the human brain. Trends Neurosci 28:264-271.
40. Macaluso E, Frith CD, Driver J (2000) Modulation of human visual cortex by crossmodal spatial attention. Science 289:1206-1208.
41. Macaluso E, George N, Dolan R, Spence C, Driver J (2004) Spatial and temporal factors during processing of audiovisual speech: a PET study. NeuroImage 21:725-732.
42. McDonald JJ, Teder-Salejarvi WA, Hillyard SA (2000) Involuntary orienting to sound improves visual perception. Nature 407:906-908.
43. McDonald JJ, Teder-Salejarvi WA, Di Russo F, Hillyard SA (2003) Neural substrates of perceptual enhancement by cross-modal spatial attention. J Cogn Neurosci 15:10-19.
44. Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215-3229.
45. Miller LM, D'Esposito M (2005) Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J Neurosci 25:5884-5893.
46. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115-128.
47. Molholm S, Ritter W, Javitt DC, Foxe JJ (2004) Multisensory visualauditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 14:452-465.
48. Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963-974.
49. Noesselt T, Fendrich R, Bonath B, Tyll S, Heinze HJ (2005) Closer in time when farther in space - spatial factors in audiovisual temporal integration. Brain Res Cogn Brain Res 25:443-458.
50. Petrides M, Iversen SD (1978) The effect of selective anterior and posterior association cortex lesions in the monkey on performance of a visual-auditory compound discrimination test. Neuropsychologia 16:527-537.
51. Schneider KA, Richter MC, Kastner S (2004) Retinotopic organization and functional subdivisions of the human lateral geniculate nucleus: a high-resolution functional magnetic resonance imaging study. J Neurosci 24:8975-8985.
52. Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, "unisensory" processing. Curr Opin Neurobiol 15:454-458.
53. Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14:187-198.
54. Schroeder CE, Smiley J, Fu KG, McGinnis T, O'Connell MN, Hackett TA (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50:5-17.
55. Slutsky DA, Recanzone GH (2001) Temporal and spatial dependency of the ventriloquism effect. NeuroReport 12:7-10.
56. Spence C, Driver J (2004) Crossmodal space and crossmodal attention. New York: Oxford UP.
57. Stein BE (1978) Development and organization of multimodal representation in cat superior colliculus. Fed Proc 37:2240-2245.
58. Stein BE, Meredith MA (1990) Multisensory integration. Neural and behavioral solutions for dealing with stimuli from different sensory modalities. Ann NY Acad Sci 608:51-65, discussion 65-70.
59. Stein BE, Meredith ME (1993) The merging of the sense. Cambridge, MA: MIT.
60. Stein BE, Meredith MA, Wallace MT (1993) The visually responsive neuron and beyond: multisensory integration in cat and monkey. Prog Brain Res 95:79-90.
61. Stein BE, Stanford TR, Wallace MT, Vaughan JW, Jiang W (2004) Crossmodal spatial interactions in subcortical and cortical circuits. In: Crossmodal space and crossmodal attention (Spence C, Driver J, eds), pp 25-50. Oxford: Oxford UP.
62. Teder-Sälejärvi WA, McDonald JJ, Di Russo F, Hillyard SA (2002) An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Cogn Brain Res 14:106-114.
63. van Atteveldt NM, Formisano E, Blomert L, Goebel R (2006) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962-974.
64. Wallace MT, Stein BE (1994) Cross-modal synthesis in the midbrain depends on input from cortex. J Neurophysiol 71:429-432.
65. Wallace MT, Stein BE (1996) Sensory organization of the superior colliculus in cat and monkey. Prog Brain Res 112:301-311.
66. Wallace MT, Stein BE (1997) Development of multisensory neurons and multisensory integration in cat superior colliculus. J Neurosci 17:2429-2444.
67. Wallace MT, Meredith MA, Stein BE (1993) Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J Neurophysiol 69:1797-1809.
68. Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246-1266.
69. Watkins S, Shams L, Tanaka S, Haynes JD, Rees G (2006) Sound alters activity in human V1 in association with illusory visual perception. NeuroImage 31:1247-1256.
70. Watkins S, Dalton P, Lavie N, Rees G (2007) Brain mechanisms mediating auditory attentional capture in humans. Cereb Cortex 17:1694-1700.