Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas

Journal of Neuroscience

Volumn 30, Issue 37, 2010, Pages 12329-12339

  Lewis, Richard ^a   Noppeney, Uta ^a  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUDITORY CORTEX; AUDITORY STIMULATION; BLOOD OXYGENATION; CAUSAL MODELING; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; MALE; NORMAL HUMAN; PRIORITY JOURNAL; TEMPORAL CORTEX; VISION; VISUAL CORTEX; VISUAL STIMULATION; ADOLESCENT; ADULT; ANATOMY AND HISTOLOGY; BRAIN MAPPING; HEARING; MOVEMENT PERCEPTION; NERVE TRACT; NEUROPSYCHOLOGICAL TEST; PERCEPTIVE DISCRIMINATION; PHOTOSTIMULATION; PHYSIOLOGY; YOUNG ADULT;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDITORY CORTEX; AUDITORY PERCEPTION; BRAIN MAPPING; DISCRIMINATION (PSYCHOLOGY); FEMALE; HUMANS; MALE; MOTION PERCEPTION; NEURAL PATHWAYS; NEUROPSYCHOLOGICAL TESTS; PHOTIC STIMULATION; VISUAL CORTEX; YOUNG ADULT;

EID: 77956867647     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.5745-09.2010     Document Type: Article

References (64)

1. Alink A, Singer W, Muckli L (2008) Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex. J Neurosci 28:2690-2697. (Pubitemid 351398870)
2. Amedi A, Jacobson G, Hendler T, Malach R, Zohary E (2002) Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb Cortex 12:1202-1212. (Pubitemid 35191421)
3. Bartels A, Logothetis NK, Moutoussis K (2008) fMRI and its interpretations: an illustration on directional selectivity in area V5/MT. Trends Neurosci 31:444-453.
4. Beauchamp MS, Lee KE, Argall BD, Martin A (2004a) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809-823.
5. Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004b) Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci 7:1190-1192. (Pubitemid 39426237)
6. Benevento LA, Fallon J, Davis BJ, Rezak M (1977) Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp Neurol 57:849-872. (Pubitemid 8242630)
7. Brosch M, Selezneva E, Scheich H (2005) Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys. J Neurosci 25:6797-6806. (Pubitemid 41023147)
8. Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46:369-384. (Pubitemid 11087648)
9. Bushara KO, Grafman J, Hallett M (2001) Neural correlates of auditoryvisual stimulus onset asynchrony detection. J Neurosci 21:300-304. (Pubitemid 32098753)
10. Calvert GA, Brammer MJ, Bullmore ET, Campbell R, Iversen SD, David AS (1999) Response amplification in sensory-specific cortices during cross-modal binding. Neuroreport 10:2619-2623. (Pubitemid 29418147)
11. Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649-657. (Pubitemid 30354700)
12. Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14:427-438.
13. Corbetta M, Miezin FM, Dobmeyer S, Shulman GL, Petersen SE (1991) Selective and divided attention during visual discriminations of shape, colour and speed: functional anatomy by positron emission tomography. J Neurosci 11:2383-2402.
14. Dhamala M, Assisi CG, Jirsa VK, Steinberg FL, Kelso JA (2007) Multisensory integration for timing engages different brain networks. Neuroimage 34:764-773. (Pubitemid 44827750)
15. Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 57:11-23.
16. Eickhoff SB, Stephan KE, Mohlberg H, Grefkes C, Fink GR, Amunts K, Zilles K (2005) A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 25:1325-1335. (Pubitemid 40591672)
17. Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16:419-423.
18. Frassinetti F, Bolognini N, Làdavas E (2002) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332-343. (Pubitemid 35341007)
19. Friston KJ, Penny WD, Phillips C, Kiebel SJ, Hinton G, Ashburner J (2002) Classical and Bayesian inference in neuroimaging: theory. NeuroImage 16:465-483. (Pubitemid 34852462)
20. Friston K (2009) Dynamic causal modeling and Granger causality. Comments on: The identification of interacting networks in the brain using fMRI: model selection, causality and deconvolution. Neuroimage. Advance online publication. Retrieved Aug. 2, 2010. doi:10.1016/j.neuroimage.2009.09.031.
21. Friston KJ, Holmes AP, Worsley KJ (1999) How many subjects constitute a study? Neuroimage 10:1-5.
22. Friston KJ, Harrison L, Penny W (2003) Dynamic causal modelling. Neuroimage 19:1273-1302.
23. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278-285.
24. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25:5004-5012.
25. Griffiths TD, Warren JD (2002) The planum temporale as a computational hub. Trends Neurosci 25:348-353.
26. Heron J, Whitaker D, McGraw PV (2004) Sensory uncertainty governs the extent of audio-visual interaction. Vision Res 44:2875-2884. (Pubitemid 39223308)
27. Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60:1615-1637.
28. Kayser C, Logothetis NK (2007) Do early sensory cortices integrate cross-modal information? Brain Struct Funct 212:121-132.
29. Kayser C, Petkov CI, Augath M, Logothetis NK (2007) Functional imaging reveals visual modulation of specific fields in auditory cortex. J Neurosci 27:1824-1835. (Pubitemid 46310306)
30. Kiebel SJ, Klöppel S, Weiskopf N, Friston KJ (2007) Dynamic causal modeling: a generative model of slice timing in fMRI. Neuroimage 34: 1487-1496. (Pubitemid 46187072)
31. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RB (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci U S A 92:8135-8139.
32. Malikovic A, Amunts K, Schleicher A, Mohlberg H, Eickhoff SB, Wilms M, Palomero-Gallagher N, Armstrong E, Zilles K (2007) Cytoarchitectonic analysis of the human extrastriate cortex in the region of V5/MT+: a probabilistic, stereotaxic map of area hOc5. Cereb Cortex 17:562-574. (Pubitemid 46254695)
33. Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389-391.
34. Meredith MA, Stein BE (1986) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophsyiol 56:640-662.
35. Munhall KG, Gribble P, Sacco L, Ward M (1996) Temporal constraints on the McGurk effect. Percept Psychophys 58:351-362.
36. Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J (2007) Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J Neurosci 27:11431-11441.
37. Noesselt T, Bergmann D, Hake M, Heinze HJ, Fendrich R (2008) Sound increases the saliency of visual events. Brain Res 1220:157-163.
38. Noguchi Y, Kaneoke Y, Kakigi R, Tanabe HC, Sadato N (2005) Role of the superior temporal region in human visual motion perception. Cereb Cortex 15:1592-1601.
39. Olivers CN, Van der Burg E (2008) Bleeping you out of the blink: sound saves vision from oblivion. Brain Res 1242:191-199.
40. Pandey PC, Kunov H, Abel SM (1986) Disruptive effects of auditory signal delay on speech perception with lipreading. J Aud Res 26:27-41.
41. Pavani F, Macaluso E, Warren JD, Driver J, Griffiths TD (2002) A common cortical substrate activated by horizontal and vertical sound movement in the human brain. Curr Biol 12:1584-1590.
42. Penny WD, Stephan KE, Mechelli A, Friston KJ (2004a) Comparing dynamic causal models. Neuroimage 22:1157-1172.
43. Penny WD, Stephan KE, Mechelli A, Friston KJ (2004b) Modelling functional integration: a comparison of structural equation and dynamic causal models. Neuroimage 23 [Suppl 1]:S264-S274.
44. Penny WD, Stephan KE, Daunizeau J, Rosa MJ, Friston K, Schofield TM, Leff AP (2010) Comparing families of dynamic causal models. PLoS Comput Biol 6:e1000709.
45. Peuskens H, Claeys KG, Todd JT, Norman JF, Van Hecke P, Orban GA (2004) Attention to 3-D shape, 3-D motion and texture in 3-D structure from motion displays. J Cogn Neurosci 16:665-682.
46. Poirier C, Collignon O, Devolder AG, Renier L, Vanlierde A, Tranduy D, Scheiber C (2005) Specific activation of the V5 brain area by auditory motion processing: an fMRI study. Brain Res Cogn Brain Res 25:650-658.
47. Roebroeck A, Formisano E, Goebel R (2009) The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution. Neuroimage. In press.
48. Ross LA, Saint-Amour D, Leavitt VM, Javitt DC, Foxe JJ (2007) Do you see what I am saying?: exploring visual enhancement of speech comprehension in noisy environments. Cereb Cor 17:1147-1153.
49. Sadaghiani S, Maier JX, Noppeney U (2009) Natural, metaphoric, and linguistic auditory direction signals have distinct influences on visual motion processing. J Neurosci 29:6490-6499.
50. Sarkheil P, Vuong QC, Bülthoff HH, Noppeney U (2008) The integration of higher order form and motion by the human brain. Neuroimage 42:1529-1536.
51. Scheef L, Boecker H, Daamen M, Fehse U, Landsberg MW, Granath DO, Mechling H, Effenberg AO (2009) Multimodal motion processing in area V5/MT: evidence from an artificial class of audio-visual events. Brain Res 1252:94-104.
52. Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, "unisensory" processing. Curr Opin Neurobiol 15:454-458.
53. Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14:187-198.
54. Sekuler R, Sekuler AB, Lau R (1997) Sound alters visual motion perception. Nature 385:308.
55. Shams L, Kamitani Y, Shimojo S (2002) Visual illusion induced by sound. Brain Res Cogn Brain Res 14:147-152.
56. Stein BE, Meredith MA (1993) The merging of the senses. Cambridge, MA: MIT.
57. Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210-1223.
58. Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Exp Brain Res 179:85-95.
59. Vinberg J, Grill-Spector K (2008) Representation of shapes, edges, and surfaces across multiple cues in the human visual cortex. J Neurophysiol 99:1380-1393.
60. Vroomen J, de Gelder B (2000) Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform 26:1583-1590.
61. Warren JD, Zielinski BA, Green GG, Rauschecker JP, Griffiths TD (2002) Perception of sound-source motion by the human brain. Neuron 34:139-148.
62. Werner S, Noppeney U (2010a) Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb Cortex 20:1829-1842.
63. Werner S, Noppeney U (2010b) Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J Neurosci 30:2662-2675.
64. Zeki S, Watson JD, Lueck CJ, Friston KJ, Kennard C, Frackowiak RS (1991) A direct demonstration of functional specialization in the human visual cortex. J Neurosci 11:641-649.