Audiovisual synchrony enhances BOLD responses in a brain network including multisensory STS while also enhancing target-detection performance for both modalities

Human Brain Mapping

Volumn 33, Issue 5, 2012, Pages 1212-1224

  Marchant, Jennifer L ^a,b   Ruff, Christian C ^a,b,c   Driver, Jon ^a,b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Asynchrony; Auditory; FMRI; Multisensory; Predictability; Superior temporal sulcus; Visual

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BOLD SIGNAL; BRAIN FUNCTION; EYE TRACKING; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MALE; NERVE CELL NETWORK; NORMAL HUMAN; PRIORITY JOURNAL; SUPERIOR TEMPORAL SULCUS; TASK PERFORMANCE; VISUAL STIMULATION;

ACOUSTIC STIMULATION; AUDITORY PERCEPTION; BRAIN; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; NERVE NET; PHOTIC STIMULATION; PSYCHOMOTOR PERFORMANCE; REACTION TIME; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84859438925     PISSN: 10659471     EISSN: 10970193     Source Type: Journal    
DOI: 10.1002/hbm.21278     Document Type: Article

References (70)

1. Balteau E, Josephs O, Ruff C, Weiskopf N ( 2008): Quiet EPI Sequence With Sinusoidal Readout for fMRI Studies. Presented at the 25th European Society for Magnetic Resonance in Medicine and Biology Conference; 2008 Oct 2-4; Valencia, Spain.
2. Beauchamp MS ( 2005): See me, hear me, touch me: Multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol 15: 145-153.
3. Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A ( 2004a): Unraveling multisensory integration: Patchy organization within human STS multisensory cortex. Nat Neurosci 7: 1190-1192.
4. Beauchamp MS, Lee KE, Argall BD, Martin A ( 2004b): Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41: 809-823.
5. Bengtsson SL, Ullen F, Ehrsson HH, Hashimoto T, Kito T, Naito E, Forssberg H, Sadato N ( 2009): Listening to rhythms activates motor and premotor cortices. Cortex 45: 62-71.
6. Bischoff M, Walter B, Blecker CR, Morgen K, Vaitl D, Sammer G ( 2007): Utilizing the ventriloquism-effect to investigate audiovisual binding. Neuropsychologia 45: 578-586.
7. Bizley JK, Nodal FR, Bajo VM, Nelken I, King AJ ( 2007): Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cereb Cortex 17: 2172-2189.
8. Brett M, Anton J-L, Valabregue R, Poline J-B ( 2002): Region of Interest Analysis Using an SPM Toolbox. Presented at the 8th International Conference on Functional Mapping of the Human Brain; 2002 June 2-6; Sendai, Japan.
9. Brett M, Penny W, Kiebel, S ( 2003): An introduction to random field theory. In: Frackowiak RSJ, Friston KJ, Frith CD, Dolan RJ, Price CJ, Zeki S, Ashburner J, Penny W, editors. Human Brain Function, 2nd ed. Oxford, UK: Elsevier. p 877.
10. Bushara KO, Grafman J, Hallett M ( 2001): Neural correlates of auditory-visual stimulus onset asynchrony detection. J Neurosci 21: 300-304.
11. Calvert CA, Spence C, Stein BE, editors ( 2004): The Handbook of Multisensory Processing. Cambridge, MA: MIT Press.
12. Calvert GA, Campbell R, Brammer MJ ( 2000): Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10: 649-657.
13. Calvert GA, Hansen PC, Iversen SD, Brammer MJ ( 2001): Detection of audiovisual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14: 427-438.
14. Cappe C, Rouiller EM, Barone P ( 2009): Multisensory anatomical pathways. Hear Res 258: 28-36.
15. Dayan P, Hinton GE, Neal RM, Zemel RS ( 1995): The Helmholtz machine. Neural Comput 7: 889-904.
16. den Ouden HE, Friston KJ, Daw ND, McIntosh AR, Stephan KE ( 2009): A dual role for prediction error in associative learning. Cereb Cortex 19: 1175-1185.
17. Doehrmann O, Naumer MJ ( 2008): Semantics and the multisensory brain: How meaning modulates processes of audiovisual integration. Brain Res 1242: 136-150.
18. Driver J, Noesselt T ( 2008): Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 57: 11-23.
19. Falchier A, Clavagnier S, Barone P, Kennedy H ( 2002): Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22: 5749-5759.
20. Falchier A, Schroeder CE, Hackett TA, Lakatos P, Nascimento-Silva S, Ulbert I, Karmos G, Smiley JF ( 2010): Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cereb Cortex 20: 1529-1538.
21. Frassinetti F, Bolognini N, Ladavas E ( 2002): Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147: 332-343.
22. Friston KJ ( 2005): A theory of cortical response. Philos Trans R Soc Lond B Biol Sci 360: 815-836.
23. Friston KJ ( 2010): The free-energy principle: A unified brain theory? Nat Rev Neurosci 11: 127-138.
24. Friston KJ, Worsley KJ, Frackowiak RSJ, Mazziotta JC, Evans JC ( 1994): Assessing the significance of focal activations using their spatial extent. Hum Brain Mapp 1: 214-220.
25. Friston KJ, Holmes AP, Worsley KJ, Poline J-P, Frith CD, Frackowiak RSJ ( 1995): Statistical parametric maps in functional imaging: A general linear approach. Hum Brain Mapp 2: 189-210.
26. Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ ( 1997): Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6: 218-229.
27. Friston KJ, Holmes AP, Price CJ, Buchel C, Worsley KJ ( 1999): Multisubject fMRI studies and conjunction analyses. Neuroimage 10: 385-396.
28. Furl N, Kumar S, Alter K, Durrant S, Shawe-Taylor J, Griffiths TD ( 2011): Neural prediction of higher-order auditory sequence statistics. Neuroimage 54: 2267-2277.
29. Ghazanfar AA, Schroeder CE ( 2006): Is neocortex essentially multisensory? Trends Cogn Sci 10: 278-285.
30. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK ( 2005): Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25: 5004-5012.
31. Grahn JA, Brett M ( 2007): Rhythm and beat perception in motor areas of the brain. J Cogn Neurosci 19: 893-906.
32. Grahn JA, Rowe JB ( 2009): Feeling the beat: Premotor and striatal interactions in musicians and nonmusicians during beat perception. J Neurosci 29: 7540-7548.
33. Hein G, Doehrmann O, Muller NG, Kaiser J, Muckli L, Naumer MJ ( 2007): Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27: 7881-7887.
34. Helmholtz H ( 1860): In: Southall JPD, editor. Helmholtz's Treatise on Physiological Optics ( 1962), Vol. 3. New York: Dover (English translation).
35. Hertz U, Amedi A ( 2010): Disentangling unisensory and multisensory components in audiovisual integration using a novel multifrequency fMRI spectral analysis. Neuroimage 52: 617-632.
36. Kayser C, Petkov CI, Augath M, Logothetis NK ( 2007): Functional imaging reveals visual modulation of specific fields in auditory cortex. J Neurosci 27: 1824-1835.
37. Kayser C, Petkov CI, Logothetis NK ( 2008): Visual modulation of neurons in auditory cortex. Cereb Cortex 18: 1560-1574.
38. Kayser C, Petkov CI, Logothetis NK ( 2009): Multisensory interactions in primate auditory cortex: fMRI and electrophysiology. Hear Res 258: 80-88.
39. Kriegeskorte N, Simmons WK, Bellgowan PSF, Baker CI ( 2009): Circular analysis in systems neuroscience: The dangers of double dipping. Nat Neurosci 12: 535-540.
40. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE ( 2007): Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53: 279-292.
41. Lewis R, Noppeney U ( 2010). Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J Neurosci 30: 12329-12339.
42. Lovelace CT, Stein BE, Wallace MT ( 2003): An irrelevant light enhances auditory detection in humans: A psychophysical analysis of multisensory integration in stimulus detection. Brain Res Cogn Brain Res 17: 447-453.
43. Macaluso E, Driver J ( 2005): Multisensory spatial interactions: A window onto functional integration in the human brain. Trends Neurosci 28: 264-271.
44. Macaluso E, Frith CD, Driver J ( 2000): Modulation of human visual cortex by crossmodal spatial attention. Science 289: 1206-1208.
45. Macaluso E, George N, Dolan R, Spence C, Driver J ( 2004): Spatial and temporal factors during processing of audiovisual speech: A PET study. Neuroimage 21: 725-732.
46. McIntosh AR, Cabeza RE, Lobaugh NJ ( 1998): Analysis of neural interactions explains the activation of occipital cortex by an auditory stimulus. J Neurophysiol 80: 2790-2796.
47. Meienbrock A, Naumer MJ, Doehrmann O, Singer W, Muckli L ( 2007): Retinotopic effects during spatial audiovisual integration. Neuropsychologia 45: 531-539.
48. Meredith MA, Nemitz JW, Stein BE ( 1987): Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7: 3215-3229.
49. Mumford D ( 1992): On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol Cybern 66: 241-251.
50. Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J ( 2007): Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J Neurosci 27: 11431-11441.
51. Noesselt T, Tyll S, Boehler CN, Budingder E, Hienze H-J, Driver J ( 2010): Sound-induced enhancement of low-intensity vision: Multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. J Neurosci 30( 21): 13609-13623.
52. Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ ( 2008): The effect of prior visual information on recognition of speech and sounds. Cereb Cortex 18: 598-609.
53. Odgaard EC, Arieh Y, Marks LE ( 2003): Cross-modal enhancement of perceived brightness: Sensory interaction versus response bias. Percept Psychophys 65: 123-132.
54. Odgaard EC, Arieh Y, Marks LE ( 2004): Brighter noise: Sensory enhancement of perceived loudness by concurrent visual stimulation. Cogn Affect Behav Neurosci 4: 127-132.
55. Overath T, Cusack R, Kumar S, von Kriegstein K, Warren JD, Grube M, Carlyon RP, Griffiths TD ( 2007): An information theoretic characterization of auditory encoding. PLOS Biol 5: e288.
56. Rao RP ( 1999): An optimal estimation approach to visual perception and learning. Vis Res 39: 1963-1989.
57. Ruff CC, Blankenburg F, Bjoertomt O, Bestmann S, Freeman E, Haynes J-D, Rees G, Josephs O, Deichmann R, Driver J ( 2006): Concurrent TMS-fMRI and psychophysics reveal frontal influences on human retinotopic visual cortex. Curr Biol 16: 1479-1488.
58. Schroeder CE, Foxe J ( 2005): Multisensory contributions to low-level, "unisensory" processing. Curr Opin Neurobiol 15: 454-458.
59. Smiley JF, Falchier A ( 2009): Multisensory connections of monkey auditory cerebral cortex. Hear Res 258: 37-46.
60. Spence C, Driver J, editors ( 2004): Crossmodal Space and Crossmodal Attention. Oxford, UK: Oxford University Press.
61. Stein BE, Meredith MA ( 1993): The Merging of the Senses. Cambridge, MA: MIT Press.
62. Stein BE, Wallace MT ( 1996): Comparisons of cross-modality integration in midbrain and cortex. Prog Brain Res 112: 289-299.
63. Stein BE, London N, Wilkinson LK, Price DD ( 1996): Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis. J Cogn Neurosci 8: 497-506.
64. Stevenson RA, James TW ( 2009): Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44: 1210-1223.
65. Stevenson RA, Altieri NA, Kim S, Pisoni DB, James TW ( 2010): Neural processing of asynchronous audiovisual speech perception. Neuroimage 49: 3308-3318.
66. Summerfield C, Koechlin E ( 2008): A neural representation of prior information during perceptual inference. Neuron 59: 336-347.
67. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M ( 2002): Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single subject brain. NeuroImage 15: 273-289.
68. van Atteveldt NM, Formisano E, Blomert L, Goebel R ( 2007): The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17: 962-974.
69. Wallace MT, Wilkinson LK, Stein BE ( 1996): Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76: 1246-1266.
70. Werner S, Noppeney U ( 2010): Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb Cortex 20: 1829-1842.