Neural basis of multisensory looming signals

NeuroImage

Volumn 65, Issue , 2013, Pages 13-22

  Tyll, Sascha ^a   Bonath, Björn ^a   Schoenfeld, Mircea Ariel ^a,b   Heinze, Hans Jochen ^a   Ohl, Frank W ^b,c   Noesselt, Tömme ^a,c  

^a OTTO VON GUERICKE UNIVERSITY   (Germany)

^b LEIBNIZ INSTITUTE FOR NEUROBIOLOGY   (Germany)

^c Center for Behavioral and Brain Sciences   (Germany)

Author keywords

Audiovisual looming; Connectivity; FMRI; Human; Multisensory integration

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; AUDITORY STIMULATION; FEMALE; FRONTAL LOBE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGE ENHANCEMENT; MALE; NERVE CELL NETWORK; NORMAL HUMAN; PARIETAL LOBE; PRIORITY JOURNAL; SUPERIOR TEMPORAL SULCUS; VISUAL CORTEX; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; COMPREHENSION; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; PHOTIC STIMULATION; REACTION TIME; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84868231370     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2012.09.056     Document Type: Article

References (93)

1. Alais D., Burr D. No direction-specific bimodal facilitation for audiovisual motion detection. Brain Res. Cogn. Brain Res. 2004, 19:185-194.
2. Alais D., Newell F.N., Mamassian P. Multisensory processing in review: from physiology to behaviour. Seeing Perceiving 2010, 23:3-38.
3. Allman B.L., Bittencourt-Navarrete R.E., Keniston L.P., Medina A.E., Wang M.Y., Meredith M.A. Do cross-modal projections always result in multisensory integration?. Cereb. Cortex 2008, 18:2066-2076.
4. Angelaki D.E., Gu Y., DeAngelis G.C. Multisensory integration: psychophysics, neurophysiology, and computation. Curr. Opin. Neurobiol. 2009, 19:452-458.
5. Bach D.R., Neuhoff J.G., Perrig W., Seifritz E. Looming sounds as warning signals: the function of motion cues. Int. J. Psychophysiol. 2009, 74:28-33.
6. Ball W., Tronick E. Infant responses to impending collision: optical and real. Science 1971, 171:818-820.
7. Barraclough N.E., Xiao D., Baker C.I., Oram M.W., Perrett D.I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 2005, 17:377-391.
8. Baumann O., Greenlee M.W. Neural correlates of coherent audiovisual motion perception. Cereb. Cortex 2007, 17:1433-1443.
9. Beauchamp M.S. See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr. Opin. Neurobiol. 2005, 15:145-153.
10. Beauchamp M.S. Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 2005, 3:93-113.
11. Beauchamp M.S., Argall B.D., Bodurka J., Duyn J.H., Martin A. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 2004, 7:1190-1192.
12. Beauchamp M.S., Nath A.R., Pasalar S. fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J. Neurosci. 2010, 30:2414-2417.
13. Benevento L.A., Fallon J., Davis B.J., Rezak M. Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp. Neurol. 1977, 57:849-872.
14. Bonath B., Noesselt T., Martinez A., Mishra J., Schwiecker K., Heinze H.J., Hillyard S.A. Neural basis of the ventriloquist illusion. Curr. Biol. 2007, 17:1697-1703.
15. Calvert G.A., Hansen P.C., Iversen S.D., Brammer M.J. Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 2001, 14:427-438.
16. Cappe C., Thut G., Romei V., Murray M.M. Selective integration of auditory-visual looming cues by humans. Neuropsychologia 2009, 47:1045-1052.
17. Cappe C., Thut G., Romei V., Murray M.M. Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources. J. Neurosci. 2010, 30:12572-12580.
18. Cappe C., Thelen A., Romei V., Thut G., Murray M.M. Looming signals reveal synergistic principles of multisensory integration. J. Neurosci. 2012, 32:1171-1182.
19. Carter C.S., van Veen V. Anterior cingulate cortex and conflict detection: an update of theory and data. Cogn. Affect. Behav. Neurosci. 2007, 7:367-379.
20. Coull J.T., Nobre A.C. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J. Neurosci. 1998, 18:7426-7435.
21. Doehrmann O., Weigelt S., Altmann C.F., Kaiser J., Naumer M.J. Audiovisual functional magnetic resonance imaging adaptation reveals multisensory integration effects in object-related sensory cortices. J. Neurosci. 2010, 30:3370-3379.
22. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 2008, 57:11-23.
23. Eickhoff S.B., Stephan K.E., Mohlberg H., Grefkes C., Fink G.R., Amunts K., Zilles K. A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 2005, 25:1325-1335.
24. Falchier A., Clavagnier S., Barone P., Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 2002, 22:5749-5759.
25. Franconeri S.L., Simons D.J. Moving and looming stimuli capture attention. Percept. Psychophys. 2003, 65:999-1010.
26. Friston K.J., Holmes A.P., Poline J.B., Grasby P.J., Williams S.C., Frackowiak R.S., Turner R. Analysis of fMRI time-series revisited. Neuroimage 1995, 2:45-53.
27. Friston K.J., Buechel C., Fink G.R., Morris J., Rolls E., Dolan R.J. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 1997, 6:218-229.
28. Fuster J.M. Temporal processing. Ann. N. Y. Acad. Sci. 1995, 769:173-181.
29. Gentile G., Petkova V.I., Ehrsson H.H. Integration of visual and tactile signals from the hand in the human brain: an FMRI study. J. Neurophysiol. 2011, 105:910-922.
30. Ghazanfar A.A., Schroeder C.E. Is neocortex essentially multisensory?. Trends Cogn. Sci. 2006, 10:278-285.
31. Ghazanfar A.A., Neuhoff J.G., Logothetis N.K. Auditory looming perception in rhesus monkeys. Proc. Natl. Acad. Sci. U. S. A. 2002, 99:15755-15757.
32. Ghazanfar A.A., Maier J.X., Hoffman K.L., Logothetis N.K. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J. Neurosci. 2005, 25:5004-5012.
33. Ghazanfar A.A., Chandrasekaran C., Logothetis N.K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. J. Neurosci. 2008, 28:4457-4469.
34. Glascher J. Visualization of group inference data in functional neuroimaging. Neuroinformatics 2009, 7:73-82.
35. Grassi M. Sex difference in subjective duration of looming and receding sounds. Perception 2010, 39:1424-1426.
36. Grassi M., Darwin C.J. The subjective duration of ramped and damped sounds. Percept. Psychophys. 2006, 68:1382-1392.
37. Gray R. Looming auditory collision warnings for driving. Hum. Factors 2011, 53:63-74.
38. Hall D.A., Moore D.R. Auditory neuroscience: the salience of looming sounds. Curr. Biol. 2003, 13:R91-R93.
39. Holmes N.P. The law of inverse effectiveness in neurons and behaviour: multisensory integration versus normal variability. Neuropsychologia 2007, 45:3340-3345.
40. Hubel D.H., Wiesel T.N. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 1968, 195:215-243.
41. Kanowski M., Rieger J.W., Noesselt T., Tempelmann C., Hinrichs H. Endoscopic eye tracking system for fMRI. J. Neurosci. Methods 2007, 160:10-15.
42. Krumbholz K., Schonwiesner M., von Cramon D.Y., Rubsamen R., Shah N.J., Zilles K., Fink G.R. Representation of interaural temporal information from left and right auditory space in the human planum temporale and inferior parietal lobe. Cereb. Cortex 2005, 15:317-324.
43. Laurienti P.J., Perrault T.J., Stanford T.R., Wallace M.T., Stein B.E. On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp. Brain Res. 2005, 166:289-297.
44. Leo F., Romei V., Freeman E., Ladavas E., Driver J. Looming sounds enhance orientation sensitivity for visual stimuli on the same side as such sounds. Exp. Brain Res. 2011, 213:193-201.
45. Lewis R., Noppeney U. Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J. Neurosci. 2010, 30:12329-12339.
46. Lewkowicz D.J. Perception of dynamic and static audiovisual sequences in 3- and 4-month-old infants. Child Dev. 2008, 79:1538-1554.
47. Liu Y.J., Wang Q., Li B. Neuronal responses to looming objects in the superior colliculus of the cat. Brain Behav. Evol. 2011, 77:193-205.
48. Love S.A., Pollick F.E., Latinus M. Cerebral correlates and statistical criteria of cross-modal face and voice integration. Seeing Perceiving 2011, 24:351-367.
49. Ma W.J., Pouget A. Linking neurons to behavior in multisensory perception: a computational review. Brain Res. 2008, 1242:4-12.
50. Macaluso E., Frith C.D., Driver J. Modulation of human visual cortex by crossmodal spatial attention. Science 2000, 289:1206-1208.
51. Maier J.X., Ghazanfar A.A. Looming biases in monkey auditory cortex. J. Neurosci. 2007, 27:4093-4100.
52. Maier J.X., Neuhoff J.G., Logothetis N.K., Ghazanfar A.A. Multisensory integration of looming signals by rhesus monkeys. Neuron 2004, 43:177-181.
53. Maier J.X., Chandrasekaran C., Ghazanfar A.A. Integration of bimodal looming signals through neuronal coherence in the temporal lobe. Curr. Biol. 2008, 18:963-968.
54. Marchant J.L., Ruff C.C., Driver J. Audiovisual synchrony enhances BOLD responses in a brain network including multisensory STS while also enhancing target-detection performance for both modalities. Hum. Brain Mapp. 2012, 33:1212-1224.
55. Morosan P., Rademacher J., Schleicher A., Amunts K., Schormann T., Zilles K. Human primary auditory cortex: cytoarchitectonic subdivisions and mapping into a spatial reference system. Neuroimage 2001, 13:684-701.
56. Nebel K., Wiese H., Stude P., de Greiff A., Diener H.C., Keidel M. On the neural basis of focused and divided attention. Brain Res. Cogn. Brain Res. 2005, 25:760-776.
57. Neuhoff J.G. Perceptual bias for rising tones. Nature 1998, 395:123-124.
58. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.J., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 2007, 27:11431-11441.
59. Noesselt T., Bonath B., Boehler C.N., Schoenfeld M.A., Heinze H.J. On perceived synchrony-neural dynamics of audiovisual illusions and suppressions. Brain Res. Cogn. Brain Res. 2008, 1220:132-141.
60. Noesselt T., Tyll S., Boehler C.N., Budinger E., Heinze H.J., Driver J. Sound-induced enhancement of low-intensity vision: multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. J. Neurosci. 2010, 30:13609-13623.
61. Noppeney U. Characterization of multisensory integration with fMRI: experimental design, statistical analysis, and interpretation. The Neural Bases of Multisensory Processes 2012, CRC Press, Boca Raton (FL), Chapter 13. M.M. Murray, M.T. Wallace (Eds.).
62. Padberg J., Seltzer B., Cusick C.G. Architectonics and cortical connections of the upper bank of the superior temporal sulcus in the rhesus monkey: an analysis in the tangential plane. J. Comp. Neurol. 2003, 467:418-434.
63. Parker A., Alais D. A bias for looming stimuli to predominate in binocular rivalry. Vis. Res. 2007, 47:2661-2674.
64. Peron S., Gabbiani F. Spike frequency adaptation mediates looming stimulus selectivity in a collision-detecting neuron. Nat. Neurosci. 2009, 12:318-326.
65. Poldrack R.A. Region of interest analysis for fMRI. Soc. Cogn. Affect. Neurosci. 2007, 2:67-70.
66. Romei V., Murray M.M., Cappe C., Thut G. Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Curr. Biol. 2009, 19:1799-1805.
67. Sadaghiani S., Maier J.X., Noppeney U. Natural, metaphoric, and linguistic auditory direction signals have distinct influences on visual motion processing. J. Neurosci. 2009, 29:6490-6499.
68. Scheperjans F., Eickhoff S.B., Homke L., Mohlberg H., Hermann K., Amunts K., Zilles K. Probabilistic maps, morphometry, and variability of cytoarchitectonic areas in the human superior parietal cortex. Cereb. Cortex 2008, 18:2141-2157.
69. Schiff W., Caviness J.A., Gibson J.J. Persistent fear responses in rhesus monkeys to the optical stimulus of "looming". Science 1962, 136:982-983.
70. Schlauch R.S., Ries D.T., DiGiovanni J.J. Duration discrimination and subjective duration for ramped and damped sounds. J. Acoust. Soc. Am. 2001, 109:2880-2887.
71. Scobey R.P., Gabor A.J. Orientation discrimination sensitivity of single units in cat primary visual cortex. Exp. Brain Res. 1989, 77:398-406.
72. Seifritz E., Neuhoff J.G., Bilecen D., Scheffler K., Mustovic H., Schachinger H., Elefante R., Di Salle F. Neural processing of auditory looming in the human brain. Curr. Biol. 2002, 12:2147-2151.
73. Seltzer B., Pandya D.N. Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J. Comp. Neurol. 1994, 343:445-463.
74. Stanford T.R., Stein B.E. Superadditivity in multisensory integration: putting the computation in context. Neuroreport 2007, 28:787-792.
75. Stanford T.R., Quessy S., Stein B.E. Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 2005, 25:6499-6508.
76. Stein B.E. Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp. Brain Res. 1998, 123:124-135.
77. Stein B., Meredith M. The Merging of the Senses 1993, The MIT Press, Cambridge, MA.
78. Stein B.E., Stanford T.R. Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 2008, 9:255-266.
79. Stein B.E., Huneycutt W.S., Meredith M.A. Neurons and behavior: the same rules of multisensory integration apply. Brain Res. 1988, 448:355-358.
80. Stein B.E., Stanford T.R., Ramachandran R., Perrault T.J., Rowland B.A. Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness. Exp. Brain Res. 2009, 198:113-126.
81. Stevenson R.A., James T.W. Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 2009, 44:1210-1223.
82. Stevenson R.A., Geoghegan M.L., James T.W. Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Exp. Brain Res. 2007, 179:85-95.
83. Takeuchi T. Visual search of expansion and contraction. Vis. Res. 1997, 37:2083-2090.
84. Talsma D., Doty T.J., Woldorff M.G. Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration?. Cereb. Cortex 2007, 2007(17):679-690.
85. Talsma D., Senkowski D., Soto-Faraco S., Woldorff M.G. The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 2010, 14:400-410.
86. Walker-Andrews A.S., Lennon E.M. Auditory-visual perception of changing distance by human infants. Child Dev. 1985, 56:544-548.
87. Wann J.P., Poulter D.R., Purcell C. Reduced sensitivity to visual looming inflates the risk posed by speeding vehicles when children try to cross the road. Psychol. Sci. 2011, 22:429-434.
88. Warren J.D., Griffiths T.D. Distinct mechanisms for processing spatial sequences and pitch sequences in the human auditory brain. J. Neurosci. 2003, 23:5799-5804.
89. Werner S., Noppeney U. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb. Cortex 2010, 20:1829-1842.
90. Werner S., Noppeney U. The contributions of transient and sustained response codes to audiovisual integration. Cereb. Cortex 2011, 21:920-931.
91. Wilkinson L.K., Meredith M.A., Stein B.E. The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp. Brain Res. 1996, 112:1-10.
92. Wittmann M., van Wassenhove V., Craig A.D., Paulus M.P. The neural substrates of subjective time dilation. Front. Hum. Neurosci. 2010, 4:2.
93. Wright T.M., Pelphrey K.A., Allison T., McKeown M.J., McCarthy G. Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 2003, 13:1034-1043.