Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment

Cerebral Cortex

Volumn 15, Issue 7, 2005, Pages 963-974

  Murray, Micah M ^a,b,c   Molholm, Sophie ^a   Michel, Christoph M ^c   Heslenfeld, Dirk J ^d   Ritter, Walter ^a   Javitt, Daniel C ^a   Schroeder, Charles E ^a,d   Foxe, John J ^a,d  

^a NATHAN S KLINE INSTITUTE FOR PSYCHIATRIC RESEARCH   (United States)

^b LAUSANNE UNIVERSITY HOSPITAL   (Switzerland)

^c GENEVA UNIVERSITY HOSPITALS   (Switzerland)

^d ALBERT EINSTEIN COLLEGE OF MEDICINE   (United States)

Author keywords

Area CM; Cross modal; Event related potential (ERP); Human; Laura source estimation; Redundant signals effect (RSE)

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BRAIN ELECTROPHYSIOLOGY; CLINICAL ARTICLE; EVENT RELATED POTENTIAL; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; NERVE POTENTIAL; NORMAL HUMAN; PRIORITY JOURNAL; SENSORY CORTEX; SOMATOSENSORY STIMULATION; SUPERIOR COLLICULUS;

ADULT; AUDITORY PERCEPTION; COGNITION; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS, AUDITORY; EVOKED POTENTIALS, SOMATOSENSORY; EVOKED POTENTIALS, VISUAL; HUMANS; PSYCHOPHYSICS; TOUCH; VISUAL PERCEPTION;

EID: 20544460923     PISSN: 10473211     EISSN: None     Source Type: Journal    
DOI: 10.1093/cercor/bhh197     Document Type: Article

References (89)

1. Ahissar M, Ahissar E, Bergman H, Vaadia E (1992) Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. J Neurophysiol 67:203-215.
2. Andersen RA, Bunco CA (2002) Intentional maps in posterior parietal cortex. Annu Rev Neurosci 25:189-220.
3. Bertelson P (1998). Starting from the ventriloquist: The perception of multimodal events. In: Advances in psychological science II: Biological and cognitive aspects (Sabourin M, Craik F, Roberts M, eds), pp. 419-439. Hove: Psychology Press.
4. Brungart DS, Kordik AJ, Simpson BD, McKinley RL (2003) Auditory localization in the horizontal plane with single and double hearing protection. Aviat Space Environ Med 74:937-946.
5. Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile 'capture' of audition. Percept Psychophys 64:616-630.
6. Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110-1123.
7. Clarke S, Bellmann-Thiran A, Maeder P, Adriani M, Vernet O, Regli L, Cuisenaire O, Thiran JP (2002) What and where in human audition: selective deficits following focal hemispheric lesions. Exp Brain Res 147:8-15.
8. Cooke DF, Taylor CS, Moore T, Graziano MS (2003) Complex movements evoked by microstimulation of the ventral intraparietal area. Proc Natl Acad Sci USA 100:6163-6168.
9. Ducommun CY, Murray MM, Thut G, Bellmann A, Viaud-Delmon I, Clarke S, Michel CM (2002). Segregated processing of auditory motion and auditory location: an ERP mapping study. NeuroImage 16:76-88.
10. Falchier A, Clavagnier S, Barone P, Kennedy H (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749-5759.
11. Farnè A, Làdavas E (2002) Auditory peripersonal space in humans. J Cogn Neurosci 14:1030-1043.
12. Farnè A, Demattè ML, Làdavas E (2003) Beyond the window: multisensory representation of peripersonal space across a transparent barrier. Int J Psychophysiol 50:51-61.
13. Fender DH (1987) Source localisation of brain electrical activity. In: Handbook of electroencephalography and clinical neurophysiology, vol. 1: Methods of analysis of brain electrical and magnetic signals (Gevins AS, Remond A, eds), pp. 355-399. Amsterdam: Elsevier.
14. Forster B, Cavina-Pratesi C, Aglioti S, Berlucchi G (2002) Redundant target effect and intersensory facilitation from visual-tactile interactiosn in simple reaction time. Exp Brain Res 143:480-487.
15. Fort A, Delpuech C, Pernier J, Giard MH (2002) Early auditory-visual interactions in human cortex during nonredundant target identification. Cogn Brain Res 14:20-30.
16. Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cogn Brain Res 10:77-83.
17. Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540-543.
18. Frassinetti F, Bolognini N, Làdavas E (2002) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332-343.
19. Fu KMG, Johnston TA, Shah AS, Arnold L, Smiley J, Hackett TA, Garraghty PE, Schroeder CE (2003) Auditory cortical neurons respond to somatosensory stimulation. J Neurosci 23:7510-7515.
20. Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473-490.
21. Gobbelé R, Schürmann M, Forss N, Juottonen K, Buchner H, Hari R (2003) Activation of the human posterior and temporoparietal cortices during audiotactile interaction. Neuroimage 20:503-511.
22. Grave de Peralta R, Gonzalez Andino S, Lantz G, Michel CM, Landis T (2001) Noninvasive localization of electromagnetic epileptic activity. I. Method descriptions and simulations. Brain Topogr 14: 131-137.
23. Grave de Peralta R, Murray MM, Michel CM, Martuzzi R, Andino SG (2004) Electrical neuroimaging based on biophysical constraints. Neuroimage 21:527-539.
24. Gulden WO, Grusser O-J (1998) Is there a vestibular cortex? Trends Neurosci 21:254-259.
25. Haeske-Dewick H, Canavan AGM, Hömberg V (1996) Sound localization in egocentric space following hemispheric lesions. Neuropsychologia 34:937-942.
26. Hairston WD, Wallace MT, Vaughan JW, Stein BE, Norris JL, Schirillo JA (2003) Visual localization ability influences cross-modal bias. J Cogn Neurosci 15:20-29.
27. Harrington LK, Peck CK (1998) Spatial disparity affects visual-auditory interactions in human sensorimotor processing. Exp Brain Res 122:247-252.
28. Hotting K, Rösler F, Röder B (2003) Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli. Exp Brain Res 148:26-37.
29. Hughes HC, Reuter-Lorenz PA, Nozawa G, Fendrich R (1994) Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J Exp Psychol Hum Percept Perform 20:131-53.
30. Jay MF, Sparks DL (1987) Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. J Neurophysiol 57:35-55.
31. Jiang W, Stein BE (2003) Cortex controls multisensory depression in superior colliculus. J Neurophysiol 90:2123-2135.
32. Jones EG, Powell TP (1970) An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 93:793-820.
33. Jousmaki V, Hari R (1998) Parchment-skin illusion: sound-biased touch. Curr Biol 8:R190.
34. Kaas JH, Hackett TA, Tramo MJ (1999) Auditory processing in the primate cerebral cortex. Curr Opin Neurobiol 9:164-170.
35. Klemm O (1909) Localisation von sinneneindrucken bei disparaten nebenreizen [Localization of sensory impressions with disparate distractors]. Psychologische Studien (Wundt) 5:73-161.
36. Lehmann D (1987) Principles of spatial analysis. In: Handbook of electroencephalography and clinical neurophysiology, vol. 1: Methods of analysis of brain electrical and magnetic signals (Gevins AS, Remond A, eds), pp. 309-354. Amsterdam: Elsevier.
37. Lehmann D, Skrandies W (1980) Reference-free identification of components of checkerboard-evoked multichannel potential fields. Electroencephalogr Clin Neurophysiol 48:609-621.
38. Leinonen L, Hyvarinen J, Sovijarvi AR (1980) Functional properties of neurons in the temporo-parietal association cortex of awake monkey. Exp Brain Res 39:203-215.
39. Lütkenhöner B, Lammertmann C, Simoes C, Hari R (2002) Magnetoencephalographic correlates of audiotactile interaction. Neuroimage 15:509-22.
40. Macaluso E, Frith CD, Driver J (2000) Modulation of human visual cortex by crossmodal spatial attention. Science 289:1206-1208.
41. Macaluso E, Frith CD, Driver J (2002) Crossmodal spatial influences of touch on extrastriate visual areas take current gaze direction into account. Neuron 34:647-658.
42. Maravita A, Spence C, Driver J (2003) Multisensory integration and the body schema: close to hand and within reach. Curr Biol 13:R531-9.
43. Massaro DW (1998) Speechreading: illusion or window into pattern recognition. Trends Cogn Sci 3:310-317.
44. Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215-3229.
45. Michel CM, Thut G, Morand S, Khateb A, Pegna AJ, Grave de Peralta R, Gonzalez S, Seeck M, Landis T (2001) Electric source imaging of human brain functions. Brain Res Rev 36:108-118.
46. Miller J (1982) Divided attention: evidence for reactivation with redundant signals. Cognit Psychol 14:247-279.
47. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn Brain Res 14:115-128.
48. Molholm S, Ritter W, Javitt DC, Foxe JJ (2004) Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cerebl Cortex 14:452-465.
49. Murray MM, Foxe JJ, Higgins BA, Javitt DC, Schroeder CE (2001) Visuo-spatial neural response interactions in early visual cortical processing during a simple reaction time task: a high-density electrical mapping study. Neuropsychologia 39:828-844.
50. Murray MM, Michel CM, Grave de Peralta R, Ortigue S, Brunet D, Andino SG, Schnider A (2004) Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging. Neuroimage 21:125-135.
51. Obayashi S, Suhara T, Kawabe K, Okauchi T, Maeda J, Akine Y, Onoe H, Iriki A (2001) Functional brain mapping of monkey tool use. Neuroimage 14:853-861.
52. Odgaard EC, Arieh Y, Marks LE (2003) Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys 65:123-132.
53. Okajima Y, Chino N, Takahashi M, Kimura A (1995) Effects of visual and auditory stimuli on median nerve somatosensory evoked potentials in man. Electromyogr Clin Neurophysiol 35:251-256.
54. Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh Inventory. Neuropsychologia 9:97-113.
55. Pascual-Marqui RD, Michel CM, Lehmann D (1995) Segmentation of brain electrical activity into microstates: model estimation and validation. IEEE Trans Biomed Eng 42:658-665.
56. Pavani F, Spence C, Driver J (2000) Visual capture of touch: Out-of-the-body experiences with rubber gloves. Psychol Sci 11:353-359.
57. Perrin F, Pernier J, Bertrand O, Giard MH, Echallier JF (1987) Mapping of scalp potentials by surface spline interpolation. Electroencephalogr Clin Neurophysiol 66:75-81.
58. Raab D (1962) Statistical facilitation of simple reaction times. Trans N Y Acad Sci 24:574-590.
59. Radeau M (1994) Auditory-visual interaction and modularity. Curr Psychol Cogn 13:3-51.
60. Recanzone GH, Guard DC, Phan ML, Su TK (2000) Correlation between the activity of single auditory cortical neurons and sound-localization behavior in the macaque monkey. J Neurophysiol 83:2723-2739.
61. Rivier F, Clarke S (1997) Cytochrome oxidase, acetylcholinesterase, and NADPH-diaphorase staining in human supratemporal and isular cortex: evidence for multiple auditory areas. Neuroimage 6:288-304.
62. Rockland KS, Ojima H (2003) Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol 50:19-26.
63. Romanski LM, Clugnet MC, Bordi F, LeDoux, JE (1993) Somatosensory and auditory convergence in the lateral nucleus of the amygdala. Behav Neurosci 107:444-450.
64. Sarno S, Erasmus, LP, Lipp B, Schlaegel W (2003) Multisensory integration after traumatic brain injury: a reaction time study between pairings Division, touch, and audition. Brain Injury 17:413-426.
65. Schroeder CE, Foxe JJ (2002) Timing and laminar profile of converging inputs in multisensory areas of the macaque neocortex. Cogn Brain Res 14:195-207.
66. Schroeder CE, Foxe JJ (2004) Multisensory convergence in early cortical processing. In: The handbook of multisensory processes (Stein BE, Calvert GA, Spence C, eds), pp. 295-309. New York: MIT Press.
67. Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322-1327.
68. Schroeder CE, Smiiley J, Fu KMG, McGinnis T, O'Connell MN, Hackett TA (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50:5-18.
69. Schroeder CE, Molholm S, Lakatos P, Ritter W, Foxe JJ (2004) Human-simian correspondence in the early cortical processing of multisensory cues. Cognit Process 5:140-151.
70. Schröger E, Widmann A (1998) Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiol 35:755-759.
71. Schürmann M, Kolev V, Menzel K, Yordanova J (2002) Spatial coincidence modulates interaction between visual and somatosensory evoked potentials. Neuroreport 13:779-783.
72. Sekuler R, Sekuler AB, Lau R (1997) Sound alters visual motion perception. Nature 385:308.
73. Shams L, Kamitani Y, Shimojo S (2000) What you see is what you hear. Nature 408:788.
74. Slutsky DA, Recanzone GH (2001) Temporal and spatial dependency of the ventriloquism effect. Neuroreport 12:7-10.
75. Spence C, Driver J (2000) Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism. Neuroreport 11:2057-2061.
76. Stein BE, Meredith MA (1993) The merging of the senses. Cambridge, MA: MIT Press.
77. Stein BE, Wallace MT (1996) Comparisons of cross-modality integration in midbrain and cortex. In: Progress in brain research, Vol. 112 (Norita M, Bando T, Stein BE, eds), pp. 289-299. Amsterdam: Elsevier.
78. Stein BE, Meredith MA, Huneycutt WS, McDade L (1989) Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J Cogn Neurosci 1:12-24.
79. Stein BE, London N, Wilkinson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8:497-506.
80. Sumby W, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Am Soc Audiol 26:212-215.
81. Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human brain. New York: Thieme.
82. Teder-Salejarvi WA, McDonald JJ, Di Russo F, Hillyard SA (2002) An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Cogn Brain Res 14:106-114.
83. Tesche C (2000) Evidence for somatosensory evoked responses in human temporal lobe. Neuroreport 11:2655-2658.
84. Thomas GJ (1941) Experimental study of the influence of vision on sound localization. J Exp Psychol 28:107-77.
85. Todd JH (1912) Reaction to multiple stimuli. Arch Psychol 21:1-65.
86. Tolias AS, Moore T, Smirnakis SM, Tehovnik EJ, Siapas AG, Schiller PH (2001) Eye movements modulate visual receptive fields of V4 neurons. Neuron 29:757-767.
87. Warren DH, Welch RB, McCarthy TJ (1981) The role of visual-auditory 'comellingness' in the ventriloquism effect: implications for transitivity among the spatial senses. Percept Psychophys 30:557-64.
88. Woldorff MG, Tempelmann C, Fell J, Tegeler C, Gaschler-Merkefski B, Hinrichs H, Heinze H-J, Scheich H (1999) Lateralized auditory spatial perception and the contralaterality of cortical processing as studied with functional magnetic resonance imaging and magnetoencephalography. Hum Brain Mapp 7:49-66.
89. Worgotter F, Eysel UT (2000) Context, state and the receptive fields of striatal cortex cells. Trends Neurosci 23:497-503.