Multisensory interactions in early evoked brain activity follow the principle of inverse effectiveness

NeuroImage

Volumn 56, Issue 4, 2011, Pages 2200-2208

  Senkowski, Daniel ^a,b,c   Saint Amour, Dave ^d,e   Höfle, Marion ^b   Foxe, John J ^a,f  

^a NATHAN S KLINE INSTITUTE FOR PSYCHIATRIC RESEARCH   (United States)

^b UNIVERSITY MEDICAL CENTER HAMBURG EPPENDORF   (Germany)

^c CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

^d UNIVERSITY OF MONTREAL   (Canada)

^e UNIVERSITÉ DU QUÉBEC À MONTRÉAL   (Canada)

^f ALBERT EINSTEIN COLLEGE OF MEDICINE   (United States)

Author keywords

Bisensory; Crossmodal; EEG; ERP; Inverse effectiveness; Multisensory

Indexed keywords

ADULT; ANALYTIC METHOD; ARTICLE; AUDITORY STIMULATION; BRAIN FUNCTION; CONTROLLED STUDY; ELECTROENCEPHALOGRAPHY; EVENT RELATED POTENTIAL; FACILITATION; FEMALE; HUMAN; HUMAN EXPERIMENT; INVERSE EFFECTIVENESS; LATENT PERIOD; MALE; NORMAL HUMAN; PRIORITY JOURNAL; PROBABILITY; REACTION TIME; RESPONSE TIME; SENSORY STIMULATION; SIGNAL TRANSDUCTION; SOMATOSENSORY STIMULATION; TEMPORAL SUMMATION; TOPOGRAPHY; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; AUDITORY PERCEPTION; BRAIN; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; HUMANS; MIDDLE AGED; PHOTIC STIMULATION; REACTION TIME; SIGNAL PROCESSING, COMPUTER-ASSISTED; VISUAL PERCEPTION; YOUNG ADULT;

EID: 79957478602     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2011.03.075     Document Type: Article

References (47)

1. Busse L., Woldorff M.G. The ERP omitted stimulus response to "no-stim" events and its implications for fast-rate event-related fMRI designs. Neuroimage 2003, 18:856-864.
2. Cappe C., Murray M.M., Barone P., Rouiller E.M. Multisensory facilitation of behavior in monkeys: effects of stimulus intensity. J. Cogn. Neurosci. 2010, 22:2850-2863.
3. Cappe C., Thut G., Romei V., Murray M.M. Selective integration of auditory-visual looming cues by humans. Neuropsychologia 2009, 47:1045-1052.
4. Cappe C., Thut G., Romei V., Murray M.M. Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources. J. Neurosci. 2010, 30:12572-12580.
5. Corneil B.D., Van Wanrooij M., Munoz D.P., Van Opstal A.J. Auditory-visual interactions subserving goal-directed saccades in a complex scene. J. Neurophysiol. 2002, 88:438-454.
6. Diederich A., Colonius H. Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept. Psychophys. 2004, 66:1388-1404.
7. Doehrmann O., Naumer M.J. Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res. 2008, 1242:136-150.
8. Ellemberg D., Hammarrenger B., Lepore F., Roy M.S., Guillemot J.P. Contrast dependency of VEPs as a function of spatial frequency: the parvocellular and magnocellular contributions to human VEPs. Spat. Vis. 2001, 15:99-111.
9. Fort A., Delpuech C., Pernier J., Giard M.H. Dynamics of cortico-subcortical cross- modal operations involved in audio-visual object detection in humans. Cereb. Cortex 2002, 12:1031-1039.
10. Foxe J.J., Morocz I.A., Murray M.M., Higgins B.A., Javitt D.C., Schroeder C.E. Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res. Cogn. Brain Res. 2000, 10:77-83.
11. Giard M.H., Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 1999, 11:473-490.
12. Gondan M., Niederhaus B., Rosler F., Roder B. Multisensory processing in the redundant-target effect: a behavioral and event-related potential study. Percept. Psychophys. 2005, 67:713-726.
13. Gondan M., Roder B. A new method for detecting interactions between the senses in event-related potentials. Brain Res. 2006, 1073-1074:389-397.
14. Guthrie D., Buchwald J.S. Significance testing of difference potentials. Psychophysiology 1991, 28:240-244.
15. Hegerl U., Gallinat J., Mrowinski D. Intensity dependence of auditory evoked dipole source activity. Int. J. Psychophysiol. 1994, 17:1-13.
16. Holle H., Obleser J., Rueschemeyer S.A., Gunter T.C. Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. Neuroimage 2010, 49:875-884.
17. Holmes N.P. The principle of inverse effectiveness in multisensory integration: some statistical considerations. Brain Topogr. 2009, 21:168-176.
18. Jaskowski P., Rybarczyk K., Jaroszyk F. The relationship between latency of auditory evoked potentials, simple reaction time, and stimulus intensity. Psychol. Res. 1994, 56:59-65.
19. Kiesel A., Miller J., Ulrich R. Systematic biases and Type I error accumulation in tests of the race model inequality. Behav. Res. Methods 2007, 39:539-551.
20. Laurienti P.J., Kraft R.A., Maldjian J.A., Burdette J.H., Wallace M.T. Semantic congruence is a critical factor in multisensory behavioral performance. Exp. Brain Res. 2004, 158:405-414.
21. Meredith M.A., Stein B.E. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol. 1986, 56:640-662.
22. Michalewski H.J., Starr A., Zeng F.G., Dimitrijevic A. N100 cortical potentials accompanying disrupted auditory nerve activity in auditory neuropathy (AN): effects of signal intensity and continous noise. Clin. Neurophysiol. 2009, 120:1352-1363.
23. Miller J. Divided attention: evidence for coactivation with redundant signals. Cognit. Psychol. 1982, 14:247-279.
24. Molholm S., Ritter W., Javitt D.C., Foxe J.J. Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb. Cortex 2004, 14:452-465.
25. Molholm S., Ritter W., Murray M.M., Javitt D.C., Schroeder C.E., Foxe J.J. Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res. Cogn. Brain Res. 2002, 14:115-128.
26. Murray M.M., Molholm S., Michel C.M., Heslenfeld D.J., Ritter W., Javitt D.C., Schroeder C.E., Foxe J.J. Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb. Cortex 2005, 15:963-974.
27. Murray M.M., Brunet D., Michel C.M. Topographic ERP analyses: a step-by-step tutorial review. Brain Topogr. 2008, 20:249-264.
28. Perrin F., Pernier J., Bertrand O., Echallier J.F. Spherical splines for scalp potential and current density mapping. Electroencephalogr. Clin. Neurophysiol. 1989, 72:184-187.
29. Rach S., Diederich A., Colonius H. On quantifying multisensory interaction effects in reaction time and detection rate. Psychol. Res. 2011, 75:77-94.
30. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 2007, 17:1147-1153.
31. Senkowski D., Linden M., Zubragel D., Bar T., Gallinat J. Evidence for disturbed cortical signal processing and altered serotonergic neurotransmission in generalized anxiety disorder. Biol. Psychiatry 2003, 53:304-314.
32. Senkowski D., Saint-Amour D., Kelly S.P., Foxe J.J. Multisensory processing of naturalistic objects in motion: a high-density electrical mapping and source estimation study. Neuroimage 2007, 36:877-888.
33. Schroeder C.E., Foxe J.J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res. 2002, 14:187-198.
34. Sperdin H.F., Cappe C., Foxe J.J., Murray M.M. Early, low-level auditory- somatosensory multisensory interactions impact reaction time speed. Front. Integr. Neurosci. 2009, 3:2.
35. Sperdin H.F., Cappe C., Murray M.M. The behavioral relevance of multisensory neural response interactions. Front. Neurosci. 2010, 4:9.
36. Stein, B.E., Meredith, M.A., 1993. The Merging of the Senses. MIT Press, Cambridge, MA.
37. Stein B.E., Stanford T.R. Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 2008, 9:255-266.
38. Stein B.E., Stanford T.R., Ramachandran R., Perrault T.J., Rowland B.A. Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness. Exp. Brain Res. 2009, 198:113-126.
39. Stekelenburg J.J., Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. J. Cogn. Neurosci. 2007, 19:1964-1973.
40. Stevenson R.A., James T.W. Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 2009, 44:1210-1223.
41. Talsma D., Doty T., Woldorff M.G. Selective attention and audiovisual integration: is attending to both modalities a prerequisite for optimal early integration?. Cereb. Cortex 2007, 17:679-690.
42. Talsma D., Senkowski D., Soto-Faraco S., Woldorff M.G. The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 2010, 14:400-410.
43. Talsma D., Woldorff M.G. Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. J. Cogn. Neurosci. 2005, 17:1098-1114.
44. Teder-Salejarvi W.A., Di Russo F., McDonald J.J., Hillyard S.A. Effects of spatial congruity on audio-visual multimodal integration. J. Cogn. Neurosci. 2005, 17:1396-1409.
45. Teder-Salejarvi W.A., McDonald J.J., Di Russo F., Hillyard S.A. An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res. Cogn. Brain Res. 2002, 14:106-114.
46. Wallace M.T., Stein B.E. Development of multisensory neurons and multisensory integration in cat superior colliculus. J. Neurosci. 1997, 17:2429-2444.
47. Werner S., Noppeney U. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb. Cortex 2010, 20:1829-1842.