The co-occurrence of multisensory facilitation and cross-modal conflict in the human brain

Journal of Neurophysiology

Volumn 106, Issue 6, 2011, Pages 2896-2909

  Diaconescu, Andreea Oliviana ^a,b   Alain, Claude ^a,b   McIntosh, Anthony Randal ^a,b  

^a ROTMAN RESEARCH INSTITUTE   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

Auditory; Event related synthetic aperture magnetometry analysis; Magnetoencephalography; Multisensory integration; Visual

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BRAIN; BRAIN REGION; CINGULATE GYRUS; CONFLICT; FEMALE; HIPPOCAMPUS; HUMAN; HUMAN EXPERIMENT; MAGNETOENCEPHALOGRAPHY; MALE; NORMAL HUMAN; PARIETAL CORTEX; POSTERIOR CINGULATE; PREFRONTAL CORTEX; PRIORITY JOURNAL; SEMANTICS; SENSORY STIMULATION; TASK PERFORMANCE; TEMPORAL CORTEX; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; ANALYSIS OF VARIANCE; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; FEMALE; HUMANS; MAGNETOENCEPHALOGRAPHY; MALE; MODELS, BIOLOGICAL; PHOTIC STIMULATION; PSYCHOPHYSICS; REACTION TIME; SEMANTICS; TIME FACTORS; VISUAL PERCEPTION; YOUNG ADULT;

EID: 83055161554     PISSN: 00223077     EISSN: 15221598     Source Type: Journal    
DOI: 10.1152/jn.00303.2011     Document Type: Article

References (67)

1. Baumann O, Greenlee MW. Neural correlates of coherent audiovisual motion perception. Cereb Cortex 17: 1433-1443, 2007.
2. Beauchamp MS, Nath AR, Pasalar S. fMRI-Guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J Neurosci 30: 2414-2417, 2010.
3. Belardinelli MO, Sestieri C, Di Matteo R, Delogu F, Del Gratta C, Ferretti A, Caulo M, Tartaro A, Romani GL. Audio-visual crossmodal interactions in environmental perception: An fMRI investigation. Cogn Process 5: 167-174, 2004.
4. Bezgin G, Wanke E, Krumnack A, Kotter R. Deducing logical relationships between spatially registered cortical parcellations under conditions of uncertainty. Neural Netw 21: 1132-1145, 2008.
5. Bishop CW, Miller LM. A multisensory cortical network for understanding speech in noise. J Cogn Neurosci 21: 1790-1805, 2009.
6. Brookes MJ, Stevenson CM, Barnes GR, Hillebrand A, Simpson MI, Francis ST, Morris PG. Beamformer reconstruction of correlated sources using a modified source model. Neuroimage 34: 1454-1465, 2007.
7. Calvert GA, Hansen PC, Iversen SD, Brammer MJ. Detection of audiovisual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14: 427-438, 2001.
8. Chen YC, Spence C. When hearing the bark helps to identify the dog: semantically-congruent sounds modulate the identification of masked pictures. Cognition 114: 389-404, 2010.
9. Cheyne D, Bakhtazad L, Gaetz W. Spatiotemporal mapping of cortical activity accompanying voluntary movements using an event-related beamforming approach. Hum Brain Mapp 27: 213-229, 2006.
10. Cheyne D, Bostan AC, Gaetz W, Pang EW. Event-related beamforming: a robust method for presurgical functional mapping using MEG. Clin Neurophysiol 118: 1691-1704, 2007.
11. Cox RW. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res 29: 162-173, 1996.
12. Diederich A, Colonius H. Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66: 1388-1404, 2004.
13. Doehrmann O, Naumer MJ. Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res 1242: 136-150, 2008.
14. Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals and other measures of statistical accuracy. Statist Sci 1: 54-77, 1986.
15. Giard MH, Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11: 473-490, 1999.
16. Grefkes C, Weiss PH, Zilles K, Fink GR. Crossmodal processing of object features in human anterior intraparietal cortex: an fMRI study implies equivalencies between humans and monkeys. Neuron 35: 173-184, 2002.
17. Gross J, Kujala J, Hamalainen M, Timmermann L, Schnitzler A, Salmelin R. Dynamic imaging of coherent sources: Studying neural interactions in the human brain. Proc Natl Acad Sci USA 98: 694-699, 2001.
18. Jaekl PM, Harris LR. Sounds can affect visual perception mediated primarily by the parvocellular pathway. Vis Neurosci 26: 477-486, 2009.
19. Hadjipapas A, Hillebrand A, Holliday IE, Singh KD, Barnes GR. Assessing interactions of linear and nonlinear neuronal sources using MEG beamformers: a proof of concept. Clin Neurophysiol 116: 1300-1313, 2005.
20. Hartcher-O'Brien J, Gallace A, Krings B, Koppen C, Spence C. When vision 'extinguishes' touch in neurologically-normal people: extending the Colavita visual dominance effect. Exp Brain Res 186: 643-658, 2008.
21. Kaiser J, Hertrich I, Ackermann H, Mathiak K, Lutzenberger W. Hearing lips: gamma-band activity during audiovisual speech perception. Cereb Cortex 15: 646-653, 2005.
22. Koppen C, Alsius A, Spence C. Semantic congruency and the Colavita visual dominance effect. Exp Brain Res 184: 533-546, 2008.
23. Koppen C, Spence C. Assessing the role of stimulus probability on the Colavita visual dominance effect. Neurosci Lett 418: 266-271, 2007a.
24. Koppen C, Spence C. Seeing the light: exploring the Colavita visual dominance effect. Exp Brain Res 180: 737-754, 2007b.
25. Kotter R, Wanke E. Mapping brains without coordinates. Philos Trans R Soc Lond B Biol Sci 360: 751-766, 2005.
26. Kutas M, Federmeier KD. Electrophysiology reveals semantic memory use in language comprehension. Trends Cogn Sci 4: 463-470, 2000.
27. Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT. Enhanced multisensory integration in older adults. Neurobiol Aging 27: 1155-1163, 2006.
28. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT. Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158: 405-414, 2004.
29. Laurienti PJ, Wallace MT, Maldjian JA, Susi CM, Stein BE, Burdette JH. Cross-modal sensory processing in the anterior cingulate and medial prefrontal cortices. Hum Brain Mapp 19: 213-223, 2003.
30. Macaluso E, George N, Dolan R, Spence C, Driver J. Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage 21: 725-732, 2004.
31. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Tootell RB. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA 92: 8135-8139, 1995.
32. Maravita A, Spence C, Kennett S, Driver J. Tool-use changes multimodal spatial interactions between vision and touch in normal humans. Cognition 83: B25-B34, 2002.
33. McDonald JJ, Teder-Salejarvi WA, Hillyard SA. Involuntary orienting to sound improves visual perception. Nature 407: 906-908, 2000.
34. McGurk H, MacDonald J. Hearing lips and seeing voices. Nature 264: 746-748, 1976.
35. McIntosh AR, Bookstein FL, Haxby JV, Grady CL. Spatial pattern analysis of functional brain images using partial least squares. Neuroimage 3: 143-157, 1996.
36. McIntosh AR, Lobaugh NJ. Partial least squares analysis of neuroimaging data: applications and advances. Neuroimage 23, Suppl 1: S250-S263, 2004.
37. Miller J. Divided attention: evidence for coactivation with redundant signals. Cogn Psychol 14: 247-279, 1982.
38. Molholm S, Ritter W, Javitt DC, Foxe JJ. Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 14: 452-465, 2004.
39. Molholm S, Sehatpour P, Mehta AD, Shpaner M, Gomez-Ramirez M, Ortigue S, Foxe JJ. Audio-visual multisensory integration in superior parietal lobule revealed by human intracranial recordings. J Neurophysiol 96: 721-729, 2006.
40. Moran RJ, Molholm S, Reilly RB, Foxe JJ. Changes in effective connectivity of human superior parietal lobule under multisensory and unisensory stimulation. Eur J Neurosci 27: 2303-2312, 2008.
41. Murray MM, Camen C, Gonzalez Andino SL, Bovet P, Clarke S. Rapid brain discrimination of sounds of objects. J Neurosci 26: 1293-1302, 2006.
42. Murray MM, Michel CM, Grave de Peralta R, Ortigue S, Brunet D, Gonzalez Andino S, Schnider A. Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging. Neuroimage 21: 125-135, 2004.
43. Nichols TE, Holmes AP. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum Brain Mapp 15: 1-25, 2002.
44. Nishitani N, Hari R. Viewing lip forms: cortical dynamics. Neuron 36: 1211-1220, 2002.
45. Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ. The effect of prior visual information on recognition of speech and sounds. Cereb Cortex 18: 598-609, 2008.
46. Noppeney U, Ostwald D, Werner S. Perceptual decisions formed by accumulation of audiovisual evidence in prefrontal cortex. J Neurosci 30: 7434-7446, 2010.
47. Peiffer AM, Mozolic JL, Hugenschmidt CE, Laurienti PJ. Age-related multisensory enhancement in a simple audiovisual detection task. Neuroreport 18: 1077-1081, 2007.
48. Pourtois G, de Gelder B. Semantic factors influence multisensory pairing: a transcranial magnetic stimulation study. Neuroreport 13: 1567-1573, 2002.
49. Quraan MA, Cheyne D. Reconstruction of correlated brain activity with adaptive filters in MEG. Neuroimage 49: 2387-2400, 2010.
50. Robinson S, Vrba J (editors). Functional Neuroimaging by Synthetic Aperture Magnetometry. Sendai, Japan: Tohoku Univ. Press, 1998.
51. Schneider TR, Debener S, Oostenveld R, Engel AK. Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-toauditory object priming. Neuroimage 42: 1244-1254, 2008.
52. Sekihara K, Nagarajan SS, Poeppel D, Marantz A, Miyashita Y. Reconstructing spatio-temporal activities of neural sources using an MEG vector beamformer technique. IEEE Trans Biomed Eng 48: 760-771, 2001.
53. Senkowski D, Saint-Amour D, Kelly SP, Foxe JJ. Multisensory processing of naturalistic objects in motion: a high-density electrical mapping and source estimation study. Neuroimage 36: 877-888, 2007.
54. Sinnett S, Soto-Faraco S, Spence C. The co-occurrence of multisensory competition and facilitation. Acta Psychol (Amst) 128: 153-161, 2008.
55. Sinnett S, Spence C, Soto-Faraco S. Visual dominance and attention: the Colavita effect revisited. Percept Psychophys 69: 673-686, 2007.
56. Snodgrass JG, Vanderwart M. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. J Exp Psychol Hum Learn 6: 174-215, 1980.
57. Stein BE, Burr D, Constantinidis C, Laurienti P, Meredith MA, Perrault TJ, Ramachandran R, Röder B, Rowland BA, Sathian K, Schroeder CE, Shams L, Stanford TR, Wallace MT, Yu L, Lewkowicz DJ. Semantic confusion regarding the development of multisensory integration: a practical solution. Eur J Neurosci 31: 1713-1720, 2010.
58. Talairach J, Tournoux P. Co-Planar Stereotaxic Atlas of the Human Brain, translated by Rayport M. New York: Thieme Medical, 1988.
59. Teder-Salejarvi WA, Di Russo F, McDonald JJ, Hillyard SA. Effects of spatial congruity on audio-visual multimodal integration. J Cogn Neurosci 17: 1396-1409, 2005.
60. Teder-Salejarvi WA, McDonald JJ, Di Russo F, Hillyard SA. An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res Cogn Brain Res 14: 106-114, 2002.
61. Thorpe S, Fize D, Marlot C. Speed of processing in the human visual system. Nature 381: 520-522, 1996.
62. van Atteveldt NM, Blau VC, Blomert L, Goebel R. Fmr-adaptation indicates selectivity to audiovisual content congruency in distributed clusters in human superior temporal cortex. BMC Neurosci 11: 11, 2010.
63. Van Petten C, Luka BJ. Neural localization of semantic context effects in electromagnetic and hemodynamic studies. Brain Lang 97: 279-293, 2006.
64. Van Wassenhove V, Grant KW, Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci USA 102: 1181-1186, 2005.
65. Weisberg J, van Turennout M, Martin A. A neural system for learning about object function. Cereb Cortex 17: 513-521, 2007.
66. Yuval-Greenberg S, Deouell LY. What you see is not (always) what you hear: induced gamma band responses reflect cross-modal interactions in familiar object recognition. J Neurosci 27: 1090-1096, 2007.
67. Yuval-Greenberg S, Deouell LY. The dog's meow: asymmetrical interaction in cross-modal object recognition. Exp Brain Res 193: 603-614, 2009.