Neural substrates of reliability-weighted visual-tactile multisensory integration

Frontiers in Systems Neuroscience

Volumn 4, Issue , 2010, Pages

  Beauchamp, Michael S ^a   Pasalar, Siavash ^a   Ro, Tony ^b  

^a UNIVERSITY OF TEXAS MEDICAL SCHOOL   (United States)

^b CITY COLLEGE OF NEW YORK   (United States)

Author keywords

Area mt; BOLD fMRI; Effective connectivity; Intraparietal cortex; Structural equation modeling; Weighted connections

Indexed keywords

ARTICLE; BOLD SIGNAL; CONTROLLED STUDY; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; INTRAPARIETAL SULCUS; NORMAL HUMAN; OCCIPITAL CORTEX; PARIETAL LOBE; RELIABILITY; SENSORY CORTEX; SENSORY STIMULATION; SOMATOSENSORY SYSTEM; STIMULUS RESPONSE; TASK PERFORMANCE; VISUAL CORTEX; VISUAL SYSTEM FUNCTION; VISUAL TACTILE MULTISENSORY INTEGRATION;

EID: 78649882272     PISSN: 16625137     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnsys.2010.00025     Document Type: Article

References (84)

1. Alais, D., and Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257-262.
2. Amedi, A., Jacobson, G., Hendler, T., Malach, R., and Zohary, E. (2002). Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb. Cortex 12, 1202-1212.
3. Amedi, A., Malach, R., Hendler, T., Peled, S., and Zohary, E. (2001). Visuo-haptic object-related activation in the ventral visual pathway. Nat. Neurosci. 4, 324-330.
4. Argall, B. D., Saad, Z. S., and Beauchamp, M. S. (2006). Simplified intersubject averaging on the cortical surface using SUMA. Hum. Brain Mapp. 27, 14-27.
5. Beauchamp, M. S., Cox, R. W., and DeYoe, E. A. (1997). Graded effects of spatial and featural attention on human area MT and associated motion processing areas. J. Neurophysiol. 77, 516-520.
6. Beauchamp, M. S., Laconte, S., and Yasar, N. (2009). Distributed representation of single touches in somatosensory and visual cortex. Hum. Brain Mapp. 30, 3163-3171.
7. Beauchamp, M. S., Lee, K. E., Haxby, J. V., and Martin, A. (2002). Parallel visual motion processing streams for manipulable objects and human movements. Neuron 34, 149-159.
8. Beauchamp, M. S., Lee, K. E., Haxby, J. V., and Martin, A. (2003). fMRI responses to video and point-light displays of moving humans and manipulable objects. J. Cogn. Neurosci. 15, 991-1001.
9. Beauchamp, M. S., Yasar, N. E., Kishan, N., and Ro, T. (2007). Human MST but not MT responds to tactile stimulation. J. Neurosci. 27, 8261-8267.
10. Blakemore, S. J., Bristow, D., Bird, G., Frith, C., and Ward, J. (2005). Somatosensory activations during the observation of touch and a case of vision-touch synaesthesia. Brain 128, 1571-1583.
11. Boussaoud, D., Ungerleider, L. G., and Desimone, R. (1990). Pathways for motion analysis: cortical connections of the middle superior temporal and fundus of the superior temporal visual areas in the macaque. J. Comp. Neurol. 296, 462-495.
12. Bremmer, F., Schlack, A., Shah, N. J., Zafiris, O., Kubischik, M., Hoffmann, K., Zilles, K., and Fink, G. R. (2001). Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys. Neuron 29, 287-296.
13. Buchel, C., and Friston, K. (2001). Interactions among neuronal systems assessed with functional neuroimaging. Rev. Neurol. (Paris) 157, 807-815.
14. Buchel, C., and Friston, K. J. (1997). Modulation of connectivity in visualpathways by attention: cortical interactions evaluated with structural equation modelling and fMRI. Cereb. Cortex 7, 768-778.
15. Buchel, C., Josephs, O., Rees, G., Turner, R., Frith, C. D., and Friston, K. J. (1998). The functional anatomy of attention to visual motion. A functional MRI study. Brain 121, 1281-1294.
16. Burton, H., Sinclair, R. J., and McLaren, D. G. (2004). Cortical activity to vibrotactile stimulation: an fMRI study in blind and sighted individuals. Hum. Brain Mapp. 23, 210.
17. Burton, H., Sinclair, R. J., Wingert, J. R., and Dierker, D. L. (2008). Multiple parietal operculum subdivisions in humans: tactile activation maps. Somatosens. Mot. Res. 25, 149-162.
18. Cox, R. W. (1996). AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162-173.
19. Culham, J. C., and Valyear, K. F. (2006). Human parietal cortex in action. Curr. Opin. Neurobiol. 16, 205-212.
20. Disbrow, E., Roberts, T., and Krubitzer, L. (2000). Somatotopic organization of cortical fields in the lateral sulcus of Homo sapiens: evidence for SII and PV. J. Comp. Neurol. 418, 1-21.
21. Downing, P. E., Jiang, Y., Shuman, M., and Kanwisher, N. (2001). A cortical area selective for visual processing of the human body. Science 293, 2470-2473.
22. Eickhoff, S. B., Grefkes, C., Fink, G. R., and Zilles, K. (2008). Functional lateralization of face, hand, and trunk representation in anatomically defined human somatosensory areas. Cereb. Cortex. 18, 2820-2830.
23. Eickhoff, S. B., Grefkes, C., Zilles, K., and Fink, G. R. (2007). The somatotopic organization of cytoarchitectonic areas on the human parietal operculum. Cereb. Cortex 17, 1800-1811.
24. Ernst, M. O., and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429-433.
25. Farne, A., Serino, A., and Ladavas, E. (2007). Dynamic size-change of peri-hand space following tool-use: determinants and spatial characteristics revealed through cross-modal extinction. Cortex 43, 436-443.
26. Felleman, D. J., and Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1-47.
27. Fetsch, C. R., Deangelis, G. C., and Angelaki, D. E. (2010) Neural corrrelates of dynamic sensory cue re-weighting in macaque area MSTd. Comput. Syst. Neurosci. doi: 10.3389/conf.fnins.2010.03.00299.
28. Fetsch, C. R., Turner, A. H., DeAngelis, G. C., and Angelaki, D. E. (2009). Dynamic reweighting of visual and vestibular cues during self-motion perception. J. Neurosci. 29, 15601-15612.
29. Fischl, B., Sereno, M. I., and Dale, A. M. (1999a). Cortical surface-based analysis. II: inflation, flattening, and a surface-based coordinate system. Neuroimage 9, 195-207.
30. Fischl, B., Sereno, M. I., Tootell, R. B., and Dale, A. M. (1999b). High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 8, 272-284.
31. Francis, S. T., Kelly, E. F., Bowtell, R., Dunseath, W. J., Folger, S. E., and McGlone, F. (2000). fMRI of the responses to vibratory stimulation of digit tips. Neuroimage 11, 188-202.
32. Genovese, C. R., Lazar, N. A., and Nichols, T. (2002). Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage 15, 870-878.
33. Gizewski, E. R., Koeze, O., Uffmann, K., de Greiff, A., Ladd, M. E., and Forsting, M. (2005). Cerebral activation using a MR-compatible piezoelectric actuator with adjustable vibration frequencies and in vivo wave propagation control. Neuroimage 24, 723-730.
34. Grefkes, C., and Fink, G. R. (2005). The functional organization of the intraparietal sulcus in humans and monkeys. J. Anat. 207, 3-17.
35. Hagen, M. C., Franzen, O., McGlone, F., Essick, G., Dancer, C., and Pardo, J. V. (2002). Tactile motion activates the human middle temporal/V5 (MT/V5) complex. Eur. J. Neurosci. 16, 957-964.
36. Haggard, P., Christakou, A., and Serino, A. (2007). Viewing the body modulates tactile receptive fields. Exp. Brain Res. 180, 187-193.
37. Helbig, H. B., and Ernst, M. O. (2007). Optimal integration of shape information from vision and touch. Exp. Brain Res. 179, 595-606.
38. Helbig, H. B., and Ernst, M. O. (2008). Visual-haptic cue weighting is independent of modality-specific attention. J. Vis. 8, 1-16.
39. Holmes, N. P., Calvert, G. A., and Spence, C. (2007). Tool use changes multisensory interactions in seconds: evidence from the crossmodal congruency task. Exp. Brain Res. 183, 465-476.
40. Horwitz, B., McIntosh, A. R., Haxby, J. V., and Grady, C. L. (1995). Network analysis of brain cognitive function using metabolic and blood flow data. Behav. Brain Res. 66, 187-193.
41. Igarashi, Y., Kimura, Y., Spence, C., and Ichihara, S. (2008). The selective effect of the image of a hand on visuotactile interactions as assessed by performance on the crossmodal congruency task. Exp. Brain Res. 184, 31-38.
42. James, T. W., Humphrey, G. K., Gati, J. S., Servos, P., Menon, R. S., and Goodale, M. A. (2002). Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40, 1706-1714.
43. Johnson, R. M., Burton, P. C., and Ro, T. (2006). Visually induced feelings of touch. Brain Res. 16, 398-406.
44. Kastner, S., De Weerd, P., Desimone, R., and Ungerleider, L. G. (1998). Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI [see comments]. Science 282, 108-111.
45. Kennett, S., Taylor-Clarke, M., and Haggard, P. (2001). Noninformative vision improves the spatial resolution of touch in humans. Curr. Biol. 11, 1188-1191.
46. Keysers, C., Wicker, B., Gazzola, V., Anton, J. L., Fogassi, L., and Gallese, V. (2004). A touching sight: SII/PV activation during the observation and experience of touch. Neuron 42, 335-346.
47. Levin, N., Dumoulin, S. O., Winawer, J., Dougherty, R. F., and Wandell, B. A. (2010). Cortical maps and white matter tracts following long period of visual deprivation and retinal image restoration. Neuron 65, 21-31.
48. Lewis, J. W., and Van Essen, D. C. (2000). Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J. Comp. Neurol. 428, 112-137.
49. Ma, W. J., Beck, J. M., Latham, P. E., and Pouget, A. (2006). Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432-1438.
50. Ma, W. J., and Pouget, A. (2008). Linking neurons to behavior in multisensory perception: a computational review. Exp. Brain Res. 1242, 4-12.
51. Ma, W. J., Zhou, X., Ross, L. A., Foxe, J. J., and Parra, L. C. (2009). Lip-reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space. PLoS ONE 4, e4638. doi: 10.1371/journal.pone.0004638.
52. Makin, T. R., Holmes, N. P., and Zohary, E. (2007). Is that near my hand? Multisensory representation of peripersonal space in human intraparietal sulcus. J. Neurosci. 27, 731-740.
53. Maravita, A., Spence, C., and Driver, J. (2003). Multisensory integration and the body schema: close to hand and within reach. Curr. Biol. 13:R531-539.
54. Mazziotta, J., Toga, A., Evans, A., Fox, P., Lancaster, J., Zilles, K., Woods, R., Paus, T., Simpson, G., Pike, B., Holmes, C., Collins, L., Thompson, P., MacDonald, D., Iacoboni, M., Schormann, T., Amunts, K., Palomero-Gallagher, N., Geyer, S., Parsons, L., Narr, K., Kabani, N., Le Goualher, G., Boomsma, D., Cannon, T., Kawashima, R., and Mazoyer, B. (2001). A probabilistic atlas and reference system for the human brain: international consortium for brain mapping (ICBM). Philos. Trans. R. Soc. Lond., B, Biol. Sci. 356, 1293-1322.
55. McGlone, F., Kelly, E. F., Trulsson, M., Francis, S. T., Westling, G., and Bowtell, R. (2002). Functional neuroimaging studies of human somatosensory cortex. Behav. Brain Res. 135, 147-158.
56. McIntosh, A. R., Grady, C. L., Ungerleider, L. G., Haxby, J. V., Rapoport, S. I., and Horwitz, B. (1994). Network analysis of cortical visual pathways mapped with PET. J. Neurosci. 14, 655-666.
57. Morgan, M. L., Deangelis, G. C., and Angelaki, D. E. (2008). Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 59, 662-673.
58. Nakashita, S., Saito, D. N., Kochiyama, T., Honda, M., Tanabe, H. C., and Sadato, N. (2008). Tactile-visual integration in the posterior parietal cortex: a functional magnetic resonance imaging study. Brain Res. Bull. 75, 513-525.
59. Norman, K. A., Polyn, S. M., Detre, G. J., and Haxby, J. V. (2006). Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. (Regul. Ed.) 10, 424-430.
60. Pasalar, S., Ro, T., Beauchamp, M. S. (2010). TMS of posterior parietal cortex disrupts visual tactile multisensory integration. Eur. J. Neurosci. 31, 1783-1790.
61. Pelphrey, K. A., Morris, J. P., Michelich, C. R., Allison, T., and McCarthy, G. (2005). Functional anatomy of biological motion perception in posterior temporal cortex: an fMRI study of eye, mouth and hand movements. Cereb. Cortex. 15, 1866-1876.
62. Peltier, S., Stilla, R., Mariola, E., LaConte, S., Hu, X., and Sathian, K. (2007). Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception. Neuropsychologia 45, 476-483.
63. Porter, T., and Duff, T. (1984). Compositing Digital Images. Comput. Graph. (ACM) 18, 253-259.
64. Prather, S. C., Votaw, J. R., and Sathian, K. (2004). Task-specific recruitment of dorsal and ventral visual areas during tactile perception. Neuropsychologia 42, 1079-1087.
65. Ro, T., Farne, A., Johnson, R. M., Wedeen, V., Chu, Z., Wang, Z. J., Hunter, J. V., and Beauchamp, M. S. (2007). Feeling sounds after a thalamic lesion. Ann. Neurol. 62, 433-441.
66. Ro, T., and Rafal, R. (2006). Visual restoration in cortical blindness: insights from natural and TMS-induced blindsight. Neuropsychol. Rehabil. 16, 377-396.
67. Ro, T., Wallace, R., Hagedorn, J., Farne, A., and Pienkos, E. (2004). Visual enhancing of tactile perception in the posterior parietal cortex. J. Cogn. Neurosci. 16, 24-30.
68. Ruben, J., Schwiemann, J., Deuchert, M., Meyer, R., Krause, T., Curio, G., Villringer, K., Kurth, R., and Villringer, A. (2001). Somatotopic organization of human secondary somatosensory cortex. Cereb. Cortex 11, 463-473.
69. Rushworth, M. F., Behrens, T. E., and Johansen-Berg, H. (2006). Connection patterns distinguish 3 regions of human parietal cortex. Cereb. Cortex 16, 1418-1430.
70. Saenz, M., Buracas, G. T., and Boynton, G. M. (2002). Global effects of feature-based attention in human visual cortex. Nat. Neurosci. 5, 631-632.
71. Saito, D. N., Okada, T., Morita, Y., Yonekura, Y., and Sadato, N. (2003). Tactile-visual cross-modal shape matching: a functional MRI study. Brain Res. Cogn. Brain Res. 17, 14-25.
72. Schaefer, M., Flor, H., Heinze, H. J., and Rotte, M. (2006). Dynamic modulation of the primary somatosensory cortex during seeing and feeling a touched hand. Neuroimage 29, 587-592.
73. Schaefer, M., Heinze, H. J., and Rotte, M. (2005). Seeing the hand being touched modulates the primary somatosensory cortex. Neuroreport 16, 1101-1105.
74. Sherman, S. M. (2007). The thalamus is more than just a relay. Curr. Opin. Neurobiol. 17, 417-422.
75. Simmons, W. K., Bellgowan, P. S., and Martin, A. (2007). Measuring selectivity in fMRI data. Nat. Neurosci. 10, 4-5.
76. Stein, J. L., Wiedholz, L. M., Bassett, D. S., Weinberger, D. R., Zink, C. F., Mattay, V. S., and Meyer-Lindenberg, A. (2007). A validated network of effective amygdala connectivity. Neuroimage 36, 736-745.
77. Stilla, R., and Sathian, K. (2008). Selective visuo-haptic processing of shape and texture. Hum. Brain Mapp. 29, 1123-1138.
78. Summers, I. R., Francis, S. T., Bowtell, R. W., McGlone, F. P., and Clemence, M. (2009). A functional-magnetic-resonance-imaging investigation of cortical activation from moving vibrotactile stimuli on the fingertip. J. Acoust. Soc. Am. 125, 1033-1039.
79. Tal, N., and Amedi, A. (2009). Multisensory visual-tactile object related network in humans: insights gained using a novel crossmodal adaptation approach. Exp. Brain Res. 198, 165-182.
80. Tipper, S., Lloyd, D., Shorland, B., Dancer, C., Howard, L., and McGlone, F. (1998). Vision influences tactile perception without proprioceptive orienting. Neuroreport 9, 1741-1744.
81. Tootell, R. B., Reppas, J. B., Kwong, K. K., Malach, R., Born, R. T., Brady, T. J., Rosen, B. R., and Belliveau, J. W. (1995). Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 15, 3215-3230.
82. Vul, E., Harris, C., Winkielman, P., and Pashler, H. (2009). Puzzlingly high correlations in fMRI studies of emotion, personality and social cognition. Perspect. Psychol. Sci. 4, 274-290.
83. Wheaton, K. J., Thompson, J. C., Syngeniotis, A., Abbott, D. F., and Puce, A. (2004). Viewing the motion of human body parts activates different regions of premotor, temporal, and parietal cortex. Neuroimage 22, 277-288.
84. Witten, I. B., and Knudsen, E. I. (2005). Why seeing is believing: merging auditory and visual worlds. Neuron 48, 489-496.