Multisensory object representation. Insights from studies of vision and touch

Progress in Brain Research

Volumn 191, Issue , 2011, Pages 165-176

  Lacey, Simon ^a   Sathian, K ^a,b,c  

^a EMORY UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b EMORY UNIVERSITY   (United States)

^c ATLANTA VA MEDICAL CENTER   (United States)

Author keywords

Cross modal; FMRI; Haptic; Visual imagery

Indexed keywords

BOOK; HAPTIC OBJECT RECOGNITION; HUMAN; MULTISENSORY OBJECT RECOGNITION; PATTERN RECOGNITION; PRIORITY JOURNAL; PROPRIOCEPTION; SPATIAL DISCRIMINATION; TOUCH; VISION; VISUAL CORTEX; VISUAL DISCRIMINATION; VISUAL MEMORY; VISUAL OBJECT RECOGNITION;

EID: 79960069996     PISSN: 00796123     EISSN: 18757855     Source Type: Book Series    
DOI: 10.1016/B978-0-444-53752-2.00006-0     Document Type: Chapter

References (86)

1. Allen H.A., Humphreys G.W. Direct tactile stimulation of dorsal occipito-temporal cortex in a visual agnosic. Current Biology 2009, 19:1044-1049.
2. Amedi A., Jacobson G., Hendler T., Malach R., Zohary E. Convergence of visual and tactile shape processing in the human lateral occipital complex. Cerebral Cortex 2002, 12:1202-1212.
3. Amedi A., Malach R., Hendler T., Peled S., Zohary E. Visuo-haptic object-related activation in the ventral visual pathway. Nature Neuroscience 2001, 4:324-330.
4. Amedi A., Stern W.M., Camprodon J.A., Bermpohl F., Merabet L., Rotman S., et al. Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neuroscience 2007, 10:687-689.
5. Amedi A., von Kriegstein K., van Atteveldt N.M., Beauchamp M.S., Naumer M.J. Functional imaging of human crossmodal identification and object recognition. Experimental Brain Research 2005, 166:559-571.
6. Arno P., De Volder A.G., Vanlierde A., Wanet-Defalque M.-C., Streel E., Robert A., et al. Occipital activation by pattern recognition in the early blind using auditory substitution for vision. Neuroimage 2001, 13:632-645.
7. Bar M. The proactive brain: Using analogies and associations to generate predictions. Trends in Cognitive Sciences 2007, 11:280-289.
8. Berryman L.J., Yau J.M., Hsiao S.S. Representation of object size in the somatosensory system. Journal of Neurophysiology 2006, 96:27-39.
9. Biederman I., Cooper E.E. Size invariance in visual object priming. Journal of Experimental Psychology: Human Perception and Performance 1992, 18:121-133.
10. Blajenkova O., Kozhevnikov M., Motes M.A. Object-spatial imagery: A new self-report imagery questionnaire. Applied Cognitive Psychology 2006, 20:239-263.
11. Bülthoff I., Newell F.N. The role of familiarity in the recognition of static and dynamic objects. Progress in Brain Research 2006, 154:315-325.
12. Cant J.S., Arnott S.R., Goodale M.A. fMR-adaptation reveals separate processing regions for the perception of form and texture in the human ventral stream. Experimental Brain Research 2009, 192:391-405.
13. Cant J.S., Goodale M.A. Attention to form or surface properties modulates different regions of human occipitotemporal cortex. Cerebral Cortex 2007, 17:713-731.
14. Combe E., Wexler M. Observer movement and size constancy. Psychological Science 2010, 21:667-675.
15. Craddock M., Lawson R. Repetition priming and the haptic recognition of familiar and unfamiliar objects. Perception and Psychophysics 2008, 70:1350-1365.
16. Craddock M., Lawson R. Do left and right matter for haptic recognition of familiar objects?. Perception 2009, 38:1355-1376.
17. Craddock M., Lawson R. The effect of size changes on haptic object recognition. Attention, Perception and Psychophysics 2009, 71:910-923.
18. Craddock M., Lawson R. Size-sensitive perceptual representations underlie visual and haptic object recognition. PLoS ONE 2009, 4:e8009. 10.1371/journal.pone.0008009.
19. Craddock M., Lawson R. The effects of temporal delay and orientation on haptic object recognition. Attention, Perception and Psychophysics 2010, 72:1975-1980.
20. De Volder A.G., Toyama H., Kimura Y., Kiyosawa M., Nakano H., Vanlierde A., et al. Auditory triggered mental imagery of shape involves visual association areas in early blind humans. Neuroimage 2001, 14:129-139.
21. Deshpande G., Hu X., Lacey S., Stilla R., Sathian K. Object familiarity modulates effective connectivity during haptic shape perception. Neuroimage 2010, 49:1991-2000.
22. Deshpande G., Hu X., Stilla R., Sathian K. Effective connectivity during haptic perception: A study using Granger causality analysis of functional magnetic resonance imaging data. Neuroimage 2008, 40:1807-1814.
23. Eger E., Ashburner J., Haynes J.-D., Dolan R.J., Rees G. fMRI activity patterns in human LOC carry information about object exemplars within category. Journal of Cognitive Neuroscience 2008, 20:356-370.
24. Eger E., Kell C.A., Kleinschmidt A. Graded size-sensitivity of object-exemplar-evoked activity patterns within human LOC regions. Journal of Neurophysiology 2008, 100:2038-2047.
25. Ewbank M.P., Schluppeck D., Andrews T.J. fMR-adaptation reveals a distributed representation of inanimate objects and places in human visual cortex. Neuroimage 2005, 28:268-279.
26. Feinberg T.E., Rothi L.J., Heilman K.M. Multimodal agnosia after unilateral left hemisphere lesion. Neurology 1986, 36:864-867.
27. Gauthier I., Hayward W.G., Tarr M.J., Anderson A.W., Skudlarski P., Gore J.C. BOLD activity during mental rotation and view-dependent object recognition. Neuron 2002, 34:161-171.
28. Goodale M.A., Milner A.D. Separate visual pathways for perception and action. Trends in Neuroscience 1992, 15:20-25.
29. Grefkes C., Geyer S., Schormann T., Roland P., Zilles K. Human somatosensory area 2: Observer-independent cytoarchitectonic mapping, interindividual variability, and population map. Neuroimage 2001, 14:617-631.
30. Grill-Spector K., Henson R., Martin A. Repetition and the brain: Neural models of stimulus-specific effects. Trends in Cognitive Sciences 2006, 10:14-23.
31. Grill-Spector K., Kushnir T., Edelman S., Avidan G., Itzchak Y., Malach R. Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron 1999, 24:187-203.
32. Iriki A., Tanaka M., Iwamura Y. Attention-induced neuronal activity in the monkey somatosensory cortex revealed by pupillometrics. Neuroscience Research 1996, 25:173-181.
33. James T.W., Humphrey G.K., Gati J.S., Menon R.S., Goodale M.A. Differential effects of view on object-driven activation in dorsal and ventral streams. Neuron 2002, 35:793-801.
34. James T.W., Humphrey G.K., Gati J.S., Servos P., Menon R.S., Goodale M.A. Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 2002, 40:1706-1714.
35. James T.W., James K.H., Humphrey G.K., Goodale M.A. Do visual and tactile object representations share the same neural substrate?. Touch and blindness: Psychology and neuroscience 2006, 139-155. Lawrence Erlbaum Associates, Mahwah, NJ. M.A. Heller, S. Ballesteros (Eds.).
36. Jolicoeur P. A size-congruency effect in memory for visual shape. Memory and Cognition 1987, 15:531-543.
37. Klatzky R.L., Lederman S.J. Identifying objects from a haptic glance. Perception and Psychophysics 1995, 57:1111-1123.
38. Klatzky R.L., Lederman S.J., Metzger V.A. Identifying objects by touch: An 'expert system'. Perception and Psychophysics 1985, 37:299-302.
39. Klatzky R.L., Lederman S.J., Reed C.L. There's more to touch than meets the eye: The salience of object attributes for haptics with and without vision. Journal of Experimental Psychology: General 1987, 116:356-369.
40. Konkle T., Oliva A. Canonical visual size for real-world objects. Journal of Experimental Psychology: Human Perception and Performance 2010, 37:23-37.
41. Kozhevnikov M., Hegarty M., Mayer R.E. Revising the visualiser-verbaliser dimension: Evidence for two types of visualisers. Cognition and Instruction 2002, 20:47-77.
42. Kozhevnikov M., Kosslyn S.M., Shephard J. Spatial versus object visualisers: A new characterisation of cognitive style. Memory and Cognition 2005, 33:710-726.
43. Krekelberg B., Boynton G.M., van Wezel R.J.A. Adaptation: From single cells to BOLD signals. Trends in Neurosciences 2006, 29:250-256.
44. Lacey S., Flueckiger P., Stilla R., Lava M., Sathian K. Object familiarity modulates the relationship between visual object imagery and haptic shape perception. Neuroimage 2010, 49:1977-1990.
45. Lacey S., Hall J., Sathian K. Are surface properties integrated into visuo-haptic object representations?. The European Journal of Neuroscience 2010, 31:1882-1888.
46. Lacey S., Lin J.B., Sathian K. Object and spatial imagery dimensions in visuo-haptic representations 2011, doi: 10.1007/s00221-011-2623-1 (in press).
47. Lacey S., Pappas M., Kreps A., Lee K., Sathian K. Perceptual learning of view-independence in visuo-haptic object representations. Experimental Brain Research 2009, 198:329-337.
48. Lacey S., Peters A., Sathian K. Cross-modal object representation is viewpoint-independent. PLoS ONE 2007, 2(9):e890. 10.1371/journal.pone0000890.
49. Lacey S., Tal N., Amedi A., Sathian K. A putative model of multisensory object representation. Brain Topography 2009, 21:269-274.
50. Lawson R. A comparison of the effects of depth rotation on visual and haptic three-dimensional object recognition. Journal of Experimental Psychology: Human Perception and Performance 2009, 35:911-930.
51. Lederman S.J., Klatzky R.L. Hand movements: A window into haptic object recognition. Cognitive Psychology 1987, 19:342-368.
52. Lucan J.N., Foxe J.J., Gomez-Ramirez M., Sathian K., Molholm S. Tactile shape discrimination recruits human lateral occipital complex during early perceptual processing. Human Brain Mapping 2010, 31:1813-1821.
53. Malach R., Reppas J.B., Benson R.R., Kwong K.K., Jiang H., Kennedy W.A., et al. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proceedings of the National Academy of Science of the United States of America 1995, 92:8135-8139.
54. Mur M., Ruff D.A., Bodurka J., Bandettini P.A., Kriegeskorte N. Face-identity change activation outside the face system: " Release from adaptation" may not always indicate neuronal selectivity. Cerebral Cortex 2010, 20:2027-2042.
55. Newell F.N., Ernst M.O., Tjan B.S., Bülthoff H.H. View dependence in visual and haptic object recognition. Psychological Science 2001, 12:37-42.
56. Newman S.D., Klatzky R.L., Lederman S.J., Just M.A. Imagining material versus geometric properties of objects: An fMRI study. Cognitive Brain Research 2005, 23:235-246.
57. Nicholson K.G., Humphrey G.K. The effect of colour congruency on shape discriminations of novel objects. Perception 2003, 32:339-353.
58. Palmer S.E., Rosch E., Chase P. Canonical perspective and the perception of objects. Attention and performance IX 1981, 135-151. Lawrence Erlbaum Associates, Hillsdale, NJ. J. Long, A. Baddeley (Eds.).
59. Pascual-Leone A., Amedi A., Fregni F., Merabet L.B. The plastic human brain. Annual Review of Neuroscience 2005, 28:377-401.
60. Pasqualotto A., Finucane C., Newell F.N. Visual and haptic representations of scenes are updated with observer movement. Experimental Brain Research 2005, 166:481-488.
61. Pasqualotto A., Newell F.N. The role of visual experience on the representation and updating of novel haptic scenes. Brain and Cognition 2007, 65:184-194.
62. Peissig J.J., Tarr M.J. Visual object recognition: Do we know more now than we did 20 years ago?. Annual Review of Psychology 2007, 58:75-96.
63. Peltier S., Stilla R., Mariola E., LaConte S., Hu X., Sathian K. Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception. Neuropsychologia 2007, 45:476-483.
64. Pourtois G., Schwarz S., Spiridon M., Martuzzi R., Vuilleumier P. Object representations for multiple visual categories overlap in lateral occipital and medial fusiform cortex. Cerebral Cortex 2009, 19:1806-1819.
65. Prather S.C., Votaw J.R., Sathian K. Task-specific recruitment of dorsal and ventral visual areas during tactile perception. Neuropsychologia 2004, 42:1079-1087.
66. Reed C.L., Caselli R.J., Farah M.J. Tactile agnosia: Underlying impairment and implications for normal tactile object recognition. Brain 1996, 119:875-888.
67. Renier L., Collignon O., Poirier C., Tranduy D., Vanlierde A., Bol A., et al. Cross modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 2005, 26:573-580.
68. Renier L., Collignon O., Tranduy D., Poirier C., Vanlierde A., Veraart C., et al. Visual cortex activation in early blind and sighted subjects using an auditory visual substitution device to perceive depth. Neuroimage 2004, 22:S1.
69. Rice N.J., Valyear K.F., Goodale M.A., Milner A.D., Culham J.C. Orientation sensitivity to graspable objects: An fMRI adaptation study. Neuroimage 2007, 36:T87-T93.
70. Riesenhuber M., Poggio T. Hierarchical models of object recognition in cortex. Nature Neuroscience 1999, 2:1019-1025.
71. Sathian K. Visual cortical activity during tactile perception in the sighted and the visually deprived. Developmental Psychobiology 2005, 46:279-286.
72. Sathian K., Lacey S. Journeying beyond classical somatosensory cortex. Canadian Journal of Experimental Psychology 2007, 61:254-264.
73. Sathian K., Lacey S., Stilla R., Gibson G.O., Deshpande G., Hu X., LaConte S., Glielmi C. Dual pathways for haptic and visual perception of spatial and texture information. NeuroImage 2011, doi: 10.1016/j.neuroimage.2011.05.011 (in press).
74. Sathian K., Zangaladze A., Hoffman J.M., Grafton S.T. Feeling with the mind's eye. Neuroreport 1997, 8:3877-3881.
75. Sawamura H., Georgieva S., Vogels R., Vanduffel W., Orban G.A. Using functional magnetic resonance imaging to assess adaptation and size invariance of shape processing by humans and monkeys. The Journal of Neuroscience 2005, 25:4294-4306.
76. Silvanto J., Schwarzkopf D.S., Gilaie-Dotan S., Rees G. Differing causal roles for lateral occipital complex and occipital face area in invariant shape recognition. The European Journal of Neuroscience 2010, 32:165-171.
77. Stilla R., Sathian K. Selective visuo-haptic processing of shape and texture. Human Brain Mapping 2008, 29:1123-1138.
78. Stoesz M., Zhang M., Weisser V.D., Prather S.C., Mao H., Sathian K. Neural networks active during tactile form perception: Common and differential activity during macrospatial and microspatial tasks. International Journal of Psychophysiology 2003, 50:41-49.
79. Tal N., Amedi A. Multisensory visual-tactile object related network in humans: Insights gained using a novel crossmodal adaptation approach. Experimental Brain Research 2009, 198:165-182.
80. Tarr M.J., Pinker S. Mental rotation and orientation dependence in shape recognition. Cognitive Psychology 1989, 21:233-282.
81. Ungerleider L.G., Mishkin M. Two cortical visual systems. Analysis of visual behavior 1982, 549-586. MIT Press, Cambridge, MA. D.J. Ingle, M.A. Goodale, R.J.W. Mansfield (Eds.).
82. Uttl B., Graf P., Siegenthaler A.L. Influence of object size on baseline identification, priming, and explicit memory. Scandinavian Journal of Psychology 2007, 48:281-288.
83. Valyear K.F., Culham J.C., Sharif N., Westwood D., Goodale M.A. A double dissociation between sensitivity to changes in object identity and object orientation in the ventral and dorsal streams: A human fMRI study. Neuropsychologia 2006, 44:218-228.
84. Woods A.T., Moore A., Newell F.N. Canonical views in haptic object perception. Perception 2008, 37:1867-1878.
85. Zhang M., Weisser V.D., Stilla R., Prather S.C., Sathian K. Multisensory cortical processing of object shape and its relation to mental imagery. Cognitive, Affective and Behavioral Neuroscience 2004, 4:251-259.
86. Zhou Y.-D., Fuster J.M. Neuronal activity of somatosensory cortex in a cross-modal (visuo-haptic) memory task. Experimental Brain Research 1997, 116:551-555.