Representing 3D space in working memory: Spatial images from vision, hearing, touch, and language

Multisensory Imagery

Volumn , Issue , 2014, Pages 131-155

  Loomis, Jack M ^a   Klatzky, Roberta L ^b   Giudice, Nicholas A ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c University of Maine   (United States)

Author keywords

Amodal; Functional equivalence; Hearing; Language; Spatial image; Spatial updating; Touch; Vision; Working memory

Indexed keywords

EID: 84885404602     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4614-5879-1_8     Document Type: Chapter

References (108)

1. Amorim MA, Glasauer S, Corpinot K, Berthoz A (1997) Updating an object's orientation and location during nonvisual navigation: a comparison between two processing modes. Atten Percept Psychophys 59:404-418
2. Andersen RA (1997) Multimodal integration for the representation of space in the posterior parietal cortex. Philos Trans R Soc Lond B Biol Sci 352:1421-1428
3. Ashmead DH, DeFord LD, Northington A (1995) Contribution of listeners' approaching motion to auditory distance perception. J Exp Psychol Hum Percept Perform 21:239-256
4. Auvray M, Myin E (2009) Perception with compensatory devices: from sensory substitution to sensorimotor extension. Cogn Sci 33:1036-1058
5. Avraamides MN (2003) Spatial updating of environments described in texts. Cogn Psychol 47:402-431
6. Avraamides MN, Kelly JW (2008) Multiple systems of spatial memory and action. Cogn Process 9:93-106
7. Avraamides MN, Kelly JW (2010) Multiple systems of spatial memory: evidence from described scenes. J Exp Psychol Learn Mem Cogn 36:635-645
8. Avraamides MN, Loomis JM, Klatzky RL, Golledge RG (2004) Functional equivalence of spatial representations derived from vision and language: evidence from allocentric judgments. J Exp Psychol Learn Mem Cogn 30:801-814
9. Bennett CR, Giudice NA, Klatzky RL, Loomis JM (2011) Spatial images developed through extended touch: comparing updating performance between haptic and visual learning. Poster presented at the 52nd Annual Psychonomics Society Meeting, Seattle, WA, November 2011
10. Blajenkova O, Kozhevnikov M, Motes M (2006) Object-spatial imagery: a new self-report imagery questionnaire. Appl Cogn Psychol 20:239-263
11. Blazhenkova O, Kozhevnikov M (2010) Visual-object ability: a new dimension of non-verbal intelligence. Cognition 117:276-301
12. Bryant DJ (1997) Representing space in language and perception. Mind Lang 12:239-264
13. Bryant DJ, Tversky B, Franklin N (1992) Internal and external spatial frameworks for representing described scenes. J Mem Lang 31:74-98
14. Burgess N (2006) Spatial memory: how egocentric and allocentric combine. Trends Cogn Sci 10:551-557
15. Burgess N (2008) Spatial cognition and the brain. Ann N Y Acad Sci 1124:77-97
16. Byrne P, Becker S, Burgess N (2007) Remembering the past and imagining the future: a neural model of spatial memory and imagery. Psychol Rev 114:340-375
17. Cattaneo Z, Vecchi T (2011) Blind Vision: The Neuroscience of Visual Impairment. MIT Press, Cambridge, M.A.
18. Cavanna AE, Trimble MR (2006) The precuneus: a review of its functional anatomy and behavioral correlates. Brain 129:564-583
19. Chang SW, Papadimitriou C, Snyder LH (2009) Using a compound gainfield to compute a reach plan. Neuron 64:744-755
20. Chebat DR, Schneider F, Kupers R, Ptito M (2011) Navigation with a sensory substitution device in congenitally blind individuals. Neuroreport 22:342-347
21. Denis M, Cocude M (1989) Scanning visual images generated from verbal descriptions. Eur J Cogn Psychol 1:293-307
22. De Vega M, Rodrigo MJ (2001) Updating spatial layouts mediated by pointing and labelling under physical and imaginary rotation. Eur J Cogn Psychol 13:369-393
23. Easton RD, Sholl MJ (1995) Object-array structure, frames of reference, and retrieval of spatial knowledge. J Exp Psychol Learn Mem Cogn 21:483-500
24. Farah MJ, Hammond KM, Levine DN, Calvanio R (1988) Visual and spatial mental imagery: dissociable systems of representation. Cogn Psychol 20:439-462
25. Farrell MJ, Robertson IH (1998) Mental rotation and the automatic updating of body-centered spatial relationships. J Exp Psychol Learn Mem Cogn 24:227-233
26. Farrell MJ, Thomson JA (1999) On-line updating of spatial information during locomotion without vision. J Mot Behav 3:39-53
27. Fernandez-Ruiz J, Goltz HH, DeSouza JFX, Vilis T, Crawford JD (2007) Human parietal "reach region" primarily encodes intrinsic visual direction, not extrinsic movement direction, in a visual-motor dissociation task. Cereb Cortex 17:2283-2292
28. Franklin N, Tversky B (1990) Searching imagined environments. J Exp Psychol Gen 119:63-76
29. Fukusima SS, Loomis JM, Da Silva JA (1997) Visual perception of egocentric distance as assessed by triangulation. J Exp Psychol Hum Percept Perform 23:86-100
30. Giudice NA, Betty MR, Loomis JM (2011) Functional equivalence of spatial images from touch and vision: evidence from spatial updating in blind and sighted individuals. J Exp Psychol Learn Mem Cogn 37:621-634
31. Giudice NA, Klatzky RL, Loomis JM (2009) Evidence for amodal representations after bimodal learning: integration of haptic-visual layouts into a common spatial image. Spat Cogn Comput 9:287-304
32. Giudice NA, Klatzky RL, Bennett CR, Loomis JM (in press) Combining locations from working memory and long-term memory into a common spatial image. Spat Cogn Comput
33. Giudice NA, Legge GE (2008) Blind navigation and the role of technology. In: Helal A, Mokhtari M, Abdulrazak B (eds) Engineering Handbook of Smart Technology for Aging, Disability, and Independence. Wiley, New York
34. Gordon MS, Rosenblum LD (2004) Perception of sound-obstructing surfaces using body-scaled judgments. Ecol Psychol 16:87-113
35. Harrison AM (2007) Reversal of the Alignment Effect: Influence of Visualization and Spatial Set Size. In: Proceedings of the 29th Annual Conference of the Cognitive Science Society. Curran Associates, Red Hook, NY
36. Hegarty M, Kozhevnikov M (1999) Types of visual-spatial representations and mathematical problem solving. J Educ Psychol 91:684-689
37. Hodgson E, Waller D (2006) Lack of set size effects in spatial updating: evidence for offline updating. J Exp Psychol Learn Mem Cogn 32:854-866
38. Hollins M, Kelley EK (1988) Spatial updating in blind and sighted people. Percept Psychophys 43:380-388
39. Horn DL, Loomis JM (2004) Spatial updating of targets in front and behind. Paidéia 14:75-81
40. Hughes B (2001) Active artificial echolocation and the nonvisual perception of aperture passability. Hum Mov Sci 20:371-400
41. Huttenlocher J, Hedges LV, Duncan S (1991) Categories and particulars: prototype effects in estimating spatial location. Psychol Rev 98:352-376
42. Kelly JW, Avraamides M, Loomis JM (2007) Sensorimotor alignment effects in the learning environment and in novel environments. J Exp Psychol Learn Mem Cogn 33:1092-1107
43. Klatzky RL, Lippa Y, Loomis JM, Golledge RG (2003) Encoding, learning, and spatial updating of multiple object locations specified by 3-D sound, spatial language, and vision. Exp Brain Res 149:48-61
44. Klatzky RL, Loomis JM, Beall AC, Chance SS, Golledge RG (1998) Spatial updating of self-position and orientation during real, imagined, and virtual locomotion. Psychol Sci 9:293-298
45. Knauff M (2009) A neuro-cognitive theory of deductive relational reasoning with mental models and visual images. Spat Cogn Comput 9:109-137
46. Koch C (2003) The Quest for Consciousness. Roberts and Co., Englewood, CO
47. Kosslyn SM, Thompson WL (2003) When is early visual cortex activated during visual mental imagery? Psychol Bull 129:723-746
48. Kozhevnikov M, Hegarty M, Mayer R (2002) Revising the cognitive style dimension: evidence for two types of visualizers. Cogn Instr 20:47-77
49. Kozhevnikov M, Kosslyn S, Shephard J (2005) Spatial versus object visualizers: a new characterization of visual cognitive style. Mem Cognit 33:710-726
50. Lacey S, Lin JB, Sathian K (2011) Object and spatial imagery dimensions in visuo-haptic representations. Exp Brain Res 213:267-273
51. Lehar S (2003) The World in your Head: A Gestalt View of the Mechanism of Conscious experience. Erlbaum, Mahwah, NJ
52. Levesque V (2009) Virtual display of tactile graphics and Braille by lateral skin deformation. Unpublished manuscript, Department of Electrical and Computer Engineering, McGill University, Montreal, Canada. Available online at http://vlevesque.com/papers/Levesque-PhD.pdf
53. Levine M, Jankovic IN, Palij M (1982) Principles of spatial problem solving. J Exp Psychol Gen 111:157-175
54. Lyon DR, Gunzelmann GM (2011) Functional equivalence and spatial path memory. Q J Exp Psychol 64:2081-2087
55. Loarer E, Savoyant A (1991) Visual Imagery in Locomotor Movement Without Vision. In: Logie RH, Denis M (eds) Mental Images in Human Cognition. Elsevier, The Hague
56. Loomis JM (1992) Distal attribution and presence. Presence (Camb) 1:113-119
57. Loomis JM, Beall AC (2004) Model-based control of perception/action. In: Vaina L, Beardsley S, Rushton S (eds) Optic Flow and Beyond. Kluwer Academic Publishers, Boston, M.A.
58. Loomis JM, Da Silva JA, Fujita N, Fukusima SS (1992) Visual space perception and visually directed action. J Exp Psychol Hum Percept Perform 18:906-921
59. Loomis JM, Klatzky RL (2007) Functional equivalence of spatial representations from vision, touch, and hearing: relevance for sensory substitution. In: Rieser JJ, Ashmead DH, Ebner FF, Corn AL (eds) Blindness and Brain Plasticity in Navigation and Object Perception. Lawrence Erlbaum Associates, New York
60. Loomis JM, Klatzky RL, Giudice NA (2012) Sensory substitution of vision: importance of perceptual and cognitive processing. In: Manduchi R, Kurniawan S (eds) Assistive Technology for Blindness and Low Vision. CRC, Boca Raton, FL
61. Loomis JM, Klatzky RL, Philbeck JW, Golledge RG (1998) Assessing auditory distance perception using perceptually directed action. Percept Psychophys 60:966-980
62. Loomis JM, Lippa Y, Klatzky RL, Golledge RG (2002) Spatial updating of locations specified by 3-D sound and spatial language. J Exp Psychol Learn Mem Cogn 28:335-345
63. Loomis JM, Philbeck JW (2008) Measuring spatial perception with spatial updating and action. In: Klatzky RL, Behrmann M, MacWhinney B (eds) Embodiment, Ego-Space, and Action. Taylor & Francis, New York
64. May M (2004) Imaginal perspective switches in remembered environments: transformation versus interference accounts. Cogn Psychol 48:163-206
65. Mazard A, Tzourio-Mazoyer N, Crivello F, Mazoyer B, Mellet E (2004) A PET meta-analysis of object and spatial imagery. Eur J Cogn Psychol 16:673-695
66. McNamara TP (2003) How are the locations of objects in the environment represented in memory? In: Freksa C, Brauer W, Habel C, Wender K (eds) Spatial Cognition III: Routes and Navigation, Human Memory and Learning, Spatial Representation and Spatial Reasoning. Springer, Berlin
67. Medendorp WP, Tweed DB, Crawford JD (2003) Motion parallax is computed in the updating of human spatial memory. J Neurosci 23:8135-8142
68. Milner AD, Goodale MA (2008) Two visual systems re-viewed. Neuropsychologia 46:774-785
69. Motes MA, Malach R, Kozhevnikov M (2008) Object-processing neural efficiency differentiates object from spatial visualizers. Neuroreport 19:1727-1731
70. Mou W, McNamara TP, Valiquette CM, Rump B (2004) Allocentric and egocentric updating of spatial memories. J Exp Psychol Learn Mem Cogn 30:142-157
71. Newell FN, Woods AT, Mernagh M, Bülthoff HH (2005) Visual, haptic and crossmodal recognition of scenes. Exp Brain Res 161:233-242
72. O'Keefe J, Nadel L (1978) The Hippocampus as a Cognitive Map. Oxford University Press, Oxford
73. Ooi TL, Wu B, He ZJJ (2001) Distance determined by the angular declination below the horizon. Nature 414:197-200
74. Ooi TL, Wu B, He ZJ (2006) Perceptual space in the dark affected by the intrinsic bias of the visual system. Perception 35:605-624
75. Pascual-Leone A, Hamilton R (2001) The metamodal organization of the brain. In: Casanova C, Ptito M (eds) Vision: from Neurons to Cognition (vol 134, Ch 27: Progress in Brain Research). Elsevier, Amsterdam
76. Pham QC, Hicheur H (2009) On the open-loop and feedback processes that underlie the formation of trajectories during visual and nonvisual locomotion in humans. J Neurophysiol 102:2800-2815
77. Philbeck JW, Loomis JM (1997) Comparison of two indicators of visually perceived egocentric distance under full-cue and reduced-cue conditions. J Exp Psychol Hum Percept Perform 23:72-85
78. Philbeck JW, Loomis JM, Beall AC (1997) Visually perceived location is an invariant in the control of action. Percept Psychophys 59:601-612
79. Presson CC, Montello DR (1994) Updating after rotational and translational body movements: coordinate structure of perspective space. Perception 23:1447-1455
80. Riecke BE, von der Heyde M, Bülthoff HH (2005) Visual cues can be sufficient for triggering automatic, reflex-like spatial updating. ACM Trans Appl Percept 2:183-215
81. Rieser JJ (1989) Access to knowledge of spatial structure at novel points of observation. J Exp Psychol Learn Mem Cogn 15:1157-1165
82. Rieser JJ, Ashmead DH, Talor CR, Youngquist GA (1990) Visual perception and the guidance of locomotion without vision to previously seen targets. Perception 19:675-689
83. Rieser JJ, Garing AE, Young MF (1994) Imagery, action, and young children's spatial orientation: it's not being there that counts, it's what one has in mind. Child Dev 65:1262-1278
84. Rieser JJ, Guth DA, Hill EW (1986) Sensitivity to perspective structure while walking without vision. Perception 15:173-188
85. Rieser JJ, Rider E (1991) Young children's spatial orientation with respect to multiple targets when walking without vision. Dev Psychol 27:97-107
86. Röder B, Rösler F (1998) Visual input does not facilitate the scanning of spatial images. J Mental Imagery 22:165-182
87. Russell B (1948) Human Knowledge: Its Scope and Limits. Simon and Schuster, New York
88. Sahm CS, Creem-Regehr SH, Thompson WB, Willemsen P (2005) Throwing versus walking as indicators of distance perception in similar real and virtual environments. ACM Trans Appl Percept 2:35-45
89. Shelton AL, McNamara TP (2004) Spatial memory and perspective taking. Mem Cognit 32:416-426
90. Shepard RN, Metzler J (1971) Mental rotation of three-dimensional objects. Science 171:701-703
91. Siegle JH, Campos JL, Mohler BJ, Loomis JM, Bülthoff HH (2009) Measurement of instantaneous perceived self-motion using continuous pointing. Exp Brain Res 195:429-444
92. Smythies JR (1994) The walls of Plato's Cave. The Science and Philosophy of Brain, Consciousness, and Perception. Avebury, Aldershot
93. Stevenson RJ, Case TI (2005) Olfactory imagery: a review. Psychon Bull Rev 12:244-264
94. Struiksma ME, Noordzij ML, Postma A (2009) What is the link between language and spatial images? Behavioral and neuralfindings in blind and sighted individuals. Acta Psychol 132:145-156
95. Tatler BW, Land MF (2011) Vision and the representation of the surroundings in spatial memory. Philos Trans R Soc B 366:596-610
96. Taylor HA, Tversky B (1992) Spatial mental models derived from survey and route descriptions. J Mem Lang 31:261-282
97. Teng S, Puri A, Whitney D (2012) Ultrafine spatial acuity of blind expert human echolocators. Exp Brain Res 216(4):483-488
98. Thomson JA (1983) Is continuous visual monitoring necessary in visually guided locomotion? J Exp Psychol Hum Percept Perform 9:427-443
99. Thompson WB, Willemsen P, Gooch AA, Creem-Regehr SH, Loomis JM, Beall AC (2004) Does the quality of the computer graphics matter when judging distances in visually immersive environments. Presence (Camb) 13:560-571
100. Tolman EC (1948) Cognitive maps in rats and men. Psychol Rev 55:189-208
101. Waller D, Hodgson E (2006) Transient and enduring spatial representations under disorientation and self-rotation. Mem Cognit 32:867-882
102. Waller D, Montello DR, Richardson AE, Hegarty M (2002) Orientation specificity and spatial updating of memories for layouts. J Exp Psychol Learn Mem Cogn 28:1051-1063
103. Wang RF, Spelke ES (2000) Updating egocentric representations in human navigation. Cognition 77:215-250
104. Wang RF, Crowell JA, Simons DJ, Irwin DE, Kramer AF, Ambinder MS et al (2006) Spatial updating relies on an egocentric representation of space: effects of the number of objects. Psychon Bull Rev 13:281-286
105. Wiener JM, Berthoz A, Wolbers T (2010) Dissociable cognitive mechanisms underlying human path integration. Exp Brain Res 208:61-71
106. Wolbers T, Hegarty M, Büchel C, Loomis JM (2008) Spatial updating: How the brain keeps track of changing object locations during observer motion. Nat Neurosci 11:1223-1230
107. Wu B, Ooi TL, He ZJ (2004) Perceiving distances accurately by a directional process of integrating ground information. Nature 428:73-77
108. Zwaan RA, Radvansky GA (1998) Situation models in language comprehension and memory. Psychol Bull 123:162-185