Swing it to the left, swing it to the right: Enacting flexible spatial language using a neurodynamic framework

Cognitive Neurodynamics

Volumn 3, Issue 4, 2009, Pages 373-400

  Lipinski, John ^a   Sandamirskaya, Yulia ^a   Schöner, Gregor ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

Dynamical systems; Neural fields; Spatial cognition; Spatial language

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; DYNAMIC FIELD THEORY; DYNAMICS; LANGUAGE; LINGUISTICS; MOTOR SYSTEM; ROBOTICS; SENSORY SYSTEM; THEORY;

EID: 72249106012     PISSN: 18714080     EISSN: 18714099     Source Type: Journal    
DOI: 10.1007/s11571-009-9096-y     Document Type: Article

References (97)

1. P Allopenna J Magnuson M Tanenhaus 1998 Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models J Mem Lang 38 419 439 10.1006/jmla.1997.2558 (Pubitemid 128348607)
2. S Amari 1977 Dynamics of pattern formation in lateral-inhibition type neural fields. Biol Cybern 27 77 87 10.1007/BF00337259 1:STN:280: DyaE1c%2FitFWlsw%3D%3D 911931
3. RA Andersen 1997 Multimodal representation of space in posterior parietal cortex and its use in planning movements Annu Rev Neurosci 20 303 330 10.1146/annurev.neuro.20.1.303 1:STN:280:DyaK2s3gvFCitA%3D%3D 9056716 (Pubitemid 27106384)
4. A Bangerter 2004 Using pointing and describing to achieve joint focus of attention. Psychol Sci 15 415 419 10.1111/j.0956-7976.2004.00694.x 15147496 (Pubitemid 38782982)
5. LW Barsalou 1999 Perceptual symbol systems Behav Brain Sci 22 577 660 1:STN:280:DC%2BD3M3ivFGguw%3D%3D 11301525
6. A Basole LE White D Fitzpatrick 2003 Mapping multiple features in the population response of visual cortex Nature 423 986 990 10.1038/nature01721 1:CAS:528:DC%2BD3sXkvVKmsrg%3D 12827202 (Pubitemid 36806913)
7. A Bastian A Riehle W Erlhagen G Schöner 1998 Prior information preshapes the population representation of movement direction in motor cortex NeuroReport 9 315 319 10.1097/00001756-199801260-00025 1:STN:280: DyaK1c7msleqsQ%3D%3D 9507975
8. A Bastian G Schöner A Riehle 2003 Preshaping and continuous evolution of motor cortical representations during movement preparation. Eur J Neurosci 18 2047 2058 10.1046/j.1460-9568.2003.02906.x 14622238 (Pubitemid 37372387)
9. R Beer 2000 Dynamical approaches to cognitive science TRENDS Cogn Sci 4 91 99 10.1016/S1364-6613(99)01440-0 10689343 (Pubitemid 30122467)
10. P beim Graben S Gerth S Vasishth 2008 Towards dynamical models of language-related brain potentials. Cogn Neurodyn 2 229 255 10.1007/s11571-008- 9041-5 19003488
11. P beim Graben D Pinotsis D Saddy R Potthast 2008 Language processing with dynamic fields Cogn Neurodyn 2 79 88 10.1007/s11571-008-9042-4 19003475
12. E Bicho G Schöner 1997 The dynamic approach to autonomous robotics demonstrated on a low-level vehicle platform Rob Auton Syst 21 23 35 10.1016/S0921-8890(97)00004-3
13. A Cangelosi T Riga 2006 An embodied model for sensorimotor grounding and grounding transfer: Experiments with epigenetic robots. Cogn Sci 30 673 689
14. L Carlson 1999 Selecting a reference frame Spat Cogn Comput 1 365 379
15. L Carlson 2006 Attention unites form and function in spatial language Spat Cogn Comput 6 295 308 10.1207/s15427633scc0604-1
16. L Carlson 2008 Inhibition within a reference frame during the interpretation of spatial language. Cognition 106 384 407 10.1016/j.cognition. 2007.03.009 17449023 (Pubitemid 350236139)
17. L Carlson G Logan 2001 Using spatial terms to select an object Mem Cognit 29 883 892 1:STN:280:DC%2BD3MnntVWmtQ%3D%3D 11716061
18. LA Carlson-Radvansky GD Logan 1997 The influence of reference frame selection on spatial template construction J Mem Lang 37 3 411 437 10.1006/jmla.1997.2519 (Pubitemid 127170022)
19. C Chambers M Tanenhaus K Eberhard H Filip G Carlson 2002 Circumscribing referential domains during real-time language comprehension J Mem Lang 47 30 49 10.1006/jmla.2001.2832 (Pubitemid 36161069)
20. P Cisek JF Kalaska 2005 Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 3 45 801 814 10.1016/j.neuron.2005.01.027 (Pubitemid 40320712)
21. Clark H (1996) Using language. Cambridge University Press, Cambridge
22. C. Colby 1998 Action-oriented spatial reference frames in cortex Neuron 20 15 24 10.1016/S0896-6273(00)80429-8 1:CAS:528:DyaK1cXnvVSltw%3D%3D 9459438 (Pubitemid 28078345)
23. Coventry K, Cangelosi A, Rajapakse R, Bacon A, Newstead S, Joyce D, Richards L (2005) Spatial prepositions and vague quantifiers: implementing the functional geometric framework. In: Freksa C (ed) Spatial cognition IV, vol LNAI 3343. Springer-Verlag, Heidelberg, pp 98-110
24. D Crowe B Averback MV Chafee 2008 Neural ensemble decoding reveals a correlate of viewer- to object-centered spatial transformation in monkey parietal cortex J Neurosci 28 20 5218 5228 10.1523/JNEUROSCI.5105-07.2008 1:CAS:528:DC%2BD1cXmt1Slt7s%3D 18480278
25. H Damasio T Grabowski D Tranel B Ponto R Hichwa AR Damasio 2001 Neural correlates of naming actions and of naming spatial relations. Neuroimage 13 1053 1064 10.1006/nimg.2001.0775 1:STN:280:DC%2BD3M3lsV2huw%3D%3D 11352611 (Pubitemid 32553536)
26. M Denis F Pazzaglia C Cornoldi L Bertolo 1999 Spatial discourse and navigation: An analysis of route direction in the city of venice Appl Cogn Psychol 13 145 174 10.1002/(SICI)1099-0720(199904)13:2<145::AID-ACP550>3. 0.CO;2-4
27. J Elman 2009 On the meaning of words and dinosaur bones: Lexical knowledge without a lexicon. Cogn Sci 33 547 582 19662108
28. W Erlhagen A Bastian D Jancke A Riehle G Schner 1999 The distribution of neuronal population activation (dpa) as a tool to study interaction and integration in cortical representations J Neurosci Methods 94 53 66 10.1016/S0165-0270(99)00125-9 1:STN:280:DC%2BD3c7gt1emsQ%3D%3D 10638815
29. W Erlhagen G Schöner 2002 Dynamic field theory of movement preparation Psychol Rev 109 545 572 10.1037/0033-295X.109.3.545 12088245
30. C Faubel G Schöner 2008 Learning to recognize objects on the fly: a neurally based dynamic field approach. Neural Netw 21 562 576 10.1016/j.neunet.2008.03.007 18501555
31. N Franklin L Henkel 1995 Parsing surrounding space into regions Mem Cognit 23 397 407 1:STN:280:DyaK2Mzps1ajtQ%3D%3D 7666754
32. J Gardner E Merriam J Movshon D Heeger 2008 Maps of visual space in human occipital cortex are retinotopic, not spatiotopic J Neurosci 28 3988 3999 10.1523/JNEUROSCI.5476-07.2008 1:CAS:528:DC%2BD1cXkvV2htr0%3D 18400898
33. K Gegenfurtner 2003 Cortical mechanisms of colour vision. Nat Rev Neurosci 4 563 572 10.1038/nrn1138 1:CAS:528:DC%2BD3sXltVWmtr8%3D 12838331 (Pubitemid 37271367)
34. AP Georgopoulos 2000 Neural aspects of cognitive motor control Curr Opin Neurobiol 10 2 238 241 10.1016/S0959-4388(00)00072-6 1:CAS:528: DC%2BD3cXivFWntro%3D 10753794 (Pubitemid 30201118)
35. AP Georgopoulos AB Schwartz RE Kettner 1986 Neural population coding of movement direction Science 233 1416 1419 10.1126/science.3749885 1:STN:280:DyaL28zhsFynsQ%3D%3D 3749885
36. A Glenberg 1997 What memory is for Behav Brain Sci 20 1 55 1:STN:280:DyaK1M7psFGgsg%3D%3D 10096994
37. A Glenberg M Kaschak 2002 Grounding language in action Psychon Bull Rev 9 558 565 12412897
38. A Glenberg D Robertson 2000 Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning J Mem Lang 43 379 401 10.1006/jmla.2000.2714
39. V Gliozzi J Mayor JF Hu K Plunkett 2009 Labels as features (not names) for infant categorization: A neurocomputational approach Cogn Sci 33 709 738
40. M Goldberg C Perfetti W Schneider 2006 Perceptual knowledge retrieval activates sensory brain regions J Neurosci 26 4917 4921 10.1523/JNEUROSCI.5389- 05.2006 1:CAS:528:DC%2BD28XkvFSgtbg%3D 16672666 (Pubitemid 44315333)
41. J Gottlieb 2007 From thought to action: The parietal cortex as a bridge between perception, action, and cognition Neuron 53 9 16 10.1016/j.neuron.2006. 12.009 1:CAS:528:DC%2BD2sXpsFSnsw%3D%3D 17196526 (Pubitemid 44969489)
42. K Grill-Spector R Malach 2004 The human visual cortex Annu Rev Neurosci 27 649 677 10.1146/annurev.neuro.27.070203.144220 1:CAS:528:DC%2BD2cXmslantLc%3D 15217346 (Pubitemid 39050416)
43. S Harnad 1990 The symbol grounding problem Physica D 42 335 346 10.1016/0167-2789(90)90087-6
44. M Hauser N Chomsky W Fitch 2002 The faculty of language: What is it, who has it, and how did it evolve? Science 298 1569 1579 10.1126/science.298.5598. 1569 1:CAS:528:DC%2BD38Xosl2ku70%3D 12446899 (Pubitemid 35387188)
45. W Hayward M Tarr 1995 Spatial language and spatial representation Cognition 55 39 84 10.1016/0010-0277(94)00643-Y 1:STN:280:DyaK2M3nvVShsg%3D%3D 7758270
46. Huyck C (2009) A psycholinguistic model of natural language parsing implemented in simulated neurons. Cogn Neurodyn (in press)
47. Jackendoff R (2002) Foundations of language. Oxford University Press, New York
48. J Johnson 2008 Moving to higher ground: the dynamic field theory and the dynamics of visual cognition. New Ideas Psychol 26 227 251 10.1016/j. newideapsych.2007.07.007 19173013
49. Johnson J, Spencer JP, Luck SJ, Schner G (2009) A dynamic neural field model of visual working memory and change detection. Psychol Sci (in press)
50. B Keysar D Barr J Balin J Brauner 2000 Taking perspective in conversation: The role of mutual knowledge in comprehension Psychol Sci 11 32 38 10.1111/1467-9280.00211 1:STN:280:DC%2BD3M7ltFOnsw%3D%3D 11228840
51. T Kohonen 1982 Self-organized formation of topologically correct feature maps Biol Cybern 43 59 69 10.1007/BF00337288
52. B Landau R Jackendoff 1993 "what" and "where" in spatial language and spatial cognition. Behav Brain Sci 16 217 265
53. Levinson S (2003) Space in language and cognition: explorations in cognitive diversity. Cambridge University Press, Cambridge
54. P Li 2009 Lexical organization and competition in first and second languages: Computational and neural mechanisms Cogn Sci 33 629 664
55. Lipinski J, Schneegans S, Sandamirskaya Y, Spencer J, Schöner G (2009) Robotic demonstrations of flexible reference frame transformations in spatial language. Manuscript under review
56. Lipinski J, Spencer JP, Samuelson L (2009) It's in the eye of the beholder: Spatial language and spatial memory use the same perceptual reference frames. In: Smith LB, Gasser M, Mix K (eds) The spatial foundations of language. Oxford University Press, Oxford (in press)
57. Logan GD (1994) Spatial attention and the apprehension of spatial relations. J Exp Psychol Hum Percept Perform 20:1015-1036
58. Logan GD, Sadler DD (1996) A computational analysis of the apprehension of spatial relations. In: Bloom P, Peterson M, Nadel L, Garrett M (eds) Language and space (language, speech, and communication). MIT Press, Cambridge, pp 493-529
59. J Magnuson J Dixon M Tanenhaus R Aslin 2007 The dynamics of lexical competition during spoken word recognition Cogn Sci 31 1 24
60. A Martin J Haxby F Lalonde C Wiggs LG Ungerleider 1995 Discrete cortical regions associated with knowledge of color and knowledge of action. Science 270 102 105 10.1126/science.270.5233.102 1:CAS:528:DyaK2MXosFOmsbw%3D 7569934
61. McNeill D (1992) Hand and mind: what gestures reveal about thought. University of Chicago Press, Chicago
62. D McNeill 2003 Aspects of aspect Gesture 3 1 17 10.1075/gest.3.1.02mcn
63. R Moratz T Tenbrink 2006 Spatial reference in linguistic human-robot interaction: iterative, empirically supported development of a model of projective relations Spat Cogn Comput 6 63 106 10.1207/s15427633scc0601-3
64. O'Keefe J (2003) Vector grammar, places, and the functional role of the spatial prepositions in english. In: van der Zee E, Slack J (eds) Representing direction in language and space. Oxford University Press, Oxford
65. Pfeifer R, Bongard J (2006) How the body shapes the way we think: a new view of intelligence. MIT Press, Cambridge
66. S. Pinker R. Jackendoff 2005 The faculty of language: what's special about it? Cognition 95 201 236 10.1016/j.cognition.2004.08.004 15694646 (Pubitemid 40203124)
67. A Pouget S Deneve JR Duhamel 2002 A computational perspective on the neural basis of multisensory spatial representation Nat Rev Neurosci 3 741 747 10.1038/nrn914 1:CAS:528:DC%2BD38Xmslagsr4%3D 12209122 (Pubitemid 135706679)
68. F Pulvermuller 2002 A brain perspective on language mechanisms: from discrete neuronal ensembles to serial order Prog Neurobiol 67 2 85 111 10.1016/S0301-0082(02)00014-X 12126657 (Pubitemid 34786166)
69. T Regier L Carlson 2001 Grounding spatial language in perception: An empirical and computational investigation J Exp Psychol Gen 130 2 273 298 10.1037/0096-3445.130.2.273 1:STN:280:DC%2BD38%2Fht1Cgsw%3D%3D 11409104
70. Samuelson L, Schutte AR, Horst J (2009) The dynamic nature of knowledge: Insights from a dynamic field model of children's novel noun generalization task. Cognition 110:322-345
71. Sandamirskaya Y, Schöner G (2006) Dynamic field theory and embodied communication. In: Wachsmuth I, Knoblich G (eds) Modeling communication with robots and virtual humans, Lecture Notes in Computer Science, vol 4930. Springer, pp 260-278
72. Sandamirskaya Y, Schöner G (2008) Dynamic field theory of sequential action: a model and its implementation. In: The seventh international conference on development and learning. Monterey, CA, pp 133-138
73. Schneegans S, Schöner G (2008) Dynamic field theory as a framework for understanding embodied cognition. In: Calvo P, Gomila T (eds) Handbook of cognitive science: an embodied approach, Elsevier, Amsterdam, pp 241-271
74. M Schober 1993 Spatial perspective-taking in conversation Cognition 47 1 24 10.1016/0010-0277(93)90060-9 1:STN:280:DyaK3s3ksFCrtg%3D%3D 8482069
75. Schöner G (2008) Dynamical systems approaches to cognition. In: Sun R (ed) The Cambridge handbook of computational psychology. Cambridge University Press, Cambridge, pp 101-126
76. Schutte AR, Spencer JP (2009) Tests of the dynamic field theory and the spatial precision hypothesis: capturing a qualitative developmental transition in spatial working memory. J Exp Psychol Hum Percept Perform (in press)
77. AR Schutte JP Spencer G Schner 2003 Testing the dynamic field theory: Working memory for locations becomes more spatially precise over development Child Dev 74 5 1393 1417 10.1111/1467-8624.00614 14552405
78. VS Simmering AR Schutte JP Spencer 2008 Generalizing the dynamic field theory of spatial cognition across real and developmental time scales Brain Res 1202 68 86 10.1016/j.brainres.2007.06.081 1:CAS:528:DC%2BD1cXjs1WjtbY%3D 17716632
79. Spencer JP, Perone S, Johnson J (2009) The dynamic field theory and embodied cognitive dynamics. In: Spencer JP, Thomas M, McClelland J (eds) Toward a new grand theory of development? connectionism and dynamic systems theory re-considered. Oxford University Press, New York (in press)
80. Spencer JP, Simmering VS, Schutte AR, Schöner G (2007) What does theoretical neuroscience have to offer the study of behavioral development? insights from a dynamic field theory of spatial cognition. In: Plumert JM, Spencer JP (eds) Emerging landscapes of mind: mapping the nature of change in spatial cognition. Oxford University Press, Oxford, pp 320-361
81. Spivey M, Dale R (2004) On the continuity of mind: toward a dynamical account of cognition. In: Ross B (ed) The psychology of learning and motivation, vol 45. Elsevier, San Diego, pp 87-142
82. M Spivey R Dale 2006 Continuous dynamics in real-time cognition Curr Dir Psychol Sci 15 207 211 10.1111/j.1467-8721.2006.00437.x (Pubitemid 44800061)
83. Spivey MJ, Tyler MJ, Eberhard KM, Tanenhaus MK (2001) Linguistically mediated visual search. Psychol Sci 12:282-286
84. Sporns O (2004) Complex neural dynamics. In: Jirsa V, Kelso JAS (eds) Coordination dynamics: issues and trends. Springer, Berlin, pp 197-215
85. W Suzuki K Matsumoto K Tanaka 2006 Neuronal Responses to Object Images in the Macaque Inferotemporal Cortex at Different Stimulus Discrimination Levels J Neurosci 26 41 10524 10535 10.1523/JNEUROSCI.1532-06.2006 1:CAS:528: DC%2BD28XhtFegurzP 17035537 (Pubitemid 44564603)
86. M Tanenhaus M Spivey-Knowlton K Eberhard J Sedivy 1995 Integration of visual and linguistic information in spoken language comprehension Science 268 1632 1634 10.1126/science.7777863 1:CAS:528:DyaK2MXmtlarurw%3D 7777863
87. H Taylor B Tversky 1992 Spatial mental models derived from survey and route descriptions J Mem Lang 31 261 282 10.1016/0749-596X(92)90014-O
88. M. Tettamanti G Buccino M Saccuman V Gallese M Danna P Scifo F Fazio G Rizzolatti S Cappa D Perani 2005 Listening to action-related sentences activates fronto-parietal motor circuits J Cogn Neurosci 17 273 281 10.1162/ 0898929053124965 15811239 (Pubitemid 40194709)
89. E Thelen G Schner C Scheier LB Smith 2001 The dynamics of embodiment: A dynamic field theory of infant perseverative reaching errors Behav Brain Sci 24 1 86 10.1017/S0140525X01003910 1:STN:280:DC%2BD3MvntFSitw%3D%3D 11515285 (Pubitemid 32725629)
90. Thelen E, Smith LB (1994) A Dynamic Systems Approach to the development of cognition and action. MIT Press, Cambridge
91. G Tononi GM Edelman O Sporns 1998 Complexity and coherency: Integrating information in the brain Trends Cogn Sci 2 474 484 10.1016/S1364-6613(98)01259-5 (Pubitemid 28557110)
92. M Wallentin S Ostergaard T Lund L Ostergaard A Roepstorff 2005 Concrete spatil language: See what i mean Brain Lang 92 221 233 10.1016/j.bandl.2004.06. 106 15721955 (Pubitemid 40249668)
93. H Wersing JJ Steil H Ritter 2001 A Competitive-Layer Model for Feature Binding and Sensory Segmentation Neural Comp 13 2 357 387 10.1162/ 089976601300014574 1:STN:280:DC%2BD3M3gt1Grug%3D%3D (Pubitemid 33595003)
94. C Wilimzig S Schneider G Schner 2006 The time course of saccadic decision making: Dynamic field theory. Neural Netw 19 1059 1074 10.1016/j.neunet.2006. 03.003 16942860 (Pubitemid 44466337)
95. Wilimzig C, Schöner G (2009) How categorical behavior emerges from continuous neural representations: dynamic field theory (in preparation)
96. HR Wilson JD Cowan 1973 A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue Kybernetik 13 55 80 10.1007/BF00288786 1:STN:280:DyaE2c%2Fmsl2ntw%3D%3D 4767470
97. J Xing RA Anderson 2000 Models of the posterior parietal cortex which perform multimodal integration and represent space in several coordinate frames J Cogn Neurosci 12 4 601 614 10.1162/089892900562363 1:STN:280: DC%2BD3cvjsVKguw%3D%3D 10936913