SOVEREIGN: An autonomous neural system for incrementally learning planned action sequences to navigate towards a rewarded goal

Neural Networks

Volumn 21, Issue 5, 2008, Pages 699-758

  Gnadt, William ^a   Grossberg, Stephen ^a  

^a BOSTON UNIVERSITY   (United States)

Author keywords

ART. CogEM; Attention; Autonomous control; Categorization; Gated dipole; Navigation; Pattern recognition; Planning; Reinforcement learning; Working memory

Indexed keywords

COMPUTER NETWORKS; DATA STORAGE EQUIPMENT; EDUCATION; ESTIMATION; LEARNING SYSTEMS; MECHANISMS; METROPOLITAN AREA NETWORKS; MOTORS; NATURAL RESOURCES EXPLORATION; NAVIGATION; NETWORK PROTOCOLS; REINFORCEMENT; REINFORCEMENT LEARNING; ROBOTS; TARGETS; VIRTUAL REALITY; WIRELESS NETWORKS;

3D VIRTUAL REALITY; ANIMATS; ARM MOVEMENTS; AUTONOMOUS NAVIGATION; COMPLEMENTARY PROPERTIES; ELSEVIER (CO); GRADUAL TRANSITIONS; INCREMENTAL LEARNING; INTERACTING SUBSYSTEMS; INVARIANT RECOGNITION; MOTION CUEING; NEURAL SYSTEMS; PLANNED ACTION; PLANNED BEHAVIORS; REAL TIME; SELF ORGANIZING; TARGET POSITIONING; VISUAL CUES; WORKING MEMORY (WM);

THREE DIMENSIONAL;

ARTICLE; ARTIFICIAL NEURAL NETWORK; ASSOCIATION; GOAL ATTAINMENT; HEAD MOVEMENT; LEARNING; LOCOMOTION; MOTOR PERFORMANCE; NONHUMAN; PRIORITY JOURNAL; PROPRIOCEPTION; REINFORCEMENT; REWARD; VIRTUAL REALITY; VISUAL MEMORY; WORKING MEMORY;

ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; CEREBRAL CORTEX; COMPUTER SIMULATION; CUES; EXPLORATORY BEHAVIOR; GOALS; LEARNING; MEMORY, SHORT-TERM; MOTION PERCEPTION; MOVEMENT; NEURAL NETWORKS (COMPUTER); ORIENTATION; PATTERN RECOGNITION, AUTOMATED; PROPRIOCEPTION; PSYCHOMOTOR PERFORMANCE; RECOGNITION (PSYCHOLOGY); REINFORCEMENT (PSYCHOLOGY); REWARD; SOFTWARE; SOFTWARE VALIDATION; USER-COMPUTER INTERFACE; VOLITION;

EID: 44649089467     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2007.09.016     Document Type: Article

References (94)

1. Baloch A.A., and Waxman A.M. Visual learning, adaptive expectations, and behavioral conditioning of the mobile robot MAVIN. Neural Networks 4 (1991) 271-302
2. Barto A.G., and Sutton R.S. Landmark learning: An illustration of associative search. Biological Cybernetics 42 (1981) 1-8
3. Berzhanskaya J., Grossberg S., and Mingolla E. Laminar cortical dynamics of visual form and motion interactions during coherent object motion perception. Spatial Vision 20 (2007) 337-395
4. Blodgett, H. C. (1929). An analysis of the behavior units in maze learning. In Proceedings of the 9th international congress on psychology, 9 (pp 85-86)
5. Bradski G., Carpenter G.A., and Grossberg S. Working memory networks for learning temporal order with application to three-dimensional visual object recognition. Neural Computation 4 (1992) 270-286
6. Bradski G., Carpenter G.A., and Grossberg S. STORE working memory networks for storage and recall of arbitrary temporal sequences. Biological Cybernetics 71 (1994) 469-480
7. Bradski G., and Grossberg S. Fast learning VIEWNET architectures for recognizing 3-D objects from multiple 2-D views. Neural Networks 8 (1995) 1053-1080
8. Brown J.W., Bullock D., and Grossberg S. How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Networks 17 (2004) 471-510
9. Bullock D., Cisek P., and Grossberg S. Cortical networks for control of voluntary arm movements under variable force conditions. Cerebral Cortex 8 (1998) 48-62
10. Bullock D., and Grossberg S. Neural dynamics of planned arm movements: Emergent invariants and speed-accuracy properties during trajectory formation. Psychological Review 95 (1988) 49-90
11. Bullock D., and Grossberg S. Adaptive neural networks for control of movement trajectories invariant under speed and force rescaling. Human Movement Science 10 (1991) 3-53
12. Bullock D., Grossberg S., and Guenther F.H. A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. Journal of Cognitive Neuroscience 5 4 (1993) 408-435
13. Burgess N., Recce M., and O'Keefe J. Hippocampus - Spatial models. The handbook of brain theory (1995), Bradford Books/MIT Press, Cambridge, MA
14. Buxton C.E. Latent learning and the goal gradient hypothesis. Contributions to Psychological Theory 2 2 (1940) 75
15. Cao Y., and Grossberg S. A laminar cortical model of stereopsis and 3D surface perception: Closure and da Vinci stereopsis. Spatial Vision 18 (2005) 515-578
16. Carpenter G.A., and Grossberg S. A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing 37 (1987) 54-115
17. Carpenter G.A., Grossberg S., Markuzon N., Reynolds J.H., and Rosen D.B. Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks 3 (1992) 698-713
18. Carpenter G.A., Grossberg S., and Rosen D.B. Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 4 (1991) 759-771
19. Carpenter, G. A., & Ross, W. D. 1993. ART-EMAP: A neural network architecture for learning and prediction by evidence accumulation. In Proceedings of the world congress on neural networks, III-649-656. Technical Report CAS/CNS-TR-93-015, Boston University
20. Carr H., and Watson J.B. Orientation of the white rat. Journal of Comparative Neurology and Psychology 18 (1908) 27-44
21. Cohen M.A., and Grossberg S. Neural dynamics of speech and language coding: Developmental programs, perceptual grouping, and competition for short-term memory. Human Neurobiology 5 (1986) 1-22
22. Cohen M.A., and Grossberg S. Masking fields: A massively parallel neural architecture for learning, recognizing, and predicting multiple groupings of patterned data. Applied Optics 26 (1987) 1866-1891
23. Contreras-Vidal J.L., Grossberg S., and Bullock D. A neural model of cerebellar learning for arm movement control: Cortico-spino-cerebellar dynamics. Learning and Memory 3 (1997) 475-502
24. Dallal N.L., and Meck W.H. Hierarchical structures: Chunking by food type facilitates spatial memory. Journal of Experimental Psychology: Animal Behavior Processes 16 1 (1990) 69-84
25. Daub C.T. The effect of doors on latent learning. Journal of Comparative Psychology 15 (1933) 49-58
26. Dayan P. Navigating through temporal difference. Neural information processing systems Vol. 3 (1987), Morgan Kaufmann Publishers, San Mateo, CA 464-470
27. Dennis W. Multiple visual discrimination in the block elevated maze. Journal of Comparative and Physiological Psychology 13 (1932) 391-396
28. Fang, L., & Grossberg, S. (2007). From stereogram to surface: How the brain sees the world in depth. Technical Report CAS/CNS TR-06-013, Boston University. Spatial Vision (in press)
29. Fazl, A., Grossberg, S., & Mingolla, E. (2007). View-invariant object category learning, recognition and search: How spatial and object attention are coordinated using surface-based attentional shrouds. Technical Report CAS/CNS TR-07-011, Boston University (submitted for publication)
30. Fiala J.C., Bullock D., and Grossberg S. Metabotropic glutamate receptor activation in cerebellar Purkinje cells as substrate for adaptive timing of the classically conditioned eye blink response. Journal of Neuroscience 16 (1996) 3760-3774
31. Fuhs M.C., and Touretzky D.S. A spin glass model of path integration in rat medial entorhinal cortex. Journal of Neuroscience 26 (2006) 4266-4276
32. Gaudiano P., and Grossberg S. Vector associative maps: Unsupervised real-time error-based learning and control of movement trajectories. Neural Networks 4 (1991) 147-183
33. Gilbert P.F.C., and Thach W.T. Purkinje cell activity during motor learning. Brain Research 128 (1977) 309-328
34. Gnadt W., and Grossberg S. SOVEREIGN: A self-organizing, vision, expectation, recognition, emotion, intelligent, goal-oriented navigation system. In: Blank D., and Meeden L. (Eds). Developmental robotics: Papers from the 2005 spring symposium (2005), American Association for Artificial Intelligence, Menlo Park, California 106-110
35. Gnadt, W., & Grossberg, S. (2005b). SOVEREIGN: A self-organizing, vision, expectation, recognition, emotion, intelligent, goal-oriented navigation system. In Dynamical neuroscience XIII: Computational cognitive neuroscience satellite symposium of the annual society for neuroscience meeting
36. Gnadt, W., & Grossberg, S. (2006). SOVEREIGN: A self-organizing, vision, expectation, recognition, emotion, intelligent, goal-oriented navigation system. In Tenth international conference on cognitive and neural systems
37. Goodale M.A., and Milner D. Separate visual pathways for perception and action. Trends in Neurosciences 15 (1992) 20-25
38. Gorchetchnikove A., and Grossberg S. Space, time, and learning in the hippocampus: How fine spatial and temporal scales are expanded into population codes for behavioral control. Neural Networks 20 (2007) 182-193
39. Greve D., Grossberg S., Guenther F., and Bullock D. Neural representations for sensory-motor control, I: Head-centered 3-D target positions from opponent eye commands. Acta Psychologica 82 (1993) 115-138
40. Grice G.R. An experimental test of the expectation theory of learning. Journal of Comparative and Physiological Psychology 41 (1948) 137-143
41. Grossberg S. A neural theory of punishment and avoidance, II: Quantitative theory. Mathematical Biosciences 15 (1972) 253-285
42. Grossberg S. Contour enhancement, short-term memory and constancies in reverberating neural networks. Studies in Applied Mathematics 52 (1973) 217-257
43. Grossberg S. A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plans. In: Rosen R., and Snell F. (Eds). Progress in theoretical biology Vol. 5 (1978), Academic Press, New York 233-374
44. Grossberg S. How does a brain build a cognitive code?. Psychological Review 87 (1980) 1-51
45. Grossberg S. Studies of mind and brain: Principles of learning, perception, development, cognition and motor control (1982), D. Reidel Publishing Co, Boston, MA
46. Grossberg S. Processing of expected and unexpected events during conditioning and attention: A psychophysiological theory. Psychological Review 89 (1982) 529-572
47. Grossberg S. Some psychophysiological and pharmacological correlates of a developmental, cognitive and motivational theory. In: Karrer R., Cohen J., and Tueting P. (Eds). Brain and information: Event related potentials (1984), New York Academy of Sciences, New York 58-142
48. Grossberg S. The complementary brain: Unifying brain dynamics and modularity. Trends in Cognitive Sciences 4 (2000) 233-246
49. Grossberg S. The imbalanced brain: From normal behavior to schizophrenia. Biological Psychiatry 48 (2000) 81-98
50. Grossberg S., Guenther F., Bullock D., and Greve D. Neural representations for sensory-motor control, II: Learning a head-centered visuomotor representation of 3-D target position. Neural Networks 6 (1993) 43-67
51. Grossberg S., and Merrill J.W.L. A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cognitive Brain Research 1 (1992) 3-38
52. Grossberg S., and Merrill J.W.L. The hippocampus and cerebellum in adaptively timed learning, recognition, and movement. Journal of Cognitive Neuroscience 8 (1996) 257-277
53. Grossberg S., Mingolla E., and Viswanathan L. Neural dynamics of motion integration and segmentation within and across apertures. Vision Research 41 (2001) 2521-2553
54. Grossberg S., and Myers C.W. The resonant dynamics of speech perception: Interword integration and duration-dependent backward effects. Psychological Review 107 (2000) 735-767
55. Grossberg, S., & Pearson, L. (2007). Laminar cortical dynamics of cognitive and motor working memory, sequence learning and performance: Towards a unified theory of how the cerebral cortex works (submitted for publication)
56. Grossberg S., and Seidman D. Neural dynamics of autistic behaviors: Cognitive, emotional, and timing substrates. Psychological Review 113 3 (2006) 483-525
57. Grossberg S., and Yazdanbakhsh A. Laminar cortical dynamics of 3D surface perception: Stratification, transparency, and neon color spreading. Vision Research 45 (2005) 1725-1743
58. Guenther F.H. Speech sound acquisition, coarticulation, and rate effects in a neural network model of speech production. Psychological Review 102 (1995) 594-621
59. Guenther F.H., Bullock D., Greve D., and Grossberg S. Neural representations for sensory-motor control, III: Learning a body-centered representation of 3-D target position. Journal of Cognitive Neuroscience 6 (1994) 341-358
60. Hafting T., Fyhn M., Molden S., Moser M.H., and Moser E.I. Microstructure of the spatial map in the entorhinal cortex. Nature 436 (2005) 801-806
61. Haney G.W. The effect of familiarity on maze performance of albino rats. University of California Publications on Psychology 4 (1931) 319-333
62. Hodgkin, A. L. (1964). The conduction of the nervous impulse. Springfield, Ill.: Charles C. Thomas
63. Hull C.L. The goal-gradient hypothesis and maze learning. Psychological Review 39 (1932) 25-43
64. Hull C.L. Differential habituation to internal stimuli in the albino rat. Journal of Comparative Psychology 16 (1933) 255-273
65. Hull C.L. The rat's speed of locomotion gradient in the approach of food. Journal of Comparative Psychology 17 (1934) 393-422
66. Ito M. The cerebellum and neural control (1984), Raven Press, New York
67. Kelly F.J., and Grossberg S. Neural dynamics of 3-D surface perception: Figure-ground separation and lightness perception. Perception and Psychophysics 62 (2000) 1596-1619
68. Kendler H.H. The effect of change of drive on t-maze performance. American Psychologist 1 (1946) 246
69. Kendler H.H. A comparison of learning under motivated and satiated conditions in the white rat. Journal of Experimental Psychology 37 (1947) 545-549
70. Kendler H.H. An investigation of latent learning in the t-maze. Journal of Comparative and Physiological Psychology 40 (1947) 265-270
71. Leeper R. The role of motivation in learning: A study of the phenomenon of differential motivational control of the utilization of habits. Journal of Genetic Psychology 46 (1935) 3-40
72. Leonard B., and McNaughton B.L. Spatial representation in the rat: Conceptual, behavioral, and neurophysiological perspectives. In: Kesner R.P., and Olton D.S. (Eds). Neurobiology of comparative cognition (1990), Lawrence Erlbaum Associates, Hillsdale, NJ 363-422
73. Mishkin M. A memory system in the monkey. Philosophical Transactions of the Royal Society of London, Series B 298 (1982) 85-95
74. Mishkin M. Cerebral memory circuits. In: Poggio T.A., and Glaser D.A. (Eds). Exploring brain functions: Models in neuroscience (1993), Wiley, New York 113-125
75. Mishkin M., Ungerleider L.G., and Macko K.A. Object vision and spatial vision: Two cortical pathways. Trends in Neurosciences 6 (1983) 414-417
76. Munn N.L. Handbook of psychological research on the rat (1950), Houghton Mifflin Co., Boston, MA
77. O'Keefe J.M., and Dostrovsky J. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Research 34 (1971) 171-175
78. O'Keefe J., and Nadel L. The hippocampus as a cognitive map (1978), Oxford University Press, New York, NY
79. Pribe C., Grossberg S., and Cohen M.A. Neural control of interlimb oscillations, II: Biped and quadruped gaits and bifurcations. Biological Cybernetics 77 (1997) 141-152
80. Raizada R., and Grossberg S. Towards a theory of the laminar architecture of cerebral cortex: Computational clues from the visual system. Cerebral Cortex 13 (2003) 100-113
81. Reynolds B. A repetition of the blodgett experiment on latent learning. Journal of Experimental Psychology 35 (1945) 504-516
82. Russell J.T. Depth discrimination in the rat. Journal of Genetic Psychology 40 (1932) 136-159
83. Saltzman E. Levels of sensorimotor representation. Journal of Mathematical Psychology 20 (1979) 91-163
84. Schalkoff R.J. Digital image processing and computer vision (1989), John Wiley and Sons, Inc., New York, NY
85. Schilling R.J. Fundamentals of robotics: Analysis and control (1990), Prentice Hall, Englewood Cliffs, NJ
86. Schmajuk N.A. Role of the hippocampus in temporal and spatial navigation: An adaptive neural network. Behavioral Brain Research 39 (1990) 205-229
87. Spence K.W., and Lippett R. An experimental test of the sign-gestalt theory of trial and error learning. Journal of Experimental Psychology 36 (1946) 491-502
88. Squire L.R., and Cohen N.J. Human memory and amnesia. In: Lynch G., McGaugh J., and Weinberger N.M. (Eds). Neurobiology of learning and memory (1984), Guilford Press, New York 3-64
89. Stoltz S.P., and Lott D.F. Establishment in rats of a persistent response producing net loss of reinforcement. Journal of Comparative and Physiological Psychology 57 (1964)
90. Thompson R.F. The neural basis of basic associative learning of discrete behavioral responses. Trends in Neurosciences 11 (1988) 152-155
91. Tolman E.C., and Honzik C.H. "Insight" in rats. University of California Publications in Psychology 4 14 (1930) 215-232
92. Ungerleider L.G., and Mishkin M. Two cortical visual systems: Separation of appearance and location of objects. In: Ingle D.L., Goodale M.A., and Mansfield R.J.W. (Eds). Analysis of Visual Behavior (1982), The MIT Press, Cambridge, MA 549-586
93. Walker E.L. Drive specificity and learning. Journal of Experimental Psychology 38 (1948) 39-49
94. Zadeh L. Fuzzy sets. Information and Control 8 (1965) 338-353