Motor primitive and sequence self-organization in a hierarchical recurrent neural network

Neural Networks

Volumn 17, Issue 8-9, 2004, Pages 1291-1309

  Paine, Rainer W ^a   Tani, Jun ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

Genetic algorithm; Hierarchical; Initial sensitivity; Motor primitive; Recurrent neural network; Robot; Self organization; Sequence

Indexed keywords

ACTIVATION STATES; NAVIGATION TASKS; PARAMETRIC BIFURCATION; PRIMITIVES;

BIFURCATION (MATHEMATICS); COMPUTER SIMULATION; GENETIC ALGORITHMS; HIERARCHICAL SYSTEMS; MOTORS; PARAMETER ESTIMATION; RECURRENT NEURAL NETWORKS; ROBOTS; SENSITIVITY ANALYSIS; TOPOLOGY;

SELF ORGANIZING MAPS;

BEHAVIOR; BRAIN MAPPING; CELL ACTIVATION; COMPUTER SIMULATION; CONFERENCE PAPER; GENETIC ALGORITHM; MOTOR PERFORMANCE; NERVE CELL NETWORK; NONLINEAR SYSTEM; PARAMETER; PRIORITY JOURNAL; ROBOTICS; SENSITIVITY ANALYSIS;

ALGORITHMS; COMPUTER SYSTEMS; GOALS; MAZE LEARNING; MODELS, GENETIC; MOTOR NEURONS; MOVEMENT; NEURAL NETWORKS (COMPUTER); ROBOTICS;

EID: 9144259348     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2004.08.005     Document Type: Conference Paper

References (54)

1. R. Aharonov-Barki, T. Beker, and E. Ruppin Spontaneous evolution of command neurons, place cells and memory mechanisms in autonomous agents In Advances in Artificial Life, ECAL'99, vol. 1674, Lecture Notes in Artificial Intelligence 1999 Springer Berlin
2. Amit, R., & Mataric, M. J. (2002). Parametric primitives for motor representation and control. International conference on robotics and automation (ICRA), Washington, DC, May 11-15, 2002.
3. M.A. Arbib Perceptual structures and distributed motor control V.B. Brooks Handbook of physiology, Section 2: The nervous system Motor control, Part 1 Vol. II 1981 American Physiological Society 1449 1480
4. R.D. Beer, and J.C. Gallagher Evolving dynamical neural networks for adaptive behavior Adaptive Behavior 1 1 1992 92 122
5. J. Blynel Evolving reinforcement learning-like abilities for robots A. Tyrrell P.C. Haddow J. Torresen Evolvable systems: from biology to hardware: Fifth international conference 2003
6. J. Blynel, and D. Floreano Levels of dynamics and adaptive behavior in evolutionary neural controllers B. Hallam D. Floreano J. Hallam G. Hayes J.A. Meyer From animals to animats 7: Proceedings of the seventh international conference on simulation of adaptive behavior 2002 MIT Press Cambridge, MA
7. T. Dietterich Hierarchical reinforcement learning with the MAXQ value function decomposition Journal of Artificial Intelligence Research 13 2000 227 303
8. K. Doya, K. Samejima, K. Katagiri, and M. Kawato Multiple model-based reinforcement learning Neural Computation 14 2002 1347 1369
9. D.H. Edwards, W.J. Heitler, and F.B. Krasne Fifty years of a command neuron: the neurobiology of escape behavior in the crayfish Trends in Neurosciences 22 4 1999 153 161
10. J. Fan, Q. Yao, and H. Tong Estimation of densities and sensitivity measures in nonlinear dynamical systems Biometrika 83 1 1996 189 206
11. R.E. Fikes, P.E. Hart, and N.J. Nilsson Learning and executing generalized robot plans Artificial Intelligence 3 1972 251 288
12. D. Floreano, and F. Mondada Automatic creation of an autonomous agent: Genetic evolution of a neural-network driven robot D. Cliff P. Husbands J. Meyer S.W. Wilson From animals to animats 3: Proceedings of the third conference on simulation of adaptive behavior 1994 MIT Press Cambridge, MA
13. R.M. French Using semi-distributed representation to overcome catastrophic forgetting in connectionist networks Proceedings of the 13th annual cognitive science society conference (pp. 173-178) 1991 Lawrence Erlbaum Hillsdale, NJ
14. K. Fujimoto, and K. Kaneko How fast elements can affect slow dynamics Physica D: Nonlinear Phenomena 180 2003 1 2
15. S.F. Giszter, F.A. Mussa-Ivaldi, and E. Bizzi Convergent force fields organized in the frog's spinal cord Journal of Neuroscience 13 2 1993 467 491
16. D.E. Goldberg The design of innovation: Lessons from and for competent genetic algorithms 2002 Kluwer Academic Publishers Boston, MA
17. S. Grossberg Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control Boston studies in the philosophy of science Vol. 70 1982 Reidel Dordrecht
18. I. Harvey, P. Husband, A. Thompson, and N. Jakobi Evolutionary robotics: the Sussex approach Robotics and Autonomous Systems 20 1997 205 224
19. S. Haykin Neural networks: A comprehensive foundation 1994 Prentice-Hall Englewood Cliffs, NJ
20. S. Hochreiter, and J. Schmidhuber Long short-term memory Neural Computation 9 8 1997 1735 1780
21. A. Ijspeert, J. Nakanishi, and S. Schaal Learning attractor landscapes for learning motor primitives S. Becker S. Thrun K. Obermayer Advances in neural information processing systems Vol. 15 2003 MIT Press Cambridge, MA
22. R. Jacobs, M. Jordan, S. Nowlan, and G. Hinton Adaptive mixtures of local experts Neural Computation 3 1 1991 79 87
23. M. Jordan, and R. Jacobs Hierarchical mixtures of experts and the EM algorithm Neural Computation 6 2 1994 181 214
24. M. Kawato, K. Furukawa, and R. Suzuki A hierarchical neural network model for the control and learning of voluntary movement Biological Cybernetics 57 1987 169 185
25. M.J. Mataric Sensory-motor primitives as a basis for imitation: Linking perception to action and biology to robotics K. Dautenhahn C.L. Nehaniv Imitation in animals and artifacts 2002 MIT Press Cambridge, MA 391 422
26. M. McCloskey, and N.J. Cohen Catastrophic interference in connectionist networks: the sequential learning problem The Psychology of Learning and Motivation 24 1989 109 165
27. McGovern, A., & Barto, A. G. (2001). Accelerating reinforcement learning through the discovery of useful subgoals. Proceedings of the sixth international symposium on artificial intelligence, robotics, and automation in space: i-SAIRAS.
28. A. McGovern, R.S. Sutton, and A.H. Fagg Roles of macro-actions in accelerating reinforcement learning Proceedings of the 1997 Grace Hopper Celebration of Women in Computing 1997 13 18
29. B. Mesot, E. Sanchez, C. Pena, and A. Perez-Uribe SOS++: finding smart behaviors using learning and evolution R.K. Standish M.A. Abbass H.A. Bedan Artificial life VIII 2002 MIT Press Cambridge, MA 264 273
30. M. Mitchell An introduction to genetic algorithms 1998 MIT Press Cambridge, MA
31. F.A. Mussa-Ivaldi, S.F. Giszter, and E. Bizzi Linear combination of primitives in vertebrate motor control Proceedings of the National Academy of Sciences, USA 91 1994 7535 7538
32. R.D. Needlman Growth and development R.E. Behrman R.M. Kliegman H.B. Jenson Nelson textbook of pediatrics, Part II 17th ed. 2003 Saunders London 33
33. Y. Ninokura, H. Mushiake, and J. Tanji Integration of temporal order and object information in the monkey lateral prefrontal cortex Journal of Neurophysiology 10 2003 1152
34. R. Nishimoto, and J. Tani Learning to generate combinatorial action sequences utilizing the initial sensitivity of deterministic dynamical systems Proceedings of the seventh international work-conference on artificial and natural neural networks (IWANN'03) (pp. 422-429) 2003 Springer Berlin
35. S. Nolfi Evolving robots able to self-localize in the environment: the importance of viewing cognition as the result of processes occurring at different time scales Connection Science 14 3 2002 231 244
36. S. Nolfi, and D. Floreano Evolutionary robotics: the biology, intelligence, and technology of self-organizing machines 2000 MIT Press Cambridge, MA
37. R. Parr, and S. Russell Reinforcement learning with hierarchies of machines M.I. Jordan M.J. Kearns S.A. Solla Advances in neural information processing systems vol. 10 1998 MIT Press Cambridge, MA 1043 1049
38. D. Precup, and R.S. Sutton Multi-time models for temporally abstract planning M.I. Jordan M.J. Kearns S.A. Solla Advances in neural information processing systems vol. 10 1997 MIT Press Cambridge, MA 1050 1056
39. K. Sakai, K. Kitaguchi, and O. Hikosaka Chunking during human visuomotor sequence learning Experimental Brain Research 152 2 2003 229 242
40. S. Schaal Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 3 1999 233 242
41. H.T. Siegelmann, and E.D. Sontag On the computational power of neural nets Journal of Computer and System Sciences 50 1 1995 132 150
42. T. Smith, P. Husbands, A. Philippides, and M. O'Shea Temporally adaptive networks: analysis of gasnet robot control networks R.K. Standish M.A. Abbass H.A. Bedan Artificial life VIII 2002 MIT Press Cambridge, MA 274 282
43. R. Sun, E. Merrill, and T. Peterson From implicit skills to explicit knowledge: a bottom-up model of skill learning Cognitive Science 25 2 2001 203 244
44. R. Sutton, D. Precup, and S. Singh Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning Artificial Intelligence 112 1999 181 211
45. J. Tani Learning to generate articulated behavior through the bottom-up and the top-down interaction processes Neural Networks 16 1 2003 11 23
46. J. Tani, and S. Nolfi Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems Neural Networks 12 1999 1131 1141
47. J. Tanji, and K. Shima Role for supplementary motor area cells in planning several movements ahead Nature 371 1994 413 416
48. T. Teyke, K.R. Weiss, and I. Kupfermann An identified neuron (CPR) evokes neuronal responses reflecting food arousal in Aplysia Science 247 1990 85 87
49. K.A. Thoroughman, and R. Shadmehr Learning of action through combination of motor primitives Nature 407 2000 742 747
50. E. Tucci, M. Quinn, and I. Harvey An evolutionary ecological approach to the study of learning behavior using a robot-based model Adaptive Behavior 10 3/4 2002 201 222
51. P. Werbos Backpropagation through time: what it does and how to do it Proceedings of the IEEE 78 1990 1550 1560
52. D.M. Wolpert, and M. Kawato Multiple paired forward and inverse models for motor control Neural Networks 11 1998 1317 1329
53. B. Yamauchi, and R.D. Beer Sequential behavior and learning in evolved dynamical neural networks Adaptive Behavior 2 3 1994 219 246
54. T. Ziemke, and M. Thieme Neuromodulation of reactive sensorimotor mappings as a short-term memory mechanism in delayed response tasks Adaptive Behavior 10 3/4 2002 185 199