Learning CPG-based biped locomotion with a policy gradient method

Robotics and Autonomous Systems

Volumn 54, Issue 11, 2006, Pages 911-920

  Matsubara, Takamitsu ^a,c   Morimoto, Jun ^b,c   Nakanishi, Jun ^b,c   Sato, Masa aki ^c   Doya, Kenji ^a,c,d  

^a NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

^b JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

^c ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^d OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

Author keywords

Biped locomotion; Central pattern generator; Policy gradient; Reinforcement learning

Indexed keywords

CENTRAL PATTERN GENERATORS; POLICY GRADIENT; REINFORCEMENT LEARNING;

COMPUTER SIMULATION; LEARNING SYSTEMS; PATTERN RECOGNITION; ROBOTS; SENSORY FEEDBACK;

BIPED LOCOMOTION;

EID: 33749990848     PISSN: 09218890     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.robot.2006.05.012     Document Type: Article

References (26)

1. Matsuoka K. Sustained oscillations generated by mutually inhibiting neurons with adaptation. Biological Cybernetics 52 (1985) 367-376
2. Taga G., Yamaguchi Y., and Shimizu H. Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biological Cybernetics 65 (1991) 147-159
3. Fukuoka Y., Kimura H., and Cohen A. Adaptive dynamic walking of a quadruped robot on irregular terrain based on biological concepts. The International Journal of Robotics Research 22 3-4 (2003) 187-202
4. G. Endo, J. Morimoto, J. Nakanishi, G. Cheng, An empirical exploration of a neural oscillator for biped locomotion control, in: IEEE International Conference on Robotics and Automation, 2004, pp. 3036-3042
5. Hase K., and Yamazaki N. Computer simulation of the ontogeny of biped walking. Anthropological Science 106 4 (1998) 327-347
6. M. Sato, Y. Nakamura, S. Ishii, Reinforcement learning for biped locomotion, in: International Conference on Artificial Neural Networks, 2002, pp. 777-782
7. Williams R.J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8 (1992) 229-256
8. Kimura H., and Kobayashi S. An analysis of actor/critic algorithms using eligibility traces: Reinforcement learning with imperfect value function. Internal Conference on Machine Learning (1998) 278-286
9. Sutton R.S., McAllester D., Singh S., and Mansour Y. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems 12 (2000) 1057-1063
10. Baxter J., and Bartlett P.L. Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research 15 (2001) 319-350
11. Konda V., and Tsitsiklis J. On actor-critic algorithms. Society for Industrial and Applied Mathematics 42 4 (2003) 1143-1166
12. Sutton R.S., and Barto A.G. Reinforcement Learning: An Introduction (1998), MIT Press
13. S. Singh, T. Jaakkola, M. Jordan, Learning without state-estimation in partially observable markovian decision processes, in: In Machine Learning: Proceedings of the Eleventh International Conference, 1994, pp. 284-292
14. H. Kimura, T. Yamashita, S. Kobayashi, Reinforcement learning of walking behavior for a four-legged robot, in: Proceedings of the IEEE Conference on Decision and Control, 2001, pp. 411-416
15. R. Tedrake, T.W. Zhang, H.S. Seung, Stochastic policy gradient reinforcement learning on a simple 3D biped, in: Proceedings of the IEEE International Conference on Intelligent Robots and Systems, 2004, pp. 2849-2854
16. Doya K. Reinforcement learning in continuous time and space. Neural Computation 12 (2000) 219-245
17. J. Morimoto, G. Zeglin, C. Atkeson, Minimax differential dynamic programming: Application to a biped walking robot, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003, pp. 1927-1932
18. Morimoto J., and Doya K. Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. Robotics and Autonomous Systems 36 (2001) 37-51
19. Kagami S., Kanehiro F., Tamiya Y., Inaba M., and Inoue H. Autobalancer: An online dynamic balance compensation scheme for humanoid robots. In: Donald B.R., Lynch K., and Rus D. (Eds). Algorithmic and Computational Robotics: New Directions (2001), A K Peters, Ltd. 329-340
20. Kagami S., Kitagawa T., Nishiwaki K., Sugihara T., and Inaba M. A fast dynamically equilibrated walking trajectory generation method of humanoid robot. Autonomous Robots 12 (2002) 71-82
21. K. Hirai, M. Hirose, Y. Haikawa, T. Takenaka, The development of Honda humanoid robot, in: IEEE International Conference on Robotics and Automation, 1998, pp. 1321-1326
22. Strogatz S.H. Nonlinear Dynamics and Chaos (1994), Westview press
23. C. Tuchiya, H. Kimura, S. Kobayashi, Policy learning by ga using importance sampling, in: The 8th Conference on Intelligent Autonomous Systems, 2004, pp. 281-290
24. D. Aberdeen, J. Baxter, Scalable internal-state policy-gradient methods for pomdps, in: ICML, 2002, pp. 3-10
25. Tesauro G. Td-gammon, A self teaching backgammon program, achieves master legel play. Neural Computation 6 (1994) 215-219
26. M.J. Mataric, Reward functions for accelerated learning, in: Machine Learning: Proceedings of the Eleventh International Conference, 1994, pp. 181-189