Learning sensory feedback to CPG with policy gradient for biped locomotion

Proceedings - IEEE International Conference on Robotics and Automation

Volumn 2005, Issue , 2005, Pages 4164-4169

  Matsubara, Takamitsu ^a,b   Morimoto, Jun ^a,c   Nakanishi, Jun ^a,c   Sato, Masa Aki ^a,c   Doya, Kenji ^a,c,d  

^a NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

^b ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^c JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

^d OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

Author keywords

Biped locomotion; Central pattern generator; Policy gradient; Reinforcement learning

Indexed keywords

BIPED LOCOMOTION; COMPUTER SIMULATION; FEEDBACK CONTROL; GRADIENT METHODS; LEARNING ALGORITHMS; MOTION PLANNING; SENSORY FEEDBACK;

CENTRAL PATTERN GENERATOR; POLICY GRADIENT; REINFORCEMENT LEARNING;

ROBOTIC ARMS;

EID: 33846174631     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ROBOT.2005.1570759     Document Type: Conference Paper

References (18)

1. K. Matsuoka, "Sustained oscillatons generated by mutually inhibiting neurons with adaptation," Biologial Cybernetics, vol. 52, pp. 367-376, 1985.
2. G. Taga, Y. Yamaguchi, and H. Shimizu, "Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment," Biological Cybernetics, vol. 65, pp. 147-159, 1991.
3. G. Endo, J. Morimoto, J. Nakanishi, and G. Cheng, "An empirical exploration of a neural oscillator for biped locomotion control," in IEEE Iternational Conference on Robotics and Automation, 2004, pp. 3036-3042.
4. Y. Fukuoka, H. Kimura, and A. Cohen, "Adaptive dynamic walking of a quadruped robot on irregular terrain based on biological concepts," The International Journal of Robotics Reserch, vol. 22, no. 3-4, pp. 187-202, 2003.
5. K. Hase and N. Yamazaki, "Computer simulation of the ontogeny of biped walking," Anthropological Science, vol. 106(4), pp. 327-347, 1998.
6. M. Sato, Y. Nakamura, and S. Ishii, "Reinforcement learning for biped locomotion," in International Conference on Artificial Neural Networks, 2002, pp. 777-782.
7. H. Kimura and S. Kobayashi, "An analysis of actor/critic algorithms using eligibility traces: Reinforcement learning with imperfect value function," Internal Conferrence on Machine Learning, pp. 278-286, 1998.
8. R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation," Advances in Neural Information Processing Systems, vol. 12, pp. 1057-1063, 2000.
9. V. Konda and J. Tsitsildis, "On actor-critic algorithms," Society for Industrial and Applied Mathematics, vol. 42, no. 4, pp. 1143-1166, 2003.
10. T. Matsubara, J. Morimoto, J. Nakanishi, M. Sato, and K. Doya, "Learning a dynamic policy by using policy gradient: Application to biped walking," in The Institute of Electronics, Information and Communication Engineers, Technical Report of IEICE, no. 2003-128, 2004, pp. 53-58, in Japanese.
11. J. Morimoto, G. Zeglin, and C. Atkeson, "Minimax differential dynamic programming: application to a biped walking robot," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003, pp. 1927-1932.
12. K. Doya, "Reinforcement learning in continuous time and space," Neural Computation, vol. 12, pp. 219-245, 2000.
13. J. Peters, S. Vijayakumar, and S. Schaal, "Reinforcement learning for humanoid robotics," in Humanoids2003, Third IEEE-RAS International Conference on Humanoid Robots, 2003, (CD-ROM).
14. S. Kagami, F. Kanehiro, Y. Tamiya, M. Inaba, and H. Inoue, "Autobalancer: An online dynamic balance compensation scheme for humanoid robots," in Algorithmic and Computational Robotics: New Directions, B. R. Donald, K. Lynch, and D. Rus, Eds. A K Peters, Ltd., 2001, pp. 329-340.
15. S. Kagami, T. Kitagawa, K. Nishiwaki, and T. sugiharaand M. Inaba, "A fast dynamically equilibrated walking trajectory generation method of humanoid robot," Autonomouns Robots, vol. 12, pp. 71-82, 2002.
16. K. Hirai, M. Hirose, Y. Haikawa, and T. Takenaka, "The development of handa humanoid robot," in IEEE International Conference on Robotics and Automation, 1998, pp. 1321-1326.
17. T. Mori, Y. Nakamura, and S. Ishii, "Reinforcement learning based on a policy gradient method for biped locomotion," in The Institute of Electronics, Information and Communication Engineers, Technical Report of IEICE, no. 2003-206, 2004, pp. 73-78, in Japanese.
18. T. Mori, Y. Nakamura, M. Sato, and S. Ishii, "Reinforcement learning for a CPG-driven biped robot," in Nineteenth National Conference on Artificial Intelligence (AAAI), 2004, pp. 623-630.