Learning biped locomotion

IEEE Robotics and Automation Magazine

Volumn 14, Issue 2, 2007, Pages 41-51

  Morimoto, Jun ^a   Atkeson, Christopher G ^b  

^a JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Biped walking; Poincar map; Reinforcement learning

Indexed keywords

BIPED WALKING; LOCAL STABILITIES; MODEL-BASED REINFORCEMENT LEARNING; NOMINAL TRAJECTORIES; SIMULATION RESULTS; VIA-POINTS; WALKING PATTERNS;

BIPED LOCOMOTION; EDUCATION; REINFORCEMENT; REINFORCEMENT LEARNING; ROBOTICS; ROBOTS;

LEARNING ALGORITHMS;

EID: 40949123394     PISSN: 10709932     EISSN: None     Source Type: Journal    
DOI: 10.1109/MRA.2007.380654     Document Type: Article

References (30)

1. H. Benbrahim and J. Franklin, "Biped dynamic walking using reinforcement learning," Robot. Autonom. Syst., vol. 22, no. 3-4, pp. 283-302, 1997.
2. C. Chew and G.A. Pratt, "Dynamic bipedal walking assisted by learning," Robotica, vol. 20, no. 5, pp. 477-491, 2002.
3. R.Q. Van der Linde, "Passive bipedal walking with phasic muscle contraction," Biol. Cybern., vol. 81, no. 3, pp. 227-237, 1999.
4. K. Doya, "Reinforcement learning in continuous time and space," Neural Comput., vol. 12, no. 1, pp. 219-245, 2000.
5. P. Dyer and S.R. McReynolds, The Computation and Theory of Optimal Control. New York: Academic, 1970.
6. Y. Ehara and S. Yamamoto, Introduction to Body-Dynamics - Analysis of Gait and Gait Initiation. Bunkyo-ku, Tokyo: Ishiyaku Publishers, Inc., 2002 (in Japanese).
7. T. Flash and N. Hogan, "The coordination of arm movements: An exper-imentally confirmed mathematical model," J. Neurosci., vol. 5, no. 7, pp. 1688-1703, 1985.
8. M. Garcia, A. Chatterjee, A. Ruina, and M.J. Coleman, "The simplest walking model: Stability, complexityu, and scaling," ASME J. Biomech. Eng., vol. 120, no. 2, pp. 281-288, 1998.
9. K. Hirai, M. Hirose, and T. Takenaka, "The Development of Honda Humanoid Robot," in Proc. 1998 IEEE Int. Conf. Robotics Automation 1998, pp. 160-165.
10. S. Hyon and T. Emura, "Symmetric Walking Control: Invariance and Global Stability," in IEEE Int. Conf. Robotics Automation, Barcelona, Spain, 2005, pp. 1456-1462.
11. D.H. Jacobson and D.Q. Mayne, Differential Dynamic Programming. New York: Elsevier, 1970.
12. H. Miyamoto and M. Kawato, "A tennis serve and upswing learning robot based on dynamics optimization theory," Neural Netw., vol. 11, no. 7-8, pp. 1331-1344, 1998.
13. H. Miyamoto, J. Morimoto, K. Doya, and M. Kawato, "Reinforcement learning with via-point representation," Neural Netw., vol. 17, no. 3, pp. 299-305, 2004.
14. H. Miyamoto, S. Schaal, F. Gandolfo, H. Gomi, Y. Koike, R. Osu, E. Nakano, Y. Wada, and M. Kawato, "A Kendama Learning Robot Based on Bi-directional Theory," Neural Netw., vol. 9, no. 8, pp. 1281-1302, 1996.
15. J. Morimoto and C.G. Atkeson, "Robust low-torque biped walking using differential dynamic programming with a minimax criterion," in Proc. 5th Int. Conf. Climbing Walking Robots, 2002, pp. 453-459.
16. J. Morimoto and C.G. Atkeson, "Minimax differential dynamic programming: An application to robust biped walking," in Advances in Neural Information Processing Systems, vol. 15, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2003, pp. 1563-1570.
17. K. Nagasaka, M. Inaba, and H. Inoue, "Stabilization of dynamic walk on a humanoid using torso position compliance control," in Proc. 17th Annu. Conf. Robotics Society Japan, 1999, pp. 1193-1194.
18. Y. Nakamura, M. Sato, and S. Ishii, "Reinforcement learning for biped robot," in Proc. 2nd Int. Symp. Adaptive Motion Animals Machines, 2003, pp. ThP-II-5.
19. J. Nakanishi, J. Morimoto, G. Endo, G. Cheng, S. Schaal, and M. Kawato, "Learning from demonstration and adaptation of biped locomotion," Robot. Autonom. Syst., vol. 47, no. 2-3, pp. 79-91, 2004.
20. R.M. Neal, "Bayesian learning for neural networks," in Lecture Notes in Statistics, vol. 118. New York: Springer-Verlag, 1996.
21. M.H. Raibert, Legged Robots That Balance. Cambridge, MA: MIT Press, 1986.
22. S. Schaal and C.G. Atkeson, "Constructive incremental learning from only local information," Neural Comput., vol. 10, no. 8, pp. 2047-2084, 1998.
23. S.H. Strogatz, Nonlinear Dynamics and Chaos. Reading, MA: Addison-Wesley, 1994.
24. R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction Cambridge, MA: MIT Press, 1998.
25. R.S. Sutton, D. McAllester, S. Singh, and Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation," in Advances in Neural Information Processing Systems, vol. 12. Cambridge, MA: MIT Press, 2000, pp. 1057-1063.
26. R.S. Sutton, D. Precup, and S. Singh, "Between MDPs and semi-MDPs: A Framework for temporal abstraction in reinforcement learning," Artif. Intell., vol. 112, no. 1-2, pp. 181-211, 1999.
27. R. Tedrake, T.W. Zhang, and H.S. Seung, "Stochastic policy gradient reinforcement learning on a simple 3-D biped," in Proc. 2004 IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2004, pp. 2849-2854.
28. K. Tsuchiya, S. Aoi, and K. Tsujita, "Locomotion control of a biped locomotion robot using nonlinear oscillators," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, Las Vegas, NV, 2003, pp. 1745-1750.
29. Y. Wada and M. Kawato, "A theory for cursive handwriting based on the minimization principle," Biol. Cybern., vol. 73, pp. 3-15, 1995.
30. E.R. Westervelt, G. Buche, and J.W. Grizzle, "Experimental validation of a framework for the design of controllers that induce stable walking in planar bipeds," Int. J. Robot. Res., vol. 23, no. 6, pp. 559-582, 2004.