Gait Balance and Acceleration of a Biped Robot Based on Q-Learning

IEEE Access

Volumn 4, Issue , 2016, Pages 2439-2449

  Lin, Jin Ling ^a   Hwang, Kao Shing ^b   Jiang, Wei Cheng ^b   Chen, Yu Jen ^c  

^a SHIH HSIN UNIVERSITY   (Taiwan)

^b NATIONAL SUN YAT SEN UNIVERSITY   (Taiwan)

^c NATIONAL CHUNG CHENG UNIVERSITY   (Taiwan)

Author keywords

Biped Robot; Continuous Action Space; Reinforcement Learning; Zero Moment Point

Indexed keywords

MEMORY ARCHITECTURE; ROBOTICS; ROBOTS;

ACTUATION CONTROL; BIPED ROBOT; COMPLEX CONTROL PROBLEMS; CONTINUOUS ACTIONS; LEARNING ARCHITECTURES; LEARNING METHODS; PRIORI KNOWLEDGE; ZERO MOMENT POINT;

REINFORCEMENT LEARNING;

EID: 84979846290     PISSN: None     EISSN: 21693536     Source Type: Journal    
DOI: 10.1109/ACCESS.2016.2570255     Document Type: Article

References (24)

1. H. Lingyun and S. Zengqi, "Reinforcement learning method-based stable gait synthesis for biped robot," in Proc. 8th Int. Conf. Control, Autom., Robot. Vis., vol. 2. Dec. 2004, pp. 1017-1022.
2. K.-S. Hwang, J.-L. Lin, and K.-H. Yeh, "Learning to adjust and refine gait patterns for a biped robot," IEEE Trans. Syst., Man, Cybern., Syst., vol. 45, no. 12, pp. 1481-1490, Dec. 2015.
3. K.-S. Hwang, J.-L. Lin, T.-C. Huang, and H.-J. Hsu, "Humanoid robot gait imitation," in Proc. SICE Annu. Conf. (SICE), Sep. 2014, pp. 2124-2128.
4. T.-H. S. Li, Y.-T. Su, S.-W. Lai, and J.-J. Hu, "Walking motion gener- ation, synthesis, and control for biped robot by using PGRL, LPI, and fuzzy logic," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 3, pp. 736-748, Jun. 2011.
5. T. Suzuki, T. Tsuji, and K. Ohnishi, "Trajectory planning of biped robot for running motion," in Proc. 32nd IEEE Conf. Ind. Electron. Soc., Nov. 2005, pp. 1815-1820.
6. D. Tlalolini, C. Chevallereau, and Y. Aoustin, "Human-like walking: Optimal motion of a bipedal robot with toe-rotation motion," IEEE/ASME Trans. Mechatronics, vol. 16, no. 2, pp. 310-320, Apr. 2011.
7. K.-C. Choi, H.-J. Lee, and M. C. Lee, "Fuzzy posture control for biped walking robot based on force sensor for ZMP," in Proc. SICE-ICASE Int. Joint Conf., Oct. 2006, pp. 1185-1189.
8. K. Suwanratchatamanee, M. Matsumoto, and S. Hashimoto, "Balance control of robot and human-robot interaction with haptic sensing foots," in Proc. 2nd Conf. Human Syst. Interact. (HSI), May 2009, pp. 68-74.
9. S. N. Napoleon, S. Nakaura, and M. Sampei, "Balance control analysis of humanoid robot based on ZMP feedback control," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., vol. 3. 2002, pp. 2437-2442.
10. T. Sato, H. Ono, and K. Ohnishi, "Gyroscope integrated environmental mode compliance control for biped robot," in Proc. IEEE Int. Workshop Adv. Motion Control, Mar. 2012, pp. 1-6.
11. L. Wang, Z. Liu, C. L. P. Chen, Y. Zhang, S. Lee, and X. Chen, "A UKF-based predictable SVR learning controller for biped walking," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 43, no. 6, pp. 1440-1450, Nov. 2011.
12. Y. Yoshida, K. Takeuchi, Y. Miyamoto, D. Sato, and D. Nenchev, "Postural balance strategies in response to disturbances in the frontal plane and their implementation with a humanoid robot," IEEE Trans. Syst., Man, Cybern., Syst., vol. 44, no. 6, pp. 692-704, Jun. 2014.
13. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 1998.
14. T.-H. S. Li, P.-H. Kuo, Y.-F. Ho, M.-C. Kao, and L.-H. Tai, "A biped gait learning algorithm for humanoid robots based on environmental impact assessed artificial bee colony," IEEE Access, vol. 3, pp. 13-26, 2015.
15. V. Gullapalli, "A stochastic reinforcement learning algorithm for learning real-valued functions," Neural Netw., vol. 3, no. 6, pp. 671-692, 1990.
16. V. Gullapalli, "Associative reinforcement learning of real-valued functions," in Proc. IEEE Int. Conf. Syst., Man, Cybern., vol. 3. Charlottesville, VA, USA, Oct. 1991, pp. 1453-1458.
17. C. J. C. H. Watkins, "Learning from delayed rewards," Ph.D. dissertation, Dept. Psychol., Univ. Cambridge, Cambridge, U.K., 1989.
18. C. J. C. H. Watkins and P. Dayan, "Q-learning," Mach. Learn., vol. 8, no. 3, pp. 279-292, May 1992.
19. C. F. Touzet, "Q-learning for robot," in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, MA, USA: MIT Press, 2003, pp. 934-937.
20. K.-S. Hwang, H.-Y. Lin, Y.-P. Hsu, and H.-H. Yu, "Self-organizing state aggregation for architecture design of Q-learning," Inf. Sci., vol. 181, no. 13, pp. 2813-2822, Jul. 2011.
21. W. T. B. Uther and M. M. Veloso, "Tree based discretization for continuous state space reinforcement learning," in Proc. 15th Nat. Conf. Artif. Intell. (AAAI), Madison, WI, USA, 1998, pp. 769-774.
22. L. D. Pyeatt and A. E. Howe, "Decision tree function approximation in reinforcement learning," in Proc. 3rd Int. Symp. Adapt. Syst., Evol. Comput. Probabilistic Graph. Models, 1998, pp. 70-77.
23. K.-S. Hwang, J.-L. Lin, and J.-S. Li, "Biped balance control by reinforce- ment learning," J. Inf. Sci. Eng., to be published.
24. IRIS. (Feb. 11, 2014). Biped Robot Walk and Experience Transfer Demo. IRIS Lab., National Sun Yat-sen University, Kaohsiung, Taiwan, accessed Nov. 16, 2015. [Online]. Available: https://www.youtube.com/ watch?v=mVahCHBFWyo