A hierarchical reinforcement learning approach for optimal path tracking of wheeled mobile robots

Neural Computing and Applications

Volumn 23, Issue 7-8, 2013, Pages 1873-1883

  Zuo, Lei ^a   Xu, Xin ^a   Liu, Chunming ^a   Huang, Zhenhua ^a  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

Graph Laplacian; Learning control; Mobile robots; Reinforcement learning

Indexed keywords

AUTONOMOUS MOBILE ROBOT; CONTINUOUS STATE SPACE; GENERALIZATION ABILITY; GRAPH LAPLACIAN; HIERARCHICAL REINFORCEMENT LEARNING; LEARNING CONTROL; MARKOV DECISION PROCESSES; PROPORTIONAL-DERIVATIVE CONTROL;

ITERATIVE METHODS; LAPLACE TRANSFORMS; LEARNING ALGORITHMS; MARKOV PROCESSES; MATHEMATICAL OPERATORS; OPTIMIZATION; REINFORCEMENT LEARNING;

MOBILE ROBOTS;

EID: 84887486066     PISSN: 09410643     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00521-012-1243-4     Document Type: Article

References (41)

1. Oh SY, Lee JH et al (2000) A new reinforcement learning vehicle control architecture for vision-based road following [J]. IEEE Trans Veh Technol 49(3): 997-1005.
2. Yamaguchi T, Sato E et al (2003) Intelligent space and human centered robotics [J]. IEEE Trans Ind Electron 50(5): 881-889.
3. Lee JM, Son K et al (2003) Localization of a mobile robot using the image of a moving object [J]. IEEE Trans Ind Electron 50(3): 612-619.
4. Lee TC, Tsai CY et al (2004) Fast parking control of mobile robots: a motion planning approach with experimental validation [J]. IEEE Trans Control Syst Technol 12(5): 661-676.
5. Palacin J, Salse JA et al (2004) Building a mobile robot for a floor-cleaning operation in domestic environments [J]. IEEE Trans Instrum Meas 53(5): 1418-1424.
6. Ding D, Cooper RA (2005) Electric-powered wheelchairs: a review of current technology and insight into future direction [J]. IEEE Control Syst Mag 25(2): 22-34.
7. Shim HS, Sung YG (2004) Stability and four-posture control for nonholonomic mobile robots [J]. IEEE Trans Robot Autom 20(1): 148-154.
8. Zhao DB, Deng XY, Yi JQ (2009) Motion and internal force control for omni-directional wheeled mobile robots [J]. IEEE ASME Trans Mechatron 14(3): 382-387.
9. Wu Y, Wang B et al (2005) Finite-time tracking controller design for nonholonomic systems with extended chained form[J]. IEEE Trans Circuit Syst II Exp Briefs 52(11): 798-802.
10. Antonelli G, Chiaverini S et al (2007) A fuzzy-logic-based approach for mobile robot path tracking[J]. IEEE Trans Fuzzy Syst 15(2): 211-221.
11. Raffo GV, Gomes GK et al (2009) A predictive controller for autonomous vehicle path tracking[J]. IEEE Trans Intell Transp Syst 10(1): 92-102.
12. Wai R, Liu C (2009) Design of dynamic petri recurrent fuzzy neural network and its application to path-tracking control of nonholonomic mobile robot[J]. IEEE Trans Ind Electron 56(7): 2667-2683.
13. Mohareri O, Dhaouadi R et al (2012) Indirect adaptive tracking control of a nonholonomic mobile robot via neural networks[J]. Neurocomputing 88: 54-66.
14. Aguiar AP, Hespanha JP (2007) Trajectory-tracking and path-following of underactuated autonomous vehicles with parametric modeling uncertainty[J]. IEEE Trans Autom Cont 52(8): 1362-1379.
15. Xu D, Zhao DB, Yi JQ, Tan XM (2009) Trajectory tracking control of omnidirectional wheeled mobile manipulators: robust neural network based sliding mode approach [J]. IEEE Trans Syst Man Cybern Part B 39(3): 788-799.
16. Park BS, Yoo SJ et al (2010) A simple adaptive control approach for trajectory tracking of electrically driven nonholonomic mobile robots[J]. IEEE Trans Control Syst Technol 18(5): 1199-1206.
17. Sutton R, Barto AG (1998) Reinforcement learning: an introduction[M]. The MIT Press, Cambridge.
18. Zhang Q, Li M, Wang XS, Zhang Y (2012) Reinforcement learning in robot path optimization [J]. J Softw 7(3): 657-662.
19. Zhang PC, Xu X, Liu C, Yuan Q (2009) Reinforcement learning control of a real mobile robot using approximate policy iteration [C]. ISNN 2009, Part III, Lecture Notes in Computer Science, LNCS 5553, pp 278-288.
20. Yen GG, Hickey TW (2004) Reinforcement learning algorithms for robotic navigation in dynamic environments. ISA Trans 43: 217-230.
21. Wang FY, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction [J]. IEEE Comput Intell Mag 4(2): 39-47.
22. Sutton R (1996) Generalization in reinforcement learning: successful examples using sparse coarse coding[C]. In: Advances in Neural Information Processing Systems 8 (Proceedings of the 1995 conference). MIT Press, pp 1038-1044.
23. Xu X, He H et al (2002) Efficient reinforcement learning using recursive least-squares methods[J]. J Art Intell Res 16: 259-292.
24. Lagoudakis MG, Parr R (2003) Least-squares policy Iteration[J]. J Mach Learn Res 4: 1107-1149.
25. Liu D, Javaherian H, Kovalenko O, Huang T (2008) Adaptive critic learning techniques for engine torque and air-fuel ratio control [J]. IEEE Trans Syst Man Cybern Part B Cybern 38(4): 988-993.
26. Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9): 1490-1503.
27. Zhang HG, Wei QL, Liu D (2011) An iterative approximate dynamic programming method to solve for a class of nonlinear zero-sum differential games. Automatica 47(1): 207-214.
28. Yang Q, Jagannathan S (2007) Online reinforcement learning neural network controller design for nanomanipulation. In: Proceedings of IEEE symposium on approximate dynamic programming and reinforcement learning. Honolulu, HI, pp 225-232.
29. Xu X, Hu DW et al (2007) Kernel-based least squares policy iteration for reinforcement learning[J]. IEEE Trans Neural Netw 18(4): 973-992.
30. Bernhard H (2003) Discovering hierarchy in reinforcement learning[D]. University of New South Wales, Australia.
31. Doina SRSP et al (1999) Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning[J]. Artif Intell 112: 181-211.
32. Parr R (1998) Hierarchical control and learning for Markov decision processes[D]. University of California, Berkeley, California.
33. Dietterich TG (2000) Hierarchical reinforcement learning with the MAXQ value function decomposition[J]. J Art Intell Res 13: 227-303.
34. Xu X, Liu C et al (2011) Hierarchical approximate policy iteration with binary-tree state space decomposition[J]. IEEE Trans Neural Netw 22(12): 1863-1877.
35. Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, Belmont, Massachusetts.
36. Vapnik V (1998) Statistical learning theory[M]. John Wiley and Sons, Inc., New York.
37. Mahadevan S, Maggioni M (2007) Proto-value functions: a Laplacian framework for learning representation and control in Markov decision processes[J]. J Mach Learn Res 8: 2169-2231.
38. Mahadevan S (2008) Learning representation and control in Markov decision processes: new Frontiers[J]. Found Trends Mach Learn 1(4): 403-565.
39. Normey-Rico JE, Alcalab I et al (2001) Mobile robot path tracking using a robust PID controller[J]. Control Eng Pract 9: 1209-1214.
40. Mahadevan S, Maggioni M (2006) Value function approximation with diffusion wavelets and Laplacian eigenfunctions[C]. In: Proceedings of the neural information processing systems (NIPS). MIT Press.
41. Munos R (2003) Error bounds for approximate policy iteration. In: Proceedings of the 20th annual international conference machine learning. p 560.