Hybrid control for robot navigation - A hierarchical Q-learning algorithm

IEEE Robotics and Automation Magazine

Volumn 15, Issue 2, 2008, Pages 37-47

  Chen, Chunlin ^a,b   Li, Han Xiong ^b   Dong, Daoyi ^c  

^a NANJING UNIVERSITY   (China)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^c INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

[No Author keywords available]

Indexed keywords

BOOLEAN FUNCTIONS; EDUCATION; LEARNING SYSTEMS; MAPS; MULTI AGENT SYSTEMS; OPTICAL PROJECTORS; TOPOLOGY;

(E ,3E) PROCESS; CON TROL ALGORITHMS; CONTROL APPROACHES; GRID TOPOLOGICAL; HYBRID CONTROL; HYBRID CONTROL ARCHITECTURE; INTEGRATED LEARNING; MARKOV DECISION PROCESS (MDP); Q-LEARNING ALGORITHMS; ROBOT NAVIGATION;

LEARNING ALGORITHMS;

EID: 46349107239     PISSN: 10709932     EISSN: None     Source Type: Journal    
DOI: 10.1109/MRA.2008.921541     Document Type: Article

References (30)

1. M. A. Salichs and L. Moreno, "Navigation of mobile robot: Open questions," Robotica, vol. 18, no. 3, pp. 227-234, 2000.
2. Y. K. Na and S. Y. Oh, "Hybrid control for autonomous mobile robot navigation using neural network based behavior modules and environment classification," Autonom. Robots, vol. 15, no. 2, pp. 193-206, 2003.
3. J. Borenstein and Y. Koren, "Real-time obstacle avoidance for fast mobile robot," IEEE Trans. Syst., Man, Cybern. B, vol. 19, no. 5, pp. 1179-1187, 1989.
4. A. Elfes, "Sonar based real world mapping and navigation," IEEE J. Robot. Automat., vol. RA-3, no. 3, pp. 249-265, 1987.
5. R. A. Brooks, "A robust layered control system for a mobile robot," IEEE J. Robot. Automat., vol. 2, no. 1, pp. 14-23, 1986.
6. A. M. Zhu, S. X. Yang, F. J. Wang, and G. S. Mittal, "A neuro-fuzzy controller for reactive navigation of a behavior-based mobile robot," Lect. Notes Comput. Sci., vol. 3498, no. 5, pp. 259-264, 2005.
7. S. Thrun, "Probabilistic algorithms in robotics," AI Mag., vol. 21, no. 4, pp. 93-109, 2000.
8. T. Kondo and K. Ito, "A reinforcement learning with evolutionary state recruitment strategy for autonomous mobile robots control," Robot. Autonom. Syst., vol. 46, no. 2, pp. 111-124, 2004.
9. C. L. Chen and Z. H. Chen, "Reinforcement learning for mobile robot: From reaction to deliberation," J. Syst. Eng. Electron., vol. 16, no. 3, pp. 611-617, 2005.
10. E. Remolina and B. Kuipers, "Towards a general theory of topological maps," Artif. Intell., vol. 152, no. 2, pp. 47-104, 2004.
11. S. Thrun, "Learning metric-topological maps for indoor mobile robot navigation," Artif. Intell., vol. 99, no. 1, pp. 21-71, 1998.
12. P. Ranganathan, J. B. Hayet, M. Devy, S. Hutchinson, and F. Lerasle, "Topological navigation and qualitative localization for indoor environment using multi-sensory perception," Robot. Autonom. Syst. vol. 41, nos. 2-3, pp. 137-144, 2002.
13. C. L. Chen, D. Y. Dong, H. X. Li, and T J. Tarn, "An Integrated Hierarchical Q-learning Method Based on Hybrid MDP," IEEE Transactions on Systems, Man, and Cybernetics B, under review, 2008.
14. J. C. H. Watkins and P. Dayan, "Q-Learning," Mach. Learn., vol. 8, nos. 3-4, pp. 279-292, 1992.
15. R. Sutton and A. G. Barto, Reinforcement Learning: An Introduction Cambridge, MA: MIT Press, 1998.
16. M. Hallerdal and J. Hallamy "Behavior selection on a mobile robot using W-learning," in Proc. 7th Int. Conf. Simulation of Adaptive Behavior on From Animals to Animates, Edinburgh, 2002, pp. 93-102.
17. M. Wiering and J. Schmidhuber, "HQ-Learning," Adapt. Behav., vol. 6, no. 2, pp. 219-246, 1997.
18. A. G. Barto and S. Mahadevan, "Recent advances in hierarchical reinforcement learning," Discrete Event Dyn. Syst., vol. 13, nos. 1-2, pp. 41-77, 2003.
19. R. Sutton, D. Precup, and S. Singh, "Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning," Artif. Intell., vol. 112, no. 1, pp. 181-211, 1999.
20. R. Parr and S. Russell, "Reinforcement learning with hierarchies of machines," Adv. Neural Inform. Process. Syst., vol. 10, no. 10, pp. 1043-1049, 1998.
21. T. G. Dietterich, "Hierarchical reinforcement learning with the Maxq value function decomposition," J. Artif. Intell. Res., vol. 13, no. 1, pp. 227-303, 2000.
22. C. L. Chen, D. Y. Dong, and Z. H. Chen, "Grey reinforcement learning for incomplete information processing," Lect. Notes Comput. Sci., vol. 3959, no. 1, pp. 399-407, 2006.
23. D. Berleant and B. Kuipers, "Qualitative and quantitative simulation: Bridging the gap," Artif. Intell., vol. 95, no. 2, pp. 215-255, 1997.
24. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
25. D. Pagac, E. M. Nebot, and H. Durrant-Whyte, "An evidential approach to map-building for autonomous vehicles," IEEE Trans. Robot. Automat. vol. 14, no. 4, pp. 623-629, 1998.
26. C. C. Kung and J. Y. Su, "Affine Takagi-Sugeno fuzzy modelling algorithm by fuzzy c-regression models clustering with a novel cluster validity criterion," IET Contr. Theor. Appl., vol. 1, no. 5, pp. 1255-1265, 2007.
27. M. Mata, J. M. Armingol, and A. Escalera, "Learning visual landmarks for mobile robot topological navigation," Stud. Comput. Intell., vol. 7, no. 1, pp. 1-55, 2005.
28. B. Kuipers, "Cognitive maps for planetary rovers," Autonom. Robots vol. 11, no. 3, pp. 325-331, 2001.
29. N. M. Sgouros, "Qualitative navigation for autonomous wheelchair robots in indoor environments," Autonom. Robots, vol. 12, no. 3, pp. 257-266, 2002.
30. D. Busquets, C. Sierra, and R. L. D. Mantaras, "A multi-agent approach to qualitative landmark-based navigation," Autonom. Robots, vol. 15, no. 2, pp. 129-154, 2003.