Robust quantum-inspired reinforcement learning for robot navigation

IEEE/ASME Transactions on Mechatronics

Volumn 17, Issue 1, 2012, Pages 86-97

  Dong, Daoyi ^a,d   Chen, Chunlin ^b   Chu, Jian ^a   Tarn, Tzyh Jong ^c  

^a STATE KEY LABORATORY OF INDUSTRIAL CONTROL TECHNOLOGY   (China)

^b NANJING UNIVERSITY   (China)

^c WASHINGTON UNIVERSITY IN ST LOUIS   (United States)

^d UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Probabilistic action selection; quantum amplitude amplification; quantum inspired reinforcement learning (QiRL); robot navigation

Indexed keywords

ACTION SELECTION; AUTONOMOUS MOBILE ROBOT; INITIAL STATE; LEARNING RATES; MARKOVIAN; NAVIGATION CONTROLS; QUANTUM AMPLITUDE AMPLIFICATION; QUANTUM MEASUREMENT; QUANTUM-INSPIRED REINFORCEMENT LEARNING (QIRL); REINFORCEMENT STRATEGIES; ROBOT NAVIGATION; SIMULATED EXPERIMENTS; STATE TRANSITIONS;

ALGORITHMS; AMPLIFICATION; COMPUTATIONAL LINGUISTICS; MOBILE ROBOTS; NAVIGATION; NAVIGATION SYSTEMS; QUANTUM COMPUTERS; QUANTUM THEORY; REINFORCEMENT;

REINFORCEMENT LEARNING;

EID: 84855964266     PISSN: 10834435     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMECH.2010.2090896     Document Type: Article

References (46)

1. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information. Cambridge, U.K.: Cambridge Univ. Press, 2000.
2. P. Benioff, "Quantum robots and environments," Phys. Rev. A, vol. 58, pp. 893-904, 1998. (Pubitemid 128602968)
3. D. Y. Dong, C. L. Chen, C. B. Zhang, and Z. H. Chen, "Quantum robot: Structure, algorithms and applications," Robotica, vol. 24, pp. 513-521, 2006.
4. S. Kak, "On quantum neural computing," Inf. Sci., vol. 83, pp. 143-160, 1995.
5. A. Narayanan and T. Menneer, "Quantum artificial neural network architectures and components," Inf. Sci., vol. 128, pp. 231-255, 2000.
6. G. G. Rigatos and S. G. Tzafestas, "Parallelization of a fuzzy control algorithm using quantum computation," IEEE Trans. Fuzzy Syst., vol. 10, no. 4, pp. 451-460, Aug. 2002. (Pubitemid 34950050)
7. A. Malossini, E. Blanzieri, and T. Calarco, "Quantum genetic optimization," IEEE Trans. Evol. Comput., vol. 12, no. 2, pp. 231-241, Apr. 2008. (Pubitemid 351551665)
8. K. H. Han and J. H. Kim, "Quantum-inspired evolutionary algorithm for a class of combinatorial optimization," IEEE Trans. Evol. Comput., vol. 6, no. 6, pp. 580-593, Dec. 2002.
9. D. Dong, C. Chen, H. Li, and T. J. Tarn, "Quantum reinforcement learning," IEEE Trans. Syst.,Man, Cybern. B, Cybern., vol. 38, no. 5, pp. 1207-1220, Oct. 2008.
10. D. Dong, C. Chen, T. J. Tarn, A. Pechen, and H. Rabitz, "Incoherent control of quantum systems with wavefunction controllable subspaces via quantum reinforcement learning," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 957-962, Aug. 2008.
11. D. Dong, C. Zhang, H. Rabitz, A. Pechen, and T. J. Tarn, "Incoherent control of locally controllable quantum systems," J.Chem. Phys., vol. 129, no. 15, pp. 154103-1-154103-10, 2008.
12. G. Brassard, P.Høyer, and A. Tapp, "Quantum counting," in Proc. 25th Int. Colloq. Automata, Lang. Program., Lecture Notes in Computer Science, 1998.
13. D. Dong, C. Chen, and H. Li, "Reinforcement strategy using quantum amplitude amplification for robot learning," in Proc. 26th Chinese Control Conf., Zhangjiajie, China, 2007, vol. 6, pp. 571-575.
14. D. Dong and C. Chen, Quantum-inspired reinforcement learning for decision-making of Markovian state transition," presented at the 2010 Int. Conf. Intell. Syst. Knowl. Eng., Hangzhou, China, Nov. 15-16, 2010.
15. C. Chen, H. Li, and D. Dong, "Hybrid control for robot navigation: A hierarchical Q-learning algorithm," IEEE Robot. Autom. Mag., vol. 15, no. 2, pp. 37-47, Jun. 2008.
16. S. Park and S. Hashimoto, "Autonomous mobile robot navigation using passive RFID in indoor environment," IEEE Trans. Ind. Electron., vol. 56, no. 7, pp. 2366-2373, Jul. 2009.
17. J. A. Fernandez-Leon, G. G. Acosta, and M. A. Mayosky, "Behavioral control through evolutionary neurocontrollers for autonomous mobile robot navigation," Robot. Auton. Syst., vol. 57, no. 4, pp. 411-419, 2009.
18. S. Shair, J. H. Chandler, V. J. Gonzalez-Villela et al., "The use of aerial images and GPS for mobile robot waypoint navigation," IEEE/ASME Trans. Mechatronics, vol. 13, no. 6, pp. 692-699, Dec. 2008.
19. A. Franchi, L. Freda, G. Oriolo et al., "The sensor-based random graph method for cooperative robot exploration," IEEE/ASME Trans. Mechatronics, vol. 14, no. 2, pp. 163-175, Apr. 2009.
20. C. Chen and D. Dong, "Grey system based reactive navigation of mobile robots using reinforcement learning," Int. J. Innov. Comput., Inf. Control, vol. 6, no. 2, pp. 789-800, 2010.
21. G. E. Jan, K. Y. Chang, and I. Parberry, "Optimal path planning for mobile robot navigation," IEEE/ASME Trans. Mechatronics, vol. 13, no. 4, pp. 451-460, Aug. 2008.
22. T. W. Manikas, K. Ashenayi, and R. L. Wainwright, "Genetic algorithms for autonomous robot navigation," IEEE Instrum. Meas. Mag., vol. 10, no. 6, pp. 26-31, Dec. 2007. (Pubitemid 351284869)
23. M. F. Selekwa, D. D. Dunlap, D. Shi, and E. G. Collins, "Robot navigation in very cluttered environments by preference-based fuzzy behaviors," Robot. Auton. Syst., vol. 56, no. 3, pp. 231-246, 2008. (Pubitemid 351273574)
24. M. A. Salichs and L. Moreno, "Navigation of mobile robot: Open questions," Robotica, vol. 18, pp. 227-234, 2000.
25. M. Rodriguez, R. Iglesias, C. V. Regueiro, J. Correa, and S. Barro, "Autonomous and fast robot learning through motivation," Robot. Auton. Syst., vol. 55, pp. 735-740, 2007. (Pubitemid 47285710)
26. N. D. Ratliff, D. Silver, and J. A. Bagnell, "Learning to search: Functional gradient techniques for imitation learning," Auton. Robots, vol. 27, pp. 25-53, 2009.
27. T. Kondo and K. Ito, "A reinforcement learning with evolutionary state recruitment strategy for autonomous mobile robots control," Robot. Auton. Syst., vol. 46, pp. 111-124, 2004.
28. R. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
29. R. Sutton, "Learning to predict by the methods of temporal difference," Mach. Learn., vol. 3, pp. 9-44, 1988.
30. C.Watkins and P. Dayan, "Q-learning," Mach. Learn., vol. 8, pp. 279-292, 1992.
31. D.Vengerov, N. Bambos, and H. Berenji, "A fuzzy reinforcement learning approach to control in wireless transmitters," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 4, pp. 768-778, Aug. 2005. (Pubitemid 41644059)
32. Y.Wang and C.W. de Silva, "Sequential Q-learning with Kalman filtering for multirobot cooperative transportation," IEEE/ASME Trans. Mechatronics, vol. 15, no. 2, pp. 261-268, Apr. 2010.
33. T. G. Dietterich, "Hierarchical reinforcement learning with theMaxq value function decomposition," J. Artif. Intell. Res., vol. 13, pp. 227-303, 2000. (Pubitemid 33682087)
34. M. J. Er and C. Deng, "Online tuning of fuzzy inference systems using dynamic fuzzy Q-learning," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 3, pp. 1478-1489, Jun. 2004.
35. S. Whiteson and P. Stone, "Evolutionary function approximation for reinforcement learning," J. Mach. Learn. Res., vol. 7, pp. 877-917, 2006. (Pubitemid 43736560)
36. M. Kaya and R. Alhajj, "A novel approach to multiagent reinforcement learning: Utilizing OLAP mining in the learning process," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 35, no. 4, pp. 582-590, Nov. 2005. (Pubitemid 41638177)
37. S. G. Tzafestas and G. G. Rigatos, "Fuzzy reinforcement learning control for compliance tasks of robotic manipulators," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 32, no. 1, pp. 107-113, Feb. 2002. (Pubitemid 34228677)
38. P. Høyer, "Arbitrary phases in quantum amplitude amplification," Phys. Rev. A, vol. 62, pp. 052304-1-052304-5, 2000.
39. G. Brassard and P. Høyer, "An exact quantum polynomial-time algorithm for Simon's problem," in Proc. 5th Israeli Symp. Theory Comput. Syst., Los Alamitos, CA, 1997, pp. 12-23.
40. L. K. Grover, "Quantum mechanics helps in searching for a needle in a haystack," Phys. Rev. Lett., vol. 79, pp. 325-327, 1997.
41. G. L. Long, "Grover algorithm with zero theoretical failure rate," Phys. Rev. A, vol. 64, pp. 022307-1-022307-4, 2001.
42. D. Dong and I. R. Petersen, "Sliding mode control of quantum systems," New J. Phys., vol. 11, pp. 105033-1-105033-18, 2009.
43. L. K. Grover, "Quantum computers can search rapidly by using almost any transformation," Phys. Rev. Lett., vol. 80, pp. 4329-4332, 1998. (Pubitemid 128621914)
44. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
45. A. Alvarez, A. Caiti, and R. Onken, "Evolutionary path planning for autonomous underwater vehicles in a variable ocean," IEEE J. Ocean. Eng., vol. 29, no. 2, pp. 418-429, Apr. 2004.
46. S. Thrun, "Learning metric-topological maps for indoor mobile robot navigation," Artif. Intell., vol. 99, pp. 21-71, 1998. (Pubitemid 128378368)