An approach to tune fuzzy controllers based on reinforcement learning for autonomous vehicle control

IEEE Transactions on Intelligent Transportation Systems

Volumn 6, Issue 3, 2005, Pages 285-293

  Dai, Xiaohui ^b   Li, Chi Kwong ^a,c,d   Rad, A B ^a,c  

^a IEEE   (Hong Kong)

^b Rockwell Automation   (China)

^c HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^d Rockwell Automation Shanghai Research Center ^*  (United Kingdom)

Author keywords

Autonomous vehicles; Fuzzy controllers; Longitudinal control; Reinforcement learning

Indexed keywords

AUTONOMOUS VEHICLE CONTROL; FUZZY CONTROLLERS; REINFORCEMENT LEARNING;

ALGORITHMS; COMPUTER SIMULATION; FUZZY CONTROL; LEARNING SYSTEMS; VEHICLES;

CONTROL EQUIPMENT;

EID: 27744536933     PISSN: 15249050     EISSN: None     Source Type: Journal    
DOI: 10.1109/TITS.2005.853698     Document Type: Article

References (36)

1. J. E. Naranjo, C. Gonzalez, J. Reviejo, R. Garcia, and T. De Pedro, "Adaptive fuzzy control for inter-vehicle gap keeping," IEEE Trans. Intell. Transp. Syst., vol. 4, no. 3, pp. 132-142, Sep. 2003.
2. L.-X. Wang, A Course in Fuzzy System and Control. Upper Saddle River, NJ: Prentice-Hall, 1997.
3. _, "Stable adaptive fuzzy control of nonlinear systems," IEEE Trans. Fuzzy Syst., vol. 1, no. 2, pp. 146-155, May 1993.
4. R. S. Sutton, A. G. Barto, and R. J. Williams, "Reinforcement learning is direct adaptive optimal control," IEEE Control Syst. Mag., vol. 12, no. 2, pp. 19-22, Apr. 1992.
5. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
6. C.-K. Chiang, H.-Y. Chung, and J.-J. Lin, "A self-learning fuzzy logic controller using genetic algorithms with reinforcements," IEEE Trans. Fuzzy Syst., vol. 5, no. 3, pp. 460-467, Jun. 1997.
7. C. J. C. H. Watkins, "Learning with delayed rewards," Ph.D. dissertation, Psychology Dept., Cambridge Univ., U.K., 1989.
8. 2 function and its error bounds using regular-center Gaussian networks," IEEE Trans. Neural Netw., vol. 5, no. 5, pp. 845-847, Sep. 1994.
9. D. Swaroop, J. K. Hedrick, and S. B. Choi, "Direct adaptive longitudinal control of vehicle platoons," IEEE Trans. Veh. Technol., vol. 50, no. 1, pp. 150-161, Jan. 2001.
10. X. W. Yan, Z. D. Deng, and Z. Q. Sun, "Competitive Takagi-Sugeno fuzzy reinforcement learning," in Proc. IEEE Int. Conf. Control Applications, San Diego, CA, 2001, pp. 878-883.
11. W. D. Smart and L. P. Kaelbling, "Practical reinforcement learning in continuous spaces," in Proc. 17th Int. Conf. Machine Learning, Stanford, CA, 2000, pp. 903-910.
12. J. A. Boyan and A. W. Moore, "Generalization in reinforcement learning: Safely approximating the value function," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. Cambridge, MA: MIT Press, 1995.
13. G. Tesauro, "TD-Gammon, a self teaching backgammon program, achieves master-level play," Neural Comput., vol. 6, no. 2, pp. 215-219, 1994.
14. A. G. Barto and M. I. Jordan, "Gradient following without backpropagation in layered networks," in Proc. IEEE 1st Annu. Conf. Neural Networks, San Diego, CA, 1987, pp. II629-II636.
15. C.-J. Lin and C.-T. Lin, "Reinforcement learning for an ART-based fuzzy adaptive learning control network," IEEE Trans. Neural Netw., vol. 7, no. 3, pp. 709-731, May 1996.
16. C.-T. Lin and I.-F. Chung, "A reinforcement neuro-fuzzy combiner for multiobjective control," IEEE Trans. Syst., Man, Cybern. B, vol. 29, no. 6, pp. 726-744, Dec. 1999.
17. S.-Y. Oh, J.-H. Lee, and D.-H. Choi, "A new reinforcement learning vehicle control architecture for vision-based road following," IEEE Trans. Veh. Technol., vol. 49, no. 3, pp. 997-1005, May 2000.
18. C. J. C. H. Watkins, "Learning from delayed rewards," Ph.D. dissertation, Psychology Dept., Cambridge Univ., U.K., 1989.
19. C.-T. Lin and C. S. G. Lee, "Reinforcement structure/parameter learning for neural-network-based fuzzy logic control systems," IEEE Trans. Fuzzy Syst., vol. 2, no. 1, pp. 46-63, Feb. 1994.
20. J. Si and Y.-T. Wang, "Online learning control by association and reinforcement," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001.
21. Y. Koike and K. Doya, "Multiple state estimation reinforcement learning for driving model: Driver model of automobile," in Proc. IEEE Int. Conf. Systems, Man, and Cybernetics (SMC), Tokyo, Japan, 1999, vol. 5, pp. 504-509.
22. L. Jouffe, "Fuzzy inference system learning by reinforcement methods," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 28, no. 3, pp. 338-355, Aug. 1998.
23. V. Gullapalli, "A stochastic reinforcement learning algorithm for learning real-valued functions," Neural Netw., vol. 3, no. 6, pp. 671-692, 1990.
24. L. C. Baird, III, "Reinforcement learning in continuous time: Advantage updating," in IEEE World Congr. Computational Intelligence., IEEE Int. Conf. Neural Networks, Orlando, FL, 1994, vol. 4, pp. 2448-2453.
25. E. Zalama, J. Gomez, M. Paul, and J. R. Peran, "Adaptive behavior navigation of a mobile robot," IEEE Trans. Syst., Man, Cybern. A, vol. 32, no. 1, pp. 160-169, Jan. 2002.
26. M. H. Hebert, C. E. Thorpe, and A. Stentz, Intelligent Unmanned Ground Vehicles: Autonomous Navigation Research at Carnegie Mellon. Norwell, MA: Kluwer, 1997.
27. S. Shladover et al., "Automated vehicle control developments in the PATH program," IEEE Trans. Veh. Technol., vol. 40, no. 1, pp. 114-130, Feb. 1991.
28. M. Taniguchi et al., "The development of autonomously controlled vehicle (PVS)," in Proc. Vehicle Navigation and Information Syst. Conf., Dearborn, MI, 1991, pp. 1137-1141.
29. D. H. Choi, S. Y. Oh, and K. Kim, "Connectionist-nonconnectionist fusion architecture for high speed road following," Neural Parallel Sci. Comput., vol. 4, no. 3, pp. 367-386, Sep. 1996.
30. H. M. Kim, J. Dickerson, and B. Kosko, "Fuzzy throttle and brake control for platoons of smart cars," Fuzzy Sets Syst., vol. 84, no. 23, pp. 209-234, Dec. 1996.
31. S. Huang and W. Ren, "Use of neural fuzzy networks with mixed genetic/gradient algorithm in automated vehicle control," IEEE Trans. Ind. Electron., vol. 46, no. 6, pp. 1090-1102, Dec. 1999.
32. J. Mar and F.-J. Lin, "An ANFIS controller for the car-following collision prevention system," IEEE Trans. Veh. Technol., vol. 50, no. 4, pp. 11061113, Jul. 2001.
33. N. Kehtarnavaz, N. Groswold, K. Miller, and P. Lascoe, "A transportable neural-network approach to autonomous vehicle following," IEEE Trans. Veh. Technol., vol. 47, no. 2, pp. 694-702, May 1998.
34. I. H. Sun, J. H. Kim, and F. C.-H. Rhee, "Fuzzy Q-learning for autonomous robot systems," in Proc. Int. Conf. Neural Networks, Houston, TX, 1997, vol. 3, pp. 1738-1743.
35. I. H. Suh, J. H. Kim, and S. R. Oh, "Region-based Q-learning for intelligent robot systems," in IEEE Int. Symp. Computational Intelligence Robotics and Automation (CIRA), Monterey, CA, 1997, pp. 172-178.
36. L. Barid and A. Moore, "Gradient descent for general reinforcement learning," in Advances in Neural Information Processing Systems 11. Cambridge, MA: MIT Press, 1999.