Q(λ)-learning adaptive fuzzy logic controllers for pursuit-evasion differential games

International Journal of Adaptive Control and Signal Processing

Volumn 25, Issue 10, 2011, Pages 910-927

  Desouky, Sameh F ^a   Schwartz, Howard M ^a  

^a CARLETON UNIVERSITY   (Canada)

Author keywords

differential game; function approximation; fuzzy control; pursuit evasion; Q( ) learning; reinforcement learning

Indexed keywords

ADAPTIVE FUZZY LOGIC CONTROLLER; CLASSICAL CONTROL; DIFFERENTIAL GAMES; FUNCTION APPROXIMATION; FUZZY INFERENCE SYSTEMS; FUZZY LOGIC CONTROLLERS; NOVEL TECHNIQUES; OUTPUT PARAMETERS; PURSUIT EVASION; Q-LEARNING; TRAINING DATA;

COMPUTER SIMULATION; CONTROLLERS; FUZZY CONTROL; FUZZY INFERENCE; GAME THEORY; NEURAL NETWORKS; REINFORCEMENT LEARNING;

FUZZY LOGIC;

EID: 80053207522     PISSN: 08906327     EISSN: 10991115     Source Type: Journal    
DOI: 10.1002/acs.1249     Document Type: Article

References (46)

1. Micera S, Sabatini AM, Dario P,. Adaptive fuzzy control of electrically stimulated muscles for arm movements. Medical and Biological Engineering and Computing 1999; 37 (6): 680-685. (Pubitemid 30028848)
2. Daldaban F, Ustkoyuncu N, Guney K,. Phase inductance estimation for switched reluctance motor using adaptive neuro-fuzzy inference system. Energy Conversion and Management 2006; 47: 485-493. (Pubitemid 41653138)
3. Labiod S, Guerra TM,. Adaptive fuzzy control of a class of SISO nonaffine nonlinear systems. Fuzzy Sets and Systems 2007; 158 (10): 1126-1137. (Pubitemid 46428433)
4. Lam HK, Leung FHF,. Fuzzy controller with stability and performance rules for nonlinear systems. Fuzzy Sets and Systems 2007; 158: 147-163. (Pubitemid 44716349)
5. Hagras H, Callaghan V, Colley M,. Learning and adaptation of an intelligent mobile robot navigator operating in unstructured environment based on a novel online fuzzy-genetic system. Fuzzy Sets and Systems 2004; 141 (1): 107-160.
6. Mucientes M, Moreno DL, Bugarin A, Barro S,. Design of a fuzzy controller in mobile robotics using genetic algorithms. Applied Soft Computing 2007; 7 (2): 540-546. (Pubitemid 46205475)
7. Wang LX,. Generating fuzzy rules by learning from examples. IEEE Transactions on Systems, Man, and Cybernetics 1992; 22 (6): 1414-1427.
8. Herrera F, Lozano M, Verdegay JL,. Tuning fuzzy logic controllers by genetic algorithms. International Journal of Approximate Reasoning 1995; 12: 299-315.
9. Lekova A, Mikhailov L, Boyadjiev D, Nabout A,. Redundant fuzzy rules exclusion by genetic algorithms. Fuzzy Sets and Systems 1998; 100: 235-243. (Pubitemid 128654405)
10. Desouky SF, Schwartz HM,. Genetic based fuzzy logic controller for a wall-following mobile robot. 2009 American Control Conference (ACC-09), St. Louis, MO. IEEE: New York, 2009; 3555-3560.
11. Desouky SF, Schwartz HM,. Different hybrid intelligent systems applied for the pursuit-evasion game. 2009 IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, 2009; 2677-2682.
12. Sutton RS, Barto AG,. Reinforcement Learning: An Introduction. MIT Press: Cambridge, MA, 1998.
13. Kaelbling LP, Littman ML, Moore AP,. Reinforcement learning: a survey. Journal of Artificial Intelligence Research 1996; 4: 237-285. (Pubitemid 126646155)
14. Smart WD, Kaelbling LP,. Effective reinforcement learning for mobile robots. IEEE International Conference on Robotics and Automation, vol. 4, Washington, D.C., 2002; 3404-3410. (Pubitemid 34916070)
15. Ye C, Yung NHC, Wang D,. A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 2003; 33 (1): 17-27.
16. Kondo T, Ito K,. A reinforcement learning with evolutionary state recruitment strategy for autonomous mobile robots control. Robotics and Autonomous Systems 2004; 46 (2): 111-124.
17. Gutnisky DA, Zanutto BS,. Learning obstacle avoidance with an operant behavior model. Artificial Life 2004; 10 (1): 65-81.
18. Rodr ìguez M, Iglesias R, Regueiro CV, Correa J, Barro S,. Autonomous and fast robot learning through motivation. Robotics and Autonomous Systems 2007; 55 (9): 735-740. (Pubitemid 47285710)
19. Dai X, Li CK, Rad AB,. An approach to tune fuzzy controllers based on reinforcement learning for autonomous vehicle control. IEEE Transactions on Intelligent Transportation Systems 2005; 6 (3): 285-293. (Pubitemid 41634332)
20. Er MJ, Deng C,. Obstacle avoidance of a mobile robot using hybrid learning approach. IEEE Transactions on Industrial Electronics 2005; 52 (3): 898-905. (Pubitemid 40871125)
21. Xiao H, Liao L, Zhou F,. Mobile robot path planning based on Q-ANN. IEEE International Conference on Automation and Logistics, Jinan, China, 2007; 2650-2654. (Pubitemid 351370855)
22. Vidal R, Shakernia O, Kim HJ, Shim DH, Sastry S,. Probabilistic pursuit-evasion games: theory, implementation, and experimental evaluation. IEEE Transactions on Robotics and Automation 2002; 18 (5): 662-669.
23. Li D, Chen G, Kwan C, Chang M,. A hierarchical approach to multi-player pursuit-evasion differential games. Forty-forth IEEE Conference on Decision and Control, and the European Control Conference 2005, Seville, Spain, 2005; 5674-5679.
24. Ishiwaka Y, Satob T, Kakazu Y,. An approach to the pursuit problem on a heterogeneous multiagent system using reinforcement learning. Robotics and Autonomous Systems 2003; 43 (4): 245-256.
25. Givigi SN, Schwartz HM, Lu X,. A reinforcement learning adaptive fuzzy controller for differential games. Journal of Intelligent and Robotic Systems 2010; 59 (1): 3-30.
26. Jouffe L,. Fuzzy inference system learning by reinforcement methods. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 1998; 28 (3): 338-355. (Pubitemid 128748635)
27. Duan Y, Hexu X,. Fuzzy reinforcement learning and its application in robot navigation. International Conference on Machine Learning and Cybernetics, Guangzhou, vol. 2. IEEE: New York, 2005; 899-904. (Pubitemid 41734053)
28. Waldock A, Carse B,. Fuzzy q-learning with an adaptive representation. 2008 IEEE 16th International Conference on Fuzzy Systems (FUZZ-IEEE), Piscataway, NJ, 2008; 720-725.
29. Watkins CJCH, Dayan P,. Qlearning. Machine Learning 1992; 8 (3): 279-292.
30. Watkins CJCH,. Learning from delayed rewards. Ph.D. Thesis, Cambridge University, 1989.
31. Bhanu B, Leang P, Cowden C, Lin Y, Patterson M,. Real-time robot learning. IEEE International Conference on Robotics and Automation, Seoul, Korea, vol. 1, 2001; 491-498. (Pubitemid 32815252)
32. Ma Cek K, Petrović I, Perić N,. A reinforcement learning approach to obstacle avoidance of mobile robots. Seventh International Workshop on Advanced Motion Control, Maribor, Slovenia, 2002; 462-466. (Pubitemid 40761208)
33. Marin TM, Duckett T,. Fast reinforcement learning for vision-guided mobile robots. IEEE International Conference on Robotics and Automation, Barcelona, Spain, 2005; 4170-4175. (Pubitemid 46084435)
34. Yang Y, Tian Y, Mei H,. Cooperative Q-learning based on blackboard architecture. International Conference on Computational Intelligence and Security Workshops, Harbin, China, 2007; 224-227.
35. Desouky SF, Schwartz HM,. A novel technique to design a fuzzy logic controller using Q(λ)-learning and genetic algorithms in the pursuit-evasion game. 2009 IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, 2009; 2683-2689.
36. Desouky SF, Schwartz HM,. A novel hybrid learning technique applied to a self-learning multi-robot system. 2009 IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, 2009; 2690-2697.
37. Stephan V, Debes K, Gross HM,. A new control scheme for combustion processes using reinforcement learning based on neural networks. International Journal of Computational Intelligence and Applications 2001; 1 (2): 121-136.
38. Yan XW, Deng ZD, Sun ZQ,. Genetic Takagi-Sugeno fuzzy reinforcement learning. IEEE International Symposium on Intelligent Control, Mexico City, Mexico, 2001; 67-72. (Pubitemid 34064417)
39. Gu D, Hu H,. Accuracy based fuzzy Q-learning for robot behaviours. International Conference on Fuzzy Systems, Budapest, Hungary, vol. 3, 2004; 1455-1460. (Pubitemid 40027911)
40. Takagi T, Sugeno M,. Fuzzy identification of systems and its applications to modelling and control. IEEE Transactions on Systems, Man and Cybernetics 1985; SMC-15 (1): 116-132.
41. Isaacs R,. Differential Games. Wiley: New York, 1965.
42. LaValle SM,. Planning Algorithms. Cambridge University Press: Cambridge, 2006.
43. Peng J, Williams RJ,. Incremental multi-step Q-learning. Machine Learning 1996; 22 (1-3): 283-290. (Pubitemid 126724369)
44. Jouffe L,. Actor-critic learning based on fuzzy inference system. IEEE International Conference on Systems, Man, and Cybernetics, Beijing, China, vol. 1, 1996; 339-344.
45. Wang XS, Cheng YH, Yi JQ,. A fuzzy actor-critic reinforcement learning network. Information Sciences 2007; 177: 3764-3781.
46. Lim SH, Furukawa T, Dissanayake G, Whyte HFD,. A time-optimal control strategy for pursuit-evasion games problems. International Conference on Robotics and Automation, New Orleans, LA, 2004.