Multi-Agent Machine Learning: A Reinforcement Approach

Multi-Agent Machine Learning: A Reinforcement Approach

Volumn 9781118362082, Issue , 2014, Pages 1-242

  Schwartz, Howard M ^a  

^a CARLETON UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; ENGINEERING EDUCATION; FUZZY CONTROL; GAME THEORY; LEARNING ALGORITHMS; LEAST SQUARES APPROXIMATIONS; MEAN SQUARE ERROR; MULTI AGENT SYSTEMS; STOCHASTIC CONTROL SYSTEMS; STOCHASTIC SYSTEMS; STUDENTS;

ADAPTIVE FUZZY CONTROL; ELECTRICAL AND COMPUTER ENGINEERING; ELIGIBILITY TRACES; FUZZY INTERFERENCE SYSTEMS; MEAN SQUARE ERROR METHOD; MULTI-AGENT Q-LEARNING; RECURSIVE LEAST SQUARE (RLS); STOCHASTIC APPROXIMATIONS;

REINFORCEMENT LEARNING;

EID: 84924356033     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118884614     Document Type: Book

References (121)

1. L. Ljung, System Identification: Theory for the User. Upper Saddle River, NJ: Prentice Hall, 2nd ed., 1999.
2. L. Ljung and T. Soderstrom, Theory and Practice of Recursive Identification. Cambridge, Massachusetts: The MIT Press, 1983.
3. R. S. Sutton and A. G. Barto, Reinforcement learning: An Introduction. Cambridge, Massachusetts:The MIT Press, 1998.
4. Astrom, K. J. and Wittenmark, B., Adaptive Control. Boston, Massachusetts:Addison-Wesley Longman Publishing Co., Inc., 2nd ed., 1994, ISBN = 0201558661.
5. L. Buşoniu and R. Babuška, and B. D. Schutter, "A comprehensive survey of multiagent reinforcement learning," IEEE Trans. Syst. Man Cybern. Part C, Vol. 38, no. 2, pp. 156-172, 2008.
6. L. Ljung and T. Soderstrom, Theory and Practice of Recursive Identification. Cambridge, Massachusetts: The MIT Press, 1983.
7. B. Farhang-Boroujeny, Adaptive Filters: Theory and Applications. New York: JohnWiley & Sons, Inc., 1998.
8. H. Robbins and S. Monro, "A stochastic approximation method," Annals of Mathematical Statistics, vol. 22, no. 3, pp. 400-407, 1951.
9. J. Wolfowitz, "On the stochastic approximation method of robbins and monro," Annals of Mathematical Statistics, vol. 23, no. 3, pp. 457-461, 1952.
10. H. J. Kushner and G. G. Yin, Recursive Approximation and Recursive Algorithms and Applications. New York: Springer-Verlag, 2nd ed., 2003.
11. R. S. Sutton and A. G. Barto, Reinforcement learning: An Introduction. Cambridge, Massachusetts:The MIT Press, 1998.
12. M. Bowling and M. Veloso, "Multiagent learning using a variable learning rate," Artificial Intelligence, vol. 136, no. 2, pp. 215-250, 2002.
13. R. Bellman, Dynamic Programming. Princeton, New Jersey: Princeton University Press, 1957.
14. D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models. Englewood Cliffs, New Jersey: Prentice-Hall, 1987.
15. R. A. Howard, Dynamic Programming and Markov Processes. Cambridge, Massachusetts:MIT Press, 1960.
16. C. J. C. H. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, no. 3, pp. 279-292, 1992.
17. T. Jaakkola, M. Jordan, and S. Singh, "On the convergence of stochasticiterative dynamic programming algorithms," Neural Computation, vol. 6, no. 6, pp. 1185-1201, 1994.
18. M. J. Osborne and A. Rubinstein, A Course in Game Theory. Cambridge, Massachusetts:The MIT Press, 1994.
19. G. Owen, Game Theory. San Diego, California: Academic Press, 1995.
20. M. Bowling andM. Veloso, "Multiagent learning using a variable learning rate," Artificial Intelligence, vol. 136, no. 2, pp. 215-250, 2002.
21. T. Başar and G. J. Olsder, Dynamic Noncooperative Game Theory, SIAM Series in Classics in Applied Mathematics. London: Academic Press, 2nd ed., 1999.
22. P. Sastry, V. Phansalkar, and M. Thathachar, "Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information," IEEE Transactions on Systems, Man, and Cybernetics, vol. 24, no. 5, pp. 769-777, 1994.
23. M. J. Osborne, An Introduction to Game Theory. Oxford: Oxford University Press, 2003.
24. M. L. Littman, "Markov games as a framework for multi-agent reinforcement learning," in 11th International Conference on Machine Learning, (New Brunswick, United States), July 1994, pp. 157-163, 1994.
25. X. Lu, Ph.D., "On Multi-Agent Reinforcement Learning in Games." Ph.D. Thesis, Carleton University, Ottawa, ON, 2012.
26. S. P. Singh, M. J. Kearns, and Y. Mansour, "Nash convergence of gradient dynamics in general-sum games," in UAI '00: Proceedings of the 16th Conference in Uncertainty in Artificial Intelligence, Stanford University, Stanford, California, USA, June 30 - July 3, 2000, pp. 541-548, 2000.
27. C. J. C. H. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, no. 3, pp. 279-292, 1992.
28. E. Yang and D. Gu, "A survey on multiagent reinforcement learning towards multi-robot systems," in Proceedings of IEEE Symposium on Computational Intelligence and Games, 2005.
29. L. Buşoniu, R. Babuška, and B. D. Schutter, "Multiagent reinforcement learning: a survey," 9th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1-6, 2006.
30. X. Lu and H. M. Schwartz, "An investigation of guarding a territory problem in a grid world," in American Control Conference, pp. 3204-3210, 2010.
31. S. Lakshmivarahan and K. S. Narendra, "Learning algorithms for two-person zero-sum stochastic games with incomplete information," Mathematics of Operations Research, vol. 6, no. 3, pp. 379-386, 1981.
32. S. Lakshmivarahan and K. S. Narendra, "Learning algorithms for two-person zero-sum stochastic games with incomplete information: a unified approach," SIAM Journal on Control and Optimization, vol. 20, no. 4, pp. 541-552, 1982.
33. F. A. Dahl, "The lagging anchor algorithm: reinforcement learning in two-player zero-sum games with imperfect information," Machine Learning, vol. 49, pp. 5-37, 2002.
34. F. A. Dahl, "The lagging anchor model for game learning-a solution to the crawford puzzle," Journal of Economic Behavior & Organization, vol. 57, pp. 287-303, 2005.
35. K. S. Narendra and M. A. L. Thathachar, LearningAutomata: An Introduction. Englewood Cliffs, New Jersey: Prentice Hall, 1989.
36. M. Thathachar and P. Sastry, Networks of Learning Automata: Techniques for Online Stochastic Optimization. Boston, Massachusetts: Kluwer Academic Publishers, 2004.
37. M. L. Littman, "Markov games as a framework for multi-agent reinforcement learning," in 11th International Conference on Machine Learning, (New Brunswick, United States), July 1994, pp. 157-163, 1994.
38. M. Bowling andM. Veloso, "Multiagent learning using a variable learning rate," Artificial Intelligence, vol. 136, no. 2, pp. 215-250, 2002.
39. R. Isaacs, Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization. New York, New York: John Wiley and Sons, Inc., 1965.
40. M. Bowling, Multiagent Learning in the Presence of Agents with Limitations. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, May 2003.
41. X. Lu, "On Multi-Agent Reinforcement Learning in Games," Ph.D. Thesis Carleton University, Ottawa, ON, Canada, 2012.
42. M. L. Littman and C. Szepesvári, "A generalized reinforcement-learning model: Convergence and applications," in Proceedings of the 13th International Conference on Machine Learning, (Bari, Italy), July 1996, pp. 310-318, 1996.
43. J. Hu and M. P. Wellman, "Multiagent reinforcement learning: theoretical framework and an algorithm," in Proceedings of the Fifteenth International Conference on Machine Learning (ICML 1998), Madison, Wisconsin, USA, July 24-27, 1998, pp. 242-250, 1998.
44. J. Hu andM. P.Wellman, "Nash q-learning for general-sum stochastic games," Journal of Machine Learning Research, vol. 4, pp. 1039-1069, 2003.
45. S. Abdallah, "Equilibrium in a stochastic n-person game," Journal of Science in Hiroshima University, Series A-I, 28: 89-93, 1964.
46. M. L. Littman, "Friend-or-foe q-learning in general-sum games," in Proceedings of the 18th International Conference on Machine Learning, (Williams College, MA), pp. 322-328, 2001.
47. C. Claus and C. Boutilier, "The dynamics of reinforcement learning in cooperative multiagent systems," in Proceedings of National Conference on Artificial Intelligence (AAAI-98), pp. 746-752, 1998.
48. C. E. Lemke and J. J. T. Howson, "Equilibrium points of bimatrix games," SIAM Journal on Applied Mathematics, vol. 12, no. 2, pp. 413-423, 1964.
49. D. D. Meredith, K.W.Wong, R.W.Woodhead, and R. H.Wortman, Design and Planning of Engineering Systems. Englewood Cliffs, New Jersey: Prentice-Hall, 1973.
50. R.W. Cottle, J.-S. Pang, and R. E. Stone, "The linear complimentary problem," Computer Science and Scientific Computing, San Diego, California: Academic Press, Inc., 1992.
51. P. De Beck-Courcelle, "Study ofMultiple Multiagent Reinforcement Learning Algorithms in Grid Games", Master's thesis, Carleton University, Ottawa, ON, Canada, 2013.
52. E. Yang and D. Gu, "A survey on multiagent reinforcement learning towards multi-robot systems," in Proceedings of IEEE Symposium on Computational Intelligence and Games, 2005.
53. L. Buşoniu, R. Babuška, and B. D. Schutter, "Multiagent reinforcement learning: a survey," in 9th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1-6, 2006.
54. X. Lu and H. M. Schwartz, "An investigation of guarding a territory problem in a grid world," in American Control Conference, pp. 3204-3210, 2010.
55. K. H. Hsia and J. G. Hsieh, "A first approach to fuzzy differential game problem: guarding a territory," Fuzzy Sets and Systems, vol. 55, pp. 157-167, 1993.
56. Y. S. Lee, K. H. Hsia, and J. G. Hsieh, "A strategy for a payoff-switching differential game based on fuzzy reasoning," Fuzzy Sets and Systems, vol. 130, no. 2, pp. 237-251, 2002.
57. L. Buşoniu, R. Babuška, and B. D. Schutter, "A comprehensive survey of multiagent reinforcement learning," IEEE Transactions on Systems, Man, and Cybernetics Part C, vol. 38, no. 2, pp. 156-172, 2008.
58. P. Stone and M. Veloso, "Multiagent systems: a survey from a machine learning perspective," Autonomous Robots, vol. 8, no. 3, pp. 345-383, 2000.
59. J. W. Sheppard, "Colearning in differential games," Machine Learning, vol. 33, pp. 201-233, 1998.
60. M. Awheda, and Schwartz, H.M., "Exponential Moving Average Q-Learning Algorithm", Proceedings of the IEEE Symposium Series on Computational Intelligence, Singapore, April 15-19, 2013.
61. A. Burkov and B. Chaib-draa, "Effective learning in the presence of adaptive counterparts," Journal of Algorithms, vol. 64, no. 4, pp. 127-138, 2009.
62. G. Tesauro, "Extending q-learning to general adaptive multi-agent systems," in Advances in Neural Information Processing Systems 16 (S. Thrun, L. K. Saul and B. Schölkopf, eds.), (Cambridge, Massachusetts), pp. 215-250, MIT Press, 2004.
63. M. Bowling, "Convergence and no-regret in multiagent learning," in Advances in Neural Information Processing Systems 17 (L. K. Saul, Y.Weiss and L. Bottou, eds.), (Cambridge, Massachusetts), pp. 209-216, MIT Press, 2005.
64. S. Abdallah and V. Lesser, "Amultiagent reinforcement learning algorithm with non-linear dynamics," Journal of Artificial Intelligence Research, vol. 33, pp. 521-549, 2008.
65. C. Zhang and V. Lesser, "Multi-agent learning with policy prediction," in Proceedings of the 24th National Conference on Artificial Intelligence (AAAI'10), Atlanta, GA, USA, pp. 746-752, 2010.
66. R. Abielmona, E. Petriu, M. Harb, and S. Wesolkowski, "Mission-driven robotic intelligent sensor agents for territorial security," IEEE Computational Intelligence Magazine, vol. 6, no. 1, pp. 55-67, 2011.
67. R. Isaacs, Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization. New York: John Wiley and Sons, Inc., 1965.
68. M. E. Harmon, L. C. Baird III, and A. H. Klopf, "Reinforcement learning applied to a differential game," Adaptive Behavior, vol. 4, no. 1, pp. 3-28, 1995.
69. J. W. Sheppard, "Colearning in differential games," Machine Learning, vol. 33, pp. 201-233, 1998.
70. S. N. Givigi, H. M. Schwartz, and X. Lu, "A reinforcement learning adaptive fuzzy controller for differential games," Journal of Intelligent and Robotic Systems, vol. 59, pp. 3-30, 2010.
71. S. F. Desouky and H. M. Schwartz, "Self-learning fuzzy logic controllers for pursuit-evasion differential games," Robotics and Autonomous Systems, vol. 59, pp. 22-33, 2011.
72. K. H. Hsia and J. G. Hsieh, "A first approach to fuzzy differential game problem: guarding a territory," Fuzzy Sets and Systems, vol. 55, pp. 157-167, 1993.
73. Y. S. Lee, K. H. Hsia, and J. G. Hsieh, "A strategy for a payoff-switching differential game based on fuzzy reasoning," Fuzzy Sets and Systems, vol. 130, no. 2, pp. 237-251, 2002.
74. L. Jouffe, "Fuzzy inference system learning by reinforcement methods," IEEE Transactions on Systems, Man, and Cybernetics Part C, vol. 28, no. 3, pp. 338-355, 1998.
75. K. M. Passino and S. Yurkovich, Fuzzy Control. Boston, Massachusetts: Addison-Wesley Longman Publishing Co., Inc., 1st ed., 1998.
76. J.-S. R. Jang and C.-T. Sun, Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence. Upper Saddle River, New Jersey: Prentice-Hall, Inc., 1997.
77. L. A. Zadeh, "Fuzzy sets," Information and Control, vol. 8, no. 3, pp. 338-353, 1965.
78. B. Al Faiya, "Learning in Pursuit-Evasion Differential Games Using Reinforcement Fuzzy Learning," Master's thesis, Carleton University, Ottawa, ON, Canada, 2012.
79. T. Takagi and M. Sugeno, "Fuzzy identification of systems and its applications to modelling and control," IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-15, pp. 116-132, 1985.
80. L.-X. Wang, A Course in Fuzzy Systems and Control. Upper Saddle River, New Jersey, Prentice-Hall, Inc., 1997.
81. J. Jantzen, Design of Fuzzy Controllers. Technical Univ. of Denmark: Technical Report (No:98-E864) Department of Automation, Hoboken, NJ, 1999.
82. T. J. Ross, Fuzzy Logic with Engineering Applications. John Wiley & Sons, Ltd, 2010.
83. X. Lu, "On Multi-Agent Reinforcement Learning in Games." Ph.D. Thesis Carleton University, Ottawa, ON, Canada, 2012.
84. X. Dai, C. Li, and A. Rad, "An approach to tune fuzzy contorllers based on reinforcement learning for autonomous vehicle control," IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 3, pp. 285-293, 2005.
85. L. X.Wang, A Course in Fuzzy Systems and Control. Englewood Cliffs, New Jersey: Prentice Hall, 1997.
86. T. Takagi and M. Sugeno, "Fuzzy identification of systems and its application to modeling and control," IEEE Transactions on Systems, Man, and Cybernetics, vol. 15, pp. 116-132, 1985.
87. J.-S. R. Jang and C.-T. Sun, Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence. Upper Saddle River, New Jersey: Prentice-Hall, Inc., 1997.
88. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge,Massachusetts:The MIT Press, 1998.
89. S. N. Givigi, H. M. Schwartz, and X. Lu, "An experimental adaptive fuzzy controller for differential games," in Proceedings IEEE Systems, Man and Cybernetics'09, (San Antonio, United States), Oct 2009.
90. W. M. van Buijtenen,G. Schram, R. Babuska, and H. B. Verbruggen, "Adaptive fuzzy control of satellite attitude by reinforcement learning," IEEE Transactions on Fuzzy Systems, vol. 6, no. 2, pp. 185-194, 1998.
91. R. Isaacs, Differential Games. New York: John Wiley & Sons, Inc., 1965.
92. A. MERZ, "The homicidal chauffeur," AIAA Journal, vol. 12, pp. 259-260, 1974.
93. T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory. New York: Academic Press, 2nd ed., SIAM Classics, 1999.
94. S. H. Lim, T. Furukawa, G. Dissanayake, and H. F. D. Whyte, "A time-optimal control strategy for pursuit-evasion games problems," in Proceedings of the 2004 IEEE International Conference on Robotics and Automation, (New Orleans, LA), April 2004.
95. S. Desouky and H. Schwartz, "Hybrid intelligent systems applied to the pursuit-evasion game," in Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on, (San Antonio, TX, October 2009, pp. 2603-2608, 2009.
96. S. Desouky and H. Schwartz, "Q(λ)-learning fuzzy logic controller for a multi-robot system," in IEEE International Conference on Systems, Man and Cybernetics, 2010. SMC 2010, (Istanbul, Turkey), October 2010.
97. S. F. Desouky and H. M. Schwartz, "Genetic based fuzzy logic controller for a wall-following mobile robot," in American Control Conference, 2009 ACC 2009, (St. Louis, MO), June 2009, pp. 3555-3560, IEEE, 2009.
98. M. Grzés and D. Kudenko, "Online learning of shaping rewards in reinforcement learning," Neural Networks, vol. 23, pp. 541-550, 2010.
99. M. J. Matarić, "Reinforcement learning in the multi-robot domain," Autonomous Robots, vol. 4, pp. 73-83, 1997.
100. J. Randløv and P. Alstrøm, "Learning to drive a bicycle using reinforcement learning and shaping," in 15th International Conference on Machine Learning, pp. 463-471, 1998.
101. E. Bonabeau, M. Dorigo, andG. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems. New York, New York: Oxford University Press, 1999.
102. P. P. Grassé, "La Reconstruction du nid et les Coordinations Inter-Individuelles chez Bellicosistermes Natalensis et Cubetermes sp. La théorie de la Stigmergie: Essai d'interprétation du Comportement des Termites Constructeurs," Insectes Sociaux, vol. 6, pp. 41-80, 1959.
103. M. Mamei, F. Zambonelli, and L. Leonardi, "Cofields: a physically inspired approach to motion cordination," IEEE Pervasive Computing, vol. 3, no. 2, pp. 52-61, 2004.
104. R. Chalmers, D. Scheidt, T. Neighoff, S. Witwicki, and R. Bamberger, "Cooperating unmanned vehicles," in AIAA 1st Intelligent Systems Technical Conference, 2004.
105. J. Borenstein and Y. Koren, "Real time obstable avoidance for fast mobile robots," IEEE Transactions on Systems, Man, and Cybernetics, vol. 19, no. 5, pp. 1179-1187, 1989.
106. M. Mynsk, The Society of Mind. New York, New York: Simon & Schuster, 1986.
107. S. N. Givigi and H.M. Schwartz, "A game theoretic approach to swarm robotics," Applied Bionics and Biomechanics, vol. 3, no. 3, pp. 131-142, 2006.
108. C. Darwin, The Expression of the Emotions in Man and Animals. Chicago, Illinois: University of Chicago Press, 1965.
109. D. Yingying, H. Yan, and J. Jing-ping, "Self-organizing multi-robot system based on personality evolution," in IEEE International Conference on Systems, Man, and Cybernetics, vol. 5, 2002.
110. S. Givigi and H. M. Schwartz, "Evolutionary swarm intelligence applied to robotics," in Prooceedings of the IEEE International Conference on Mechatronics andAutomation, pp. 1005-1010, 2005.
111. E. Sahin, "Swarm robotics: from sources of inspiration to domains of application," in Swarm Robotics: SAB 2004 International Workshop, Santa Monica, CA, USA, July 17, 2004: revised selected papers (E. Sahin and W. M. Spears, eds.), (Berlin, Heidelberg), pp. 10-20, Springer-Verlag, 2005.
112. G. Beni, "Fromswarmintelligence to swarmrobotics," in SwarmRobotics: SAB 2004 International workshop, Santa Monica, CA, USA, July 17, 2004: revised selected papers (E. Sahin and W. M. Spears, eds.), (Berlin, Heidelberg), pp. 1-9, Springer-Verlag, 2005.
113. M. Dorigo, V. Trianni, E. Şahin, R. Groß, T. Labella, G. Baldassarre, S. Nolfi, J. Deneubourg, F. Mondada, D. Floreano, and L. Gambardella, "Evolving self-organizing behaviours for a swarm-bot," Autonomous Robots, vol. 17, pp. 223-245, 2004.
114. P. D. Straffin, Game Theory and Strategy. Washington, District of Columbia: The Mathematical Association of America, 1993.
115. J. S. Shamma and G. Arslan, "Unified convergence proofs of continuous-time fictitious play," IEEE Transactions on Automatic Control, vol. 49, no. 7, pp. 1137-1142, 2004.
116. J. W. Weibull, Evolutionary Game Theory. Cambridge, Massachusetts: MIT Press, 1995.
117. J. Robinson, "An iterative method of solving a game," Annals of Mathmatics, vol. 54, no. 2, pp. 296-301, 1951.
118. D. P. Bertsekas, Dynamic Programming and Optimal Control. Belmont, Massachusetts:Athena Scientific, 1995.
119. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: a survey," Journal of Artificial Intelligence Research, vol. 4, pp. 237-285, 1996.
120. R. Dawkins, The Selfish Gene. New York, New York: Oxford University Press, new ed., 1989.
121. G. Sidney, "Analysis and Design of Swarm Based Robots Using Game Theory". Ph.D. Thesis, Ottawa, ON: Carleton University, Sept. 2009.