Multiagent learning: Basics, challenges, and prospects

AI Magazine

Volumn 33, Issue 3, 2012, Pages 41-52

  Tuyls, Karl ^a   Weiss, Gerhard ^a  

^a MAASTRICHT UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL LIFE; INTELLIGENT AGENTS; LEARNING ALGORITHMS; MACHINE LEARNING; MARKOV PROCESSES; MULTI AGENT SYSTEMS; OPTIMAL SYSTEMS; REINFORCEMENT LEARNING; ROBOTS; SURVEYS;

AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS; DISTRIBUTED TRAFFIC LIGHT CONTROL; FUTURE RESEARCH DIRECTIONS; LARGE-SCALE AGENT SYSTEMS; MACHINE LEARNING RESEARCH; MACHINE LEARNING TECHNIQUES; MARKOV DECISION PROCESSES; TECHNOLOGICAL CHALLENGES;

AUTONOMOUS AGENTS;

EID: 84861480021     PISSN: 07384602     EISSN: None     Source Type: Journal    
DOI: 10.1609/aimag.v33i3.2426     Document Type: Conference Paper

References (85)

1. Abbeel, P.; Coates, A.; Quigley, M.; and Ng, A. Y. 2007. An Application of Reinforcement Learning to Aerobatic Helicopter Flight. In Neural Information Processing Systems, volume 19. Cambridge, MA: The MIT Press.
2. Abdallah, S., and Lesser, V. R. 2008. A Multiagent Reinforcement Learning Algorithm with Non-Linear Dynamics. Journal of Artificial Intelligence Research 33: 521-549.
3. Agogino, A. K., and Tumer, K. 2012. A Multiagent Approach to Managing Air Traffic Flow. Journal of Autonomous Agents and MultiAgent Systems 24(1): 1-25.
4. Agogino, A. K., and Tumer, K. 2008. Analyzing and Visualizing Multiagent Rewards in Dynamic and Stochastic Environments. Journal of Autonomous Agents and MultiAgent Systems 17(2): 320-338.
5. Alers, S.; Bloembergen, D.; Hennes, D.; de Jong, S.; Kaisers, M.; Lemmens, N.; Tuyls, K.; and Weiss, G. 2011. Bee-Inspired Foraging in an Embodied Swarm. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems.
6. Ammar, H. Y. B., and Taylor, M. E. 2011. Common Subspace Transfer for Reinforcement Learning Tasks. Paper presented at the AAMAS 2011 Workshop on Adaptive Learning Agents and Multiagent Systems, Taipei, Taiwan, May 2.
7. Ando, R. K., and Zhang, T. 2005. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data. Journal of Machine Learning Research 6(11): 1817- 1853.
8. Banerjee, A. 1992. A Simple Model of Herd Behavior. Quarterly Journal of Economics 107(3): 797-817.
9. Bellman, R. 1957. Dynamic Programming. Princeton, NJ: Princeton University Press.
10. Bernstein, D. S.; Zilberstein, S.; and Immerman, N. 2000. The Complexity of Decentralized Control of Markov Decision Processes. In Proceedings of the 16th Conference in Uncertainty in Artificial Intelligence, 32-37. San Francisco: Morgan Kaufmann.
11. Bonabeau, E.; Dorigo, M.; and Theraulaz, G. 1999. Swarm Intelligence: From Natural to Artificial Systems. New York: Oxford University Press.
12. Bowling, M. 2004. Convergence and No- Regret In Multiagent Learning. In Advances in Neural Information Processing Systems, volume 17, 209-216, Cambridge, MA: MIT Press.
13. Bowling, M., and Veloso, M. 2002. Multiagent Learning Using a Variable Learning Rate. Artificial Intelligence 136(2): 215-250.
14. Boyd, R., and Richerson, P. 1985. Culture and the Evolutionary Process. Chicago: The University of Chicago Press.
15. Breazeal, C.; Berlin, M.; Brooks, A.; Gray, J.; and Thomaz, A. 2006. Using Perspective Taking to Learn from Ambiguous Demonstrations. Robotics and Autonomous Systems 54(5): 385-393.
16. Bui, H.; Kieronska, D.; and Venkatesh, S. 1996. Learning Other Agents' Preferences in Multiagent Negotiation. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, 114-119. Menlo Park, CA: AAAI Press.
17. Busoniu, L.; Babuska, R.; and De Schutter, B. 2008. A Comprehensive Survey of Multiagent Reinforcement Learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 38(2): 156-172.
18. Chakraborty, D., and Stone, P. 2010. Convergence, Targeted Optimality and Safety in Multiagent Learning. In Proceedings of the 27th International Conference on Machine Learning. Madison, WI: Omnipress.
19. Christoudias, C.; Urtasun, R.; and Darrell, T. 2008. Multiview Learning in the Presence of View Disagreement. In Proceedings of the Twenty-Fourth Annual Conference on Uncertainty in Artificial Intelligence, 88-96. Corvallis, OR: Association for Uncertainty in Artificial Intelligence Press.
20. Claus, C., and Boutilier, C. 1998. The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems. In Proceedings of the 15th National Conference on Artificial Intelligence, 746-752. Menlo Park, CA: AAAI Press.
21. Colorni, A.; Dorigo, M.; and Maniezzo, V. 1992. Distributed Optimization by Ant Colonies. In Towards a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, ed. F. J. Varela and P. Bourgine, 134-142. Cambridge, MA: MIT Press.
22. Dorigo, M., and Gambardella, L. M. 1997. Ant Colony Systems: A Cooperative Learning Approach to the Travelling Salesman Problem. IEEE Transactions on Evolutionary Computation 1(1): 53-66.
23. Dorigo, M., and Stützle, T. 2004. Ant Colony Optimization. Cambridge, MA: The MIT Press/Bradford Books.
24. Ficici, S. G., and Pollack, J. B. 1998. Challenges in Coevolutionary Learning: Arms- Race Dynamics, Openendedness, and Mediocre Stable States. In Proceedings of the 6th International Conference on Artificial Life, 238-247. Cambridge, MA: The MIT Press.
25. Galef, B. 1988. Imitation in Animals: History, Definition, and Interpretation of Data from the Psychological Laboratory. In Social Learning: Psychologic and Biological Perspectives, ed. T. Zentall, and B. Galef. Hillsdale, NJ: Lawrence Erlbaum Associates.
26. Haynes, T.; Lau, K.; and Sen, S. 1996. Learning Cases to Compliment Rules for Conflict Resolutions in Multiagent Systems. In Adaptation, Coevolution and Learning in Multiagent Systems. Papers from the 1996 AAAI Symposium, ed. S. Sen. Technical Report SS-96-01, 51-56. Menlo Park, CA: AAAI Press.
27. He, X.; Zhu, Y.; Hu, K.; and Niu, B. 2007. A Swarm-Based Learning Method Inspired by Social Insects. In Advanced Intelligent Computing Theories and Applications with Aspects of Artificial Intelligence: Third International Conference on Intelligent Computing, volume 4682 of Lecture Notes in Computer Science, ed. D.-S. Huang, L. Heutte, and M. Loog, 525-533. Berlin: Springer.
28. Hu, J., and Wellman, M. P. 2000. Experimental Results on Q-Learning for General- Sum Stochastic Games. In Proceedings of the Seventeenth International Conference on Machine Learning, 407-414. Morgan Kaufmann Publishers Inc.
29. Hu, J., and Wellman, M. P. 2003. Nash QLearning for General-Sum Stochastic Games. Journal of Machine Learning Research 4(11): 1039-1069.
30. Huhns, M., and Weiss, G., eds. 1998. Special Issue on Multiagent Learning. Machine Learning Journal 33(2-3).
31. Kalyanakrishnan, S.; Liu, Y.; and Stone, P. 2006. Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study. In RoboCup 2006: Robot Soccer World Cup XI, Lecture Notes in Computer Science 4434, 72-85. Berlin: Springer
32. Laland, K.; Richerson, P.; and Boyd, R. 1993. Animal Social Learning: Toward a New Theoretical Approach. In Perspectives in Ethology, ed. P. Klopfer, P. Bateson, and N. Thomson. New York: Plenum press.
33. Lemmens, N., and Tuyls, K. 2009. Stigmergic Landmark Foraging. In Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, 497-504. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems.
34. Littman, M. 1994. Markov Games as a Framework for Multiagent Reinforcement Learning. In Proceedings of the Eleventh International Conference on Machine Learning, 157-163. San Francisco: Morgan Kaufmann.
35. Littman, M. L. 2001. Friend-or-Foe Q-Learning in General-Sum Games. In Proceedings of the Eighteenth International Conference on Machine Learning, 322-328. San Francisco: Morgan Kaufmann.
36. Manderick, B., and Moyson, F. 1988. The Collective Behavior of Ants: An Example of Self-Organization in Massive Parallelism. Paper presented at the AAAI Spring Symposium on Parallel Models of Intelligence. March 1988.
37. Manderick, B., and Spiessens, P. 1989. Fine- Grained Parallel Genetic Algorithms. In Proceedings of the 3rd International Conference on Genetic Algorithms, 428-433. San Francisco: Morgan Kaufmann.
38. Meng, W., and Han, X. 2009. Parallel Reinforcement Learning Algorithm and Its Application. Computer Engineering and Applications 45(34): 25-28.
39. Michalski, R., and Tecuci, G., eds. 1995. Machine Learning. A Multistrategy Approach. San Francisco: Morgan-Kaufmann.
40. Nagendra Prasad, M.; Lesser, V.; and Lander, S. 1996. Learning Organizational Roles in a Heterogeneous Multiagent System. In Proceedings of the Second International Conference on Multiagent Systems, 291-298. Menlo Park, CA: AAAI Press.
41. Ontañón, S., and Plaza, E. 2010. Multiagent Inductive Learning: An Argumentation- Based Approach. In Proceedings of the 27th International Conference on Machine Learning, 839-846. Madison, WI: Omnipress.
42. Panait, L., and Luke, S. 2005. Cooperative Multiagent Learning: The State of The Art. Autonomous Agents and MultiAgent Systems 11(3): 387-434.
43. Panait, L.; Tuyls, K.; and Luke, S. 2008. Theoretical Advantages of Lenient Learners: An Evolutionary Game Theoretic Perspective. Journal of Machine Learning Research 9: 423-457
44. Paredis, J. 1995. Coevolutionary Computation. Artificial Life 2(4): 355-375.
45. Pfeifer, R.; Schreter, Z.; Fogelman-Soulie, F.; and Steels, L. 1989. Connectionism in Perspective. Amsterdam: Elsevier Science Publishers.
46. Price, B., and Boutilier, C. 2003. Accelerating Reinforcement Learning Through Implicit Imitation. Journal of Artificial Intelligence Research 19: 569-629.
47. Proper, S., and Tadepalli, P. 2009. Multiagent Transfer Learning Via Assignment-Based Decomposition. In Proceedings of the 2009 International Conference on Machine Learning and Applications, 345-350. Los Alamitos, CA: IEEE Computer Society.
48. Provost, F., and Kolluri, V. 1999. A Survey of Methods for Scaling Up Inductive Algorithms. Data Mining and Knowledge Discovery 3(2): 131-169.
49. Puterman, M. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: John Wiley and Sons.
50. Richter, S.; Aberdeen, D.; and Yu, J. 2006. Natural Actor-Critic for Road Traffic Optimisation. In Neural Information Processing Systems, volume 18, 1169-1176. Cambridge, MA: The MIT Press.
51. Sen, S., and Weiss, G. 1999. Learning in Multiagent Systems. In Multiagent Systems, ed. G. Weiss. Cambridge, MA: The MIT Press. 259-298.
52. Sen, S.; Sekaran, M.; and Hale, J. 1994. Learning to Coordinate Without Sharing Information. In Proceedings of the Twelfth National Conference on Artificial Intelligence, 426-431. Menlo Park, CA: AAAI Press.
53. Sen, S., ed. 1996. Adaptation, Coevolution and Learning in Multiagent Systems. Papers from the 1996 Spring Symposium. Technical Report SS-96-01. Menlo Park, CA: AAAI Press.
54. Shoham, Y.; Powers, R.; and Grenager, T. 2007. If Multiagent Learning Is the Answer, What Is the Question? Artificial Intelligence 171(7): 365-377.
55. Singh, S. P.; Kearns, M. J.; and Mansour, Y. 2000. Nash Convergence of Gradient Dynamics in General-Sum Games. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, 541-548. San Francisco: Morgan Kaufmann Publishers Inc.
56. Smith, B. L., and Gregor, J. T. 1992. What Is Collaborative Learning? In Collaborative Learning: A Sourcebook for Higher Education, ed. A. Goodsell, M. Maher, V. Tinto, B. L. Smith, and J. MacGregor, 1-49. University Park, PA: National Center on Postsecondary Teaching, Learning, and Assessment, Pennsylvania State University.
57. Steels, L. 1987. Massive Parallelism for Artificial Intelligence. Microprocessing and Microprogramming 21(1-5): 17-19.
58. Steels, L. 1988. Artificial Intelligence and Complex Dynamics. In Concepts and Techniques of Knowledge-Based Systems. Amsterdam, The Netherlands: North Hollad.
59. Steels, L. 1996. Self-Organising Vocabularies. In Proceedings of Artificial Life, Volume V. Cambridge, MA: The MIT Press, Cambridge.
60. Stone, P., and Veloso, M. 2000. Multiagent Systems: A Survey from a Machine Learning Perspective. Autonomous Robots 8(3): 345-383.
61. Stone, P. 2007. Multiagent Learning Is Not the Answer. It Is the Question. Artificial Intelligence 171(7): 402-405.
62. 't Hoen, P.; Tuyls, K.; Panait, L.; Luke, S.; and la Poutré, H. 2006. An Overview of Cooperative and Competitive Multiagent Learning. In Learning and Adaptation in MultiAgent Systems, Lecture Notes in Artificial Intelligence volume 3898, ed. K. Tuyls, P. 't Hoen, K. Verbeeck, and S. Sen, 1-50. Berlin: Springer
63. Tan, M. 1993. Multiagent Reinforcement Learning: Independent Versus Cooperative Agents. In Proceedings of the Tenth International Conference on Machine Learning, 330- 337. San Francisco: Morgan Kaufmann.
64. Taylor, M. E., and Stone, P. 2009. Transfer Learning for Reinforcement Learning Domains: A Survey. Journal of Machine Learning Research 10(1): 1633-1685.
65. Taylor, M. E., and Stone, P. 2011. An Introduction to Inter-Agent Transfer for Reinforcement Learning. AI Magazine 32(1): 15- 34.
66. Tumer, K.; Agogino, A.; and Wolpert, D. 2002. Learning Sequences of Actions in Collectives of Autonomous Agents. In Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems, 378-385. New York: Association for Computing Machinery.
67. Tuyls, K., and Nowe, A. 2005. Evolutionary Game Theory and Multiagent Reinforcement Learning. The Knowledge Engineering Review 20(1): 63-90.
68. Tuyls, K., and Parsons, S. 2007. What Evolutionary Game Theory Tells Us About Multiagent Learning. Artificial Intelligence 171(7): 406-416.
69. Tuyls, K.; 't Hoen, P.; and Vanschoenwinkel, B. 2006. An Evolutionary Dynamical Analysis of Multiagent Learning in Iterated Games. Journal of Autonomous Agents and Multiagent Systems 12(1): 115-153.
70. Vega-Redondo, F. 2003. Economics and the Theory of Games. Cambridge: Cambridge University Press.
71. Vidal, J., and Durfee, E. 1996. The Impact of Nested Agent Models in an Information Economy. In Proceedings of the 2nd International Conference on Multiagent Systems, 377- 384. Menlo Park, CA: AAAI Press.
72. Watkins, C. 1989. Learning with Delayed Rewards. Ph.D. Dissertation, Cambridge University, Cambridge, UK.
73. Weibull, J. W. 1996. Evolutionary Game Theory. Cambridge, MA: The MIT Press.
74. Weiss, G. 1993. Learning to Coordinate Actions In Multiagent Systems. In Proceedings of the 13th International Joint Conference on Artificial Intelligence, 311-316. San Francisco: Morgan Kaufmann.
75. Weiss, G. 1996. Adaptation and Learning in Multiagent Systems: Some Remarks and a Bibliography. In Adaption and Learning in Multiagent Systems, volume 1042 of Lecture Notes in Artificial Intelligence, ed. G. Weiss and S. Sen, S. Berlin: Springer-Verlag.
76. Weiss, G., ed. 1997. Distributed Artificial Intelligence Meets Machine Learning, volume 1221 of Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.
77. Weiss, G., ed. 1998. Special Issue on Learning in Distributed Artificial Intelligence Systems. Journal of Experimental and Theoretical Artificial Intelligence 10(3).
78. Weiss, G., and Dillenbourg, P. 1999. What Is "Multi" in Multiagent Learning? In Collaborative Learning: Cognitive and Computational Approaches, ed. P. Dillenbourg, 64-80. Pergamon Press.
79. Weiss, G., and Sen, S., eds. 1996. Adaption and Learning in Multiagent Systems, volume 1042 of Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.
80. Whitehead, S. D. 1991. A Complexity Analysis of Cooperative Mechanisms in Reinforcement Learning. In Proceedings of the Ninth National Conference on Artificial Intelligence, 607-613. Menlo Park, CA: AAAI Press.
81. Wilson, A.; Fern, A.; Ray, S.; and Tadepalli, P. 2008. Learning and Transferring Roles in Multiagent. In Transfer Learning for Complex Tasks: Papers from the AAAI Workshop. AAAI Technical Report WS-08-13. Menlo Park, CA: AAAI PRess.
82. Wilson, A.; Fern, A.; and Tadepalli, P. 2010. Bayesian Role Discovery for Multiagent Reinforcement Learning. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent System, 1587-1588. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems.
83. Wolpert, D. H., and Tumer, K. 2001. Optimal Payoff Functions for Members of Collectives. Advances in Complex Systems 4(2/3): 265-279.
84. Yang, E., and Gu, D. 2009. Multirobot Systems with Agent-Based Reinforcement Learning: Evolution, Opportunities and Challenges. International Journal on Modelling, Identification and Control 6(4): 271-286.
85. Zinkevich, M. 2003. Online Convex Programming and Generalized Infinitesimal Gradient Ascent. In Proceedings of the Twentieth International Conference on Machine Learning, 928-936. Menlo Park, AAAI Press.