Theoretical advantages of lenient learners: An evolutionary game theoretic perspective

Journal of Machine Learning Research

Volumn 9, Issue , 2008, Pages 423-457

  Panait, Liviu ^a   Tuyls, Karl ^b   Luke, Sean ^c  

^a GOOGLE INC   (United States)

^b MAASTRICHT UNIVERSITY   (Netherlands)

^c George Mason University   (United States)

Author keywords

Basins of attraction; Cooperative coevolution; Evolutionary game theory; Formal models; Multiagent learning; Reinforcement learning; Visualization

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; EVOLUTIONARY ALGORITHMS; GAME THEORY; MULTI AGENT SYSTEMS;

BASINS OF ATTRACTION; COOPERATIVE COEVOLUTION; EVOLUTIONARY GAME THEORY; FORMAL MODELS; MULTIAGENT LEARNING;

REINFORCEMENT LEARNING;

EID: 41549123971     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (41)

1. Adrian K. Agogino and Kagan Turner. Evolving distributed agents for managing air traffic. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1888 -1895, 2007.
2. Adrian K. Agogino and Kagan Turner. Handling communication restrictions and team formation in congestion games. Journal of Autonomous Agents and Multi Agent Systems, 13(1):97-11.5, 2006.
3. Stephane Airiau, Kagan Turner, and Adrian K. Agogino. Learning agents for distributed and robust spacecraft power management. In AAMAS-06 Workshop on Adaptation and Learning in Autonomous Agents and Multi Agent Systems, 2006.
4. Caroline Claus and Craig Boutilier. The dynamics of reinforcement learning in cooperative multi-agent systems. In Proceedings of National Conference on Artificial Intelligence (AAAI-98), pages 746-752,1998.
5. Luc Devroye. Non-Uniform Random Varíate Generation. Springer-Verlag, 1986.
6. Roger Eriksson and Björn Olsson. Cooperative coevolution in inventory control optimisation. In G. Smith, N. Steele, and R. Albrecht, editors, Artificial Neural Nets and Genetic Algorithms: Proceedings of the International Conference, ICANNGA97, 1997.
7. Nancy Fulda and Dan Ventura. Predicting and preventing coordination problems in cooperative learning systems. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-07), 2007.
8. Herbert Gintis. Game Theory Evolving: A Problem-Centered Introduction to Modeling Strategic Interaction. Princeton University Press, 2001.
9. Faustino Gomez, Jürgen Schmidhuber, and Risto Miikkulainen. Efficient non-linear control through neuroevolution. In Lecture Notes in Computer Science (LNCS 4212), pages 654-662. 2006.
10. Josef Hofbauer and Karl Sigmund. Evolutionary Games and Population Dynamics. Cambridge University Press, 1998.
11. Phil Husbands and Frank Mill. Simulated coevolution as the mechanism for emergent planning and scheduling. In Proceedings of the Fourth International Conference on Genetic Algorithms, pages 264-270,1991.
12. Thomas Jansen and R. Paul Wiegand. Exploring the explorative advantage of the CC (1+1) EA. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pages 310-321, 2003.
13. Thomas Jansen and R. Paul Wiegand. The cooperative coevolutionary (1+1) ea. Evolutionary Computation, 12(4):405-434, 2004.
14. Kenneth De Jong. Evolutionary Computation: A Unified Approach. MIT Press, 2006.
15. Spiros Kapetanakis and Daniel Kudenko. Reinforcement learning of coordination in cooperative multi-agent systems. In Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI-02), 2002.
16. Matt Knudson and Kagan Turner. Effective policies for resource limited agents. In Proceedings of the AAMAS-07 Workshop on Adaptive and Learning Agents Workshop (ALAg-07), 2007.
17. Martin Lauer and Martin Riedmiller. An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 535-542, 2000.
18. John Maynard Smith and George R. Price. The logic of animal conflict. Nature, 146:15-18, 1973.
19. Liviu Panait and Sean Luke. Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11(3):387-434, 2005a.
20. Liviu Panait and Sean Luke. Time-dependent collaboration schemes for cooperative coevolutionary algorithms. In Proceedings of the 2005 AAAI Fall Symposium on Coevolutionary and Coadaptive Systems, 2005b.
21. Liviu Panait and Sean Luke. Selecting informative actions improves cooperative multiagent learning. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multi Agent Systems - AAMAS-2006, 2006.
22. Liviu Panait, R. Paul Wiegand, and Sean Luke. Improving coevolutionary search for optimal multiagent behaviors. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003.
23. Liviu Panait, R. Paul Wiegand, and Sean Luke. A visual demonstration of convergence properties of cooperative coevolution. In Parallel Problem Solving from Nature - PPSN-2004, pages 892-901, 2004.
24. Liviu Panait, Keith Sullivan, and Sean Luke. Lenience towards teammates helps in cooperative multiagent learning. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multi Agent Systems - AAMAS-2006, 2006.
25. Mitchell Potter. The Design and Analysis of a Computational Model of Cooperative CoEvolution. PhD thesis, George Mason University, Fairfax, Virginia, 1997.
26. Mitchell Potter and Kenneth De Jong. A cooperative coevolutionary approach to function optimization. In Proceedings of the Third International Conference on Parallel Problem Solving from Nature (PPSN III), pages 249-257, 1994.
27. Mitchell Potter and Kenneth De Jong. Cooperative coevolution: An architecture for evolving coad-apted subcomponents. Evolutionary Computation, 8(1): 1-29, 2000.
28. Satinder P. Singh, Michael J. Kearns, and Yishay Mansour. Nash convergence of gradient dynamics in general-sum games. In UAI '00: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pages 541-548, 2000.
29. John Maynard Smith. Evolution and the Theory of Games. Cambridge University Press, 1982.
30. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
31. Kagan Turner and Adrian K. Agogino. Coordinating multi-rover systems: Evaluation functions for dynamic and noisy environments. In Evolutionary Computation in Dynamic and Uncertain Environments, pages 371-388. Springer, 2007.
32. Karl Tuyls, Katja Verbeeck, and Tom Lenaerts. A selection-mutation model for q-learning in multi-agent systems. In The Second International Joint Conference on Autonomous Agents and MultiAgent Systems. ACM Press, Melbourne, Australia, 2003.
33. Karl Tuyls, Pieter Jan 't Hoen, and Bram Vanschoenwinkel. An evolutionary dynamical analysis of multi-agent learning in iterated games. The Journal of Autonomous Agents and Multi-Agent Systems, 12:115-153, 2006.
34. Michael Vose. The Simple Genetic Algorithm. MIT Press, 1999.
35. Christopher J. Watkins. Models of Delayed Reinforcement Learning. PhD thesis, Psychology Department, Cambridge University, Cambridge, United Kingdom, 1989.
36. Christopher J. Watkins and Peter Dayan. Q-learning. Machine Learning, 8:279-292, 1992.
37. Jorgen W. Weibull. Evolutionary Game Theory. MIT Press, 1996.
38. R. Paul Wiegand. An Analysis of Cooperative Coevolutionary Algorithms. PhD thesis, George Mason University, Fairfax, Virginia, 2004.
39. R. Paul Wiegand and Mitchell Potter. Robustness in cooperative coevolution. In GECCO '06: Proceedings of the 8th annual Conference on Genetic and Evolutionary Computation, pages 369-376, 2006.
40. R. Paul Wiegand, William Liles, and Kenneth De Jong. An empirical analysis of collaboration methods in cooperative coevolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pages 1235-1242, 2001.
41. R. Paul Wiegand, William Liles, and Kenneth De Jong. Modeling variation in cooperative coevolution using evolutionary game theory. In Foundations of Genetic Algorithms VII, pages 231-248, 2002.