Biasing coevolutionary search for optimal multiagent behaviors

IEEE Transactions on Evolutionary Computation

Volumn 10, Issue 6, 2006, Pages 629-644

  Panait, Liviu ^b   Luke, Sean ^a,b   Wiegand, R Paul ^c  

^a IEEE   (United States)

^b George Mason University   (United States)

^c NAVAL RESEARCH LABORATORY   (United States)

Author keywords

Biased coevolution; Coevolution; Cooperative coevolution; Multiagent learning; Multipopulation symmetric coevolution; Optimal collaboration

Indexed keywords

BIASED COEVOLUTION; COEVOLUTIONARY SEARCH; COOPERATIVE COEVOLUTION; MULTIAGENT LEARNING; MULTIPOPULATION SYMMETRIC COEVOLUTION; OPTIMAL COLLABORATION;

COMPUTER SUPPORTED COOPERATIVE WORK; GAME THEORY; LEARNING SYSTEMS; MULTI AGENT SYSTEMS; PARAMETER ESTIMATION; ROBUSTNESS (CONTROL SYSTEMS);

EVOLUTIONARY ALGORITHMS;

EID: 33750253706     PISSN: 1089778X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TEVC.2006.880330     Document Type: Article

References (62)

1. R. Wiegand and M. Potter, "Robustness in compositonal coevolution," in Proc. Genetic Evol. Comput. Conf. (GECCO), 2006, pp. 369-376.
2. T. Jansen and R. P. Wiegand, "The cooperative coevolutionary (1 + 1) EA," Evol. Comput., vol. 12, no. 4, pp. 405-434, 2004.
3. R. P. Wiegand, K. A. De Jong, and W. C. Liles, "The effects of representational bias on collaboration methods in cooperative coevolution," in Pwc. 7th Int. Conf. Parallel Problem Solving from Nature (PPSN VII), J. J. M. Guervós, Ed. et al. Berlin, Germany: Springer-Verlag, 2002, pp. 257-268.
4. L. Panait and S. Luke, "Cooperative multi-agent learning: The state of the art," Autonomous Agents and Multi-Agent Systems, vol. 11, no. 3, pp. 387-134, 2005.
5. S. Luke and L. Spector, "Evolving teamwork and coordination with genetic programming," in Proc. 1st Annu. Conf. Genetic Program., J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, Eds., Stanford Univ., Stanford, CA, 1996, pp. 150-156.
6. A. Hara and T. Nagao, "Emergence of cooperative behavior using ADG; automatically defined groups," in Proc. 1999 Genetic and Evol. Comput. Conf. (GECCO-99), 1999, pp. 1038-1046.
7. K. Stanley, B. Bryant, and R. Miikkulainen, "Real-time neuroevolution in the NERO video game," IEEE Trans. Evol. Comput., vol. 9, no. 6, pp. 653-668, Dec. 2005.
8. C. Claus and C. Boutilier, "The dynamics of reinforcement learning in cooperative multiagent systems," in Proc. Nat. Conf. Artif. Intell., 1998, pp. 746-752.
9. M. Lauer and M. Riedmiller, "An algorithm for distributed reinforcement learning in cooperative multi-agent systems," in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 535-542.
10. S. Kapetanakis and D. Kudenko, "Reinforcement learning of coordination in cooperative multi-agent systems," in Proc. 18th Nat. Conf. Artif. Intell., 2002, pp. 326-331.
11. P. Husbands and F. Mill, "Simulated coevolution as the mechanism for emergent planning and scheduling," in Proc. 4th Int. Conf. Genetic Algorithms, R. Belew and L. Booker, Eds., 1991, pp. 264-270.
12. M. Potter and K. De Jong, "A cooperative coevolutionary approach to function optimization," in Proc. 3rd Int. Conf. Parallel Problem Solving from Nature (PPSN III), Y. Davidor and H.-P. Schwefel, Eds., 1994, pp. 249-257.
13. M. Potter, "The design and analysis of a computational model of cooperative coevolution," Ph.D. dissertation, George Mason Univ., Fairfax, VA, 1997.
14. R. P. Wiegand, W. Liles, and K. De Jong, "An empirical analysis of collaboration methods in cooperative coevolutionary algorithms," in Proc. Genetic and Evol. Comput. Conf., L. Spector, Ed. et al., 2001, pp. 1235-1242.
15. L. Bull, "On coevolutionary genetic algorithms," Soft Comput., vol. 5, pp. 201-207, 2001.
16. L. Bull, "Evolutionary computing in multi-agent environments: Partners," in Proc. 7th Int. Conf. Genetic Algorithms, T. Bäck, Ed., 1997, pp. 370-377.
17. R. Eriksson and B. Olsson, "Cooperative coevolution in inventory control optimization," in Proc. 3rd Int. Conf. Artif. Neural Netw. Genetic Algorithms, G. Smith, N. Steele, and R. Albrecht, Eds., Univ. East Anglia, Norwich, U.K., 1997, pp. 583-587.
18. M. Potter and K. De Jong, "Cooperative coevolution: an archecture for evolving coadapted subcomponents," Evol. Comput., vol. 8, no. 1, pp. 1-29, 2000.
19. L. Bull and T. C. Fogarty, "Evolving cooperative communicating classifier systems," in Proc. 3rd Annu. Conf. Evol. Program., A. V. Sebald and L. J. Fogel, Eds., 1994, pp. 308-315.
20. L. Bull, T. C. Fogarty, and M. Snaith, "Evolution in multi-agent systems: Evolving communicating classifier systems for GA in a quadrupedal robot," in Proc. 6th Int. Conf. Genetic Algorithms, L. Eshelman, Ed., 1995, pp. 382-388.
21. M. Potter, L. Meeden, and A. Schultz, "Heterogeneity in the coevolved behaviors of mobile robots: The emergence of specialists," in Proc. 17th Int. Conf. Artif. Intell., 2001, pp. 1337-1343.
22. M. Potter and K. De Jong, "The coevolution of antibodies for concept learning," in Proc. 5th Int. Conf. Parallel Problem Solving from Nature (PPSN V), A. E. Eiben, Ed. et al., 1998, pp. 530-539.
23. M. A. Potter, K. A. De Jong, and J. J. Grefenstette, "A coevolutionary approach to learning sequential decision rules," in Proc. 6th Int. Conf. Genetic Algorithms, 1995, pp. 366-372.
24. R. P. Wiegand, "An analysis of cooperative coevolutionary algorithms," Ph.D. dissertation, George Mason Univ., Fairfax, VA, 2003.
25. R. Axelrod, The Evolution of Cooperation. New York: Basic Books, 1984.
26. R. P. Wiegand, W. Liles, and K. De Jong, "Modeling variation in cooperative coevolution using evolutionary game theory," in Foundations of Genetic Algorithms (FOGA) VII, R. Poli, Ed. et al. San Mateo, CA: Morgan Kaufmann, 2002, pp. 231-248.
27. T. Jansen and R. P. Wiegand, "Exploring the explorative advantage of the CC (1 + 1) EA," in Proc. Genetic and Evol. Comput. Conf., E. Cantú-Paz, Ed. et al., 2003, pp. 310-321.
28. D. Wolpert and W. Macready, "No free lunch theorems for optimization," IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67-82, Feb. 1997.
29. S. Kauffman, The Origins of Order: Self-Organization and Selection in Evolution. Oxford, U.K.: Oxford Univ. Press, 1993.
30. J. M. Smith, Evolution and the Theory of Games. Cambridge, U.K.: Cambridge Univ. Press, 1982.
31. J. Hofbauer and K. Sigmund, Evolutionary Games and Population Dynamics. Cambridge, U.K.: Cambridge Univ. Press, 1998.
32. G. Fogel, P. Andrews, and D. Fogel, "On the instability of evolutionary stable strategies in small populations," Ecological Modeling, vol. 109, pp. 283-294, 1998.
33. S. Ficici and J. Pollack, "A game-theoretic approach to the simple coevolutionary algorithm," in Proc. Sixth Int. Conf. Parallel Problem Solving from Nature (PPSN VI), M. Schoenauer, Ed., 2000, pp. 467-476.
34. S. Ficici, O. Melnik, and J. Pollack, "A game-theoretic and dynamical-systems analysis of selection methods in coevolution," IEEE Trans. Evol. Comput., vol. 9, no. 6, pp. 580-602, Dec. 2005.
35. R. P. Wiegand, W. Liles, and K. De Jong, "Analyzing cooperative coevolution with evolutionary game theory," in Proc. CEC, D. Fogel, Ed., 2002, pp. 1600-1605.
36. A. Liekens, "Evolution of fine populations in dynamic environment," Ph.D. dissertation, Technische Universe Eindhoven, Eindhoven, The Netherlands, 2005.
37. L. M. Schmitt, "Theory of coevolutionary genetic algorithms," in Proc. Int. Symp. Parallel, Distrib. Process. Appl., M. Guo and L. T. Yang, Eds., 2003, pp. 285-293.
38. M. Chang, K. Ohkura, K. Ueda, and M. Sugiyama, "Modeling coevolutionary genetic algorithms on two-bit landscapes: Random partnering," in Proc. Genetic Evol. Comput. Conf., R. Poli, Ed. et al., 2004, pp. 513-524.
39. R. Watson and J. Pollack, "Coevolutionary dynamics in a minimal substrate," in Proc. Genetic Evol. Comput. Conf., L. Spector, Ed. et al., 2001, pp. 702-709.
40. A. Bucci and J. B. Pollack, "Focusing versus intransitivity. Geometrical aspects of co-evolution," in Proc. Genetic and Evol. Comput. Conf. (GECCO), E. Cantú-Paz, Ed. et al., 2003, pp. 250-261.
41. C. Rosin and R. Belew, "New methods for competitive coevolution," Evol. Comput., vol. 5, no. 1, pp. 1-29, 1997.
42. S. Ficici and J. Pollack, "A game-theoretic memory mechanism for coevolution," in Proc. Genetic Evol. Comput. Conf., E. Cantú-Paz, Ed. et al., 2003, pp. 286-297.
43. E. de Jong and J. Pollack, "Ideal evaluation from coevolution," Evol. Comput., vol. 12, no. 2, pp. 159-192, 2004.
44. A. Bucci and J. Pollack, "On identifying global optima in cooperative coevolution," in Proc. Genetic Evol. Comput. Conf. (GECCO), H. Beyer, Ed. et al., 2004, pp. 539-544.
45. J. Sarma, "An analysis of decentralized and spatially distributed genetic algorithms," Ph.D. dissertation, George Mason Univ., Fairfax, VA, 1998.
46. M. Giacobini, E. Alba, and M. Tomassini, "Selection intensy in asynchronous cellular evolutionary algorithms," in Proc. Genetic Evol. Comput. Conf. (GECCO), E. Cantú-Paz, Ed. et al., 2003, pp. 955-966.
47. B. Dorronsoro, E. Alba, M. Giacobini, and M. Tomassini, "The influence of grid shape and asynchronicity on cellular evolutionary algorithms," in Proc. CEC, G. Greenwood, Ed. et al., 2004, pp. 2152-2158.
48. D. Cliff and G. F. Miller, "Tracking the red queen: Measurements of adaptive progress in co-evolutionary simulations," in Proc. 3rd Eur. Conf. Artificial Life, 1995, pp. 200-218.
49. D. Hillis, "Co-evolving parasites improve simulated evolution as an optimization procedure," Artificial Life II, SFI Studies in the Sciences of Complexity, vol. 10, pp. 313-324, 1991.
50. P. Husbands, "Distributed coevolutionary genetic algorithms for multi-criteria and multi-constraint optimization," in Proc. Evol. Comput., AISB Workshop for Selected Papers, 1994, pp. 150-165.
51. A. Ronge and M. G. Nordahl, "Genetic programs and co-evolution developing robust general purpose controllers using local mating in two dimensional populations," in Lecture Notes in Computer Science, H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel, Eds. Berlin, Germany: Springer-Verlag, 1996, Proc. Int. Conf. Evol. Comput. Parallel Problem Solving from Nature IV, pp. 81-90.
52. L. Pagie, "Information integration in evolutionary processes," Ph.D. dissertation, Universiteit Utrecht, Utrecht, The Netherlands, 1999.
53. R. P. Wiegand and J. Sarma, "Spatial embedding and loss of gradient in cooperative coevolutionary algorithms," in Proc. 8th Int. Conf. Parallel Problem Solving from Nature (PPSN VIII), H. P. S. Schwefel, Ed. et al., 2004, pp. 912-922.
54. N. Williams and M. Mitchell, "Investigating the success of spatial coevolution," in Proc. Genetic Evol. Comput. Conf., H. Beyer, Ed. et al., 2004, pp. 523-530.
55. L. Panait, R. P. Wiegand, and S. Luke, "A visual demonstration of convergence properties of cooperative coevolution," in Proc. 8th Int. Conf. Parallel Problem Solving from Nature (PPSN VIII), H. P. S. Schwefel, Ed. et al., 2004, pp. 892-901.
56. S. Luke, A Java EC Research System. 2002 [Online]. Available: http://www.cs.umd.edu/projects/plus/ec/ecj/
57. D. Goldberg and K. Deb, "A comparative analysis of selection schemes used in genetic algorithms," in Foundations of Genetic Algorithms (FOGA), G. Rawlins, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 69-93.
58. D. Whitley, "The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best," in Proc. 3rd Int. Conf. Genetic Algorithms, J. D. Schaffer, Ed., 1989, pp. 116-121.
59. H. Mühlenbein and D. Schlierkamp-Voosen, "Predictive models for the breeder genetic algorithm," Evol. Comput., vol. 1, no. 1, pp. 25-49, 1993.
60. K. Deb, Multi-Objective Optimization Using Evolutionary Algorithms. New York: Wiley, 2001.
61. Y. Sawaragi, H. Nakayama, and T. Tanino, Theory of Multiobjective Optimization. Orlando, FL: Academic, 1985.
62. L. Panait, R. P. Wiegand, and S. Luke, "A sensitivity analysis of a cooperative coevolutionary algorithm biased for optimization," in Proc. Genetic Evol. Comput. Conf., R. Poli, Ed. et al., 2004, pp. 573-584.