Coevolution of role-based cooperation in multiagent systems

IEEE Transactions on Autonomous Mental Development

Volumn 1, Issue 3, 2009, Pages 170-186

  Yong, Chern Han ^a   Miikkulainen, Risto ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b University of Texas at Austin   (United States)

Author keywords

Coevolution; Communication; Cooperation; Heterogeneous teams; Multiagent systems; Neuroevolution; Prey capture task; Stigmergy

Indexed keywords

COEVOLUTION; MULTI-AGENT; MULTIPLE AGENTS; NEUROEVOLUTION; PREY-CAPTURE; ROLE-BASED; STIGMERGY;

NEURAL NETWORKS;

MULTI AGENT SYSTEMS;

EID: 77955686719     PISSN: 19430604     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAMD.2009.2037732     Document Type: Article

References (65)

1. C. W. Anderson, "Learning to control an inverted pendulum using neural networks," IEEE Control Syst. Mag., vol.9, no.3, pp. 31-37, Apr. 1989.
2. T. Balch, "Learning roles: Behavioral diversity in robot teams," in Multiagent Learning: Papers From the AAAI Workshop. Technical Report WS-97-03, S. Sen, Ed., Menlo Park, CA, 1997, pp. 7-12.
3. G. Baldassarre, S. Nolfi, and D. Parisi, "Evolving mobile robots able to display collective behaviors," Artif. Life, vol.9, pp. 255-267, 2003.
4. A. G. Barto, R. S. Sutton, and C. W. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Trans. Syst., Man, Cybern., vol.SMC-13, no.5, pp. 834-846, Sep. 1983.
5. J. Batali, "Innate biases and critical periods: Combining evolution and learning in the acquisition of syntax," in Proc. 4th Int.Workshop Synth. Simul. Living Syst., R. A. Brooks and P. Maes, Eds., Cambridge, MA, 1994, pp. 160-171, MIT Press.
6. M. Benda, V. Jagannathan, and R. Dodhiawala, "On optimal cooperation of knowledge sources-An empirical investigation," Boeing Adv. Tech. Center, Tech. Rep. BCS-G2010-28, 1986.
7. B. D. Bryant and R. Miikkulainen, "Neuroevolution for adaptive teams," in Proc. Congr. Evol. Comput., Piscataway, NJ IEEE, 2003, vol.3, pp. 2194-2201.
8. E. di Paolo, "Behavioral coordination, structural congruence and entrainment in a simulation of acoustically coupled agents," Adapt. Behav., vol.8, pp. 25-46, 2000.
9. J. L. Elman, "Incremental learning, or the importance of starting small," in Proc. 13th Annu. Conf. Cogn. Sci. Soc., K. J. Hammond and D. Gentner, Eds., Hillsdale, NJ, 1991, pp. 443-448, Erlbaum.
10. S. G. Ficici and J. B. Pollack, "Coevolving communicative behavior in a linear pursuer-evader game," in Proc. 5th Int. Conf. Simul. Adapt. Behav., R. Pfeifer, B. Blumberg, J.-A. Meyer, and S. W. Wilson, Eds., Cambridge, MA, 1998, pp. 505-514, MIT Press.
11. D. Floreano, S. Mitri, S. Magnenat, and L. Keller, "Evolutionary conditions for the emergence of communication in robots," Current Biol., vol.17, pp. 514-519, 2007.
12. M. S. Fontán and M. J. Matarić, "Territorial multi-robot task division," IEEE Trans. Robot. Autom., vol.14, no.5, pp. 815-822, Oct. 1998.
13. S. Franklin, "CoordinationWithout Communication," Inst. Intell. Syst., Dept. Math. Sci., Univ. Memphis, Memphis, TN, , 1996.
14. F. Gomez, "Robust non-linear control through neuroevolution," Ph.D. dissertation, Dept. Comput. Sci., Univ. Texas at Austin, Austin, TX, 2003.
15. F. Gomez and R. Miikkulainen, "Incremental evolution of complex general behavior," Adapt. Behav., vol.5, pp. 317-342, 1997.
16. F. Gomez and R. Miikkulainen, "Solving non-Markovian control tasks with neuroevolution," in Proc. 16th Int. Joint Conf. Artif. Intell., San Francisco, CA, 1999, pp. 1356-1361, Kaufmann.
17. F. Gomez, J. Schmidhuber, and R. Miikkulainen, "Efficient non-linear control through neuroevolution," in Proc. Euro. Conf. Mach. Learn., Berlin, Germany, 2006, pp. 654-662, Springer-Verlag.
18. F. Gomez, J. Schmidhuber, and R. Miikkulainen, "Accelerated neural evolution through cooperatively coevolved synapses," J. Mach. Learn. Res., vol.9, pp. 937-965, 2008.
19. P.-P. Grassé, "La reconstruction du nid et les coordinations inter-individuelles chez Bellicositermes natalensis et Cubitermes sp. la théorie de la stigmergie: Essai d'interpretation des termites constructeurs," Insectes Sociaux, vol.6, pp. 41-83, 1959.
20. F. Gruau, D. Whitley, and L. Pyeatt, "A comparison between cellular encoding and direct encoding for genetic neural networks," in Proc. 1st Annu. Conf. Genet. Progr., J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, Eds., Cambridge, MA, 1996, pp. 81-89, MIT Press.
21. N. Hansen and A. Ostermeier, "Completely derandomized self-adaptation in evolution strategies," Evol. Comput., vol.9, pp. 159-195, 2001.
22. T. Haynes and S. Sen, "Evolving behavioral strategies in predators and prey," in Adaptation and Learning in Multi-Agent Systems,G.Weib and S. Sen, Eds. Berlin, Germany: Springer-Verlag, 1996, pp. 113-126.
23. T. Haynes and S. Sen, "Crossover operators for evolving a team," in Proc. 2nd Annu. Conf. Genet. Progr., J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo, Eds., San Francisco, CA, 1997, pp. 162-167, Kaufmann.
24. T. D. Haynes and S. Sen, "Co-adaptation in a team," Int. J. Comput. Intell. Org., vol.1, pp. 231-233, 1997.
25. J. H. Holland, "Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems," in Machine Learning: An Artificial Intelligence Approach, R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds. San Francisco, CA: Kaufmann, 1986, vol.2, pp. 275-304.
26. O. Holland and C. Melhuish, "Stigmergy, self-organization, and sorting in collective robotics," Artif. Life, vol.5, pp. 173-202, 1999.
27. K.-C. Jim and C. L. Giles, "Talking helps: Evolving communicating agents for the predator-prey pursuit problem," Artif. Life, vol.6, no.3, pp. 237-254, 2000.
28. Y. le Cun, J. S. Denker, and S. A. Solla, "Optimal brain damage," in Advances in Neural Information Processing Systems 2,D. S. Touretzky, Ed. San Francisco, CA: Kaufmann, 1990, pp. 598-605.
29. V. R. Lesser, "Cooperative multiagent systems: A personal view of the state of the art," IEEE Trans. Knowl. Data Eng., vol.11, no.1, pp. 122-142, Jan. 1999.
30. S. Luke, "Genetic programming produced competitive soccer softbot teams for Robocup97," in Proc. 3rd Annu, Conf. Genet. Progr., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo, Eds., San Francisco, CA, 1998, pp. 214-222, Kaufmann.
31. S. W. Mahfoud, "Niching Methods for Genetic Algorithms," Ph.D. dissertation, Univ. Illinois at Urbana-Champaign, Urbana, IL, 1995.
32. N. Meuleau, L. Peshkin, K.-E. Kim, and L. P. Kaelbling, "Learning finite-state controllers for partially observable environments," in Proc. 15th Int. Conf. Uncertainty Artif. Intell., San Francisco, CA, 1999, pp. 427-436, Kaufmann.
33. G. Miller and D. Cliff, "Co-evolution of pursuit and evasion I: biological and game-theoretic foundations" School Cogn. Comput. Sci., Univ. Sussex, Brighton, U.K., Tech. Rep. CSRP311, 1994.
34. M. Mitchell, "Coevolutionary learning with spatially distributed populations," in Computational Intelligence: Principles and Practice, G. Y. Yen and D. B. Fogel, Eds. Piscataway, NJ: IEEE Computational Intelligence Society, 2006.
35. D. E. Moriarty, "Symbiotic evolution of Nneural networks in sequential decision tasks," Ph.D. dissertation, Dept. Comput. Sci., Univ. Texas at Austin, Austin, TX, 1997.
36. D. E. Moriarty and R. Miikkulainen, "Efficient reinforcement learning through symbiotic evolution," Mach. Learn., vol.22, pp. 11-32, 1996.
37. D. E. Moriarty and R. Miikkulainen, "Forming neural networks through efficient and adaptive co-evolution," Evol. Comput., vol.5, pp. 373-399, 1997.
38. D. E. Moriarty, A. C. Schultz, and J. J. Grefenstette, "Evolutionary algorithms for reinforcement learning," J. Artif. Intell. Res., vol.11, pp. 199-229, 1999.
39. H. Mühlenbein, M. Schomisch, and J. Born, "The parallel genetic algorithm as function optimizer," Parallel Comput., vol.17, pp. 619-632, 1991.
40. M. Pendrith, "On reinforcement learning of control actions in noisy and non-markovian domains," School Comput. Sci. Eng., Univ. New South Wales, Sydney, Australia, Tech. Rep. UNSW-CSE-TR-9410, 1994.
41. M. A. Potter and K. A. D. Jong, "Cooperative coevolution: An architecture for evolving coadapted subcomponents," Evol. Comput., vol.8, pp. 1-29, 2000.
42. M. Quinn, L. Smith, G. Mayley, and P. Husbands, "Evolving teamwork and role-allocation with real robots," in Proc. 8th Int. Conf. Artif. Life, Cambridge, MA, 2002, pp. 302-311, MIT Press.
43. S. Ramachandran and L. Y. Pratt, "Information measure based skeletonisation," in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. San Francisco, CA: Kaufmann, 1992, pp. 1080-1087.
44. J. Reisinger, K. O. Stanley, and R. Miikkulainen, "Evolving reusable neural modules," in Proc. Genet. Evol. Comput. Conf., New York, NY, 2004, pp. 69-81, Springer-Verlag.
45. C. D. Rosin and R. K. Belew, "New methods for competitive evolution," Evol. Comput., vol.5, pp. 1-99, 1997.
46. J. Santamaria, R. Sutton, and A. Ram, "Experiments with reinforcement learning in problems with continuous state and action spaces," Adapt. Behav., vol.6, no.2, pp. 163-217, 1998. (Pubitemid 127175211)
47. N. Saravanan and D. B. Fogel, "Evolving neural control systems," IEEE Expert, vol.10, no.3, pp. 23-27, Jun. 1995.
48. G. M. Saunders and J. B. Pollack, "The evolution of communication schemes over continuous channels," in Proc. 4th Int. Conf. Simul. Adapt. Behav., P. Maes, M. J. Mataric, J.-A. Meyer, J. Pollack, and S. W. Wilson, Eds., Cambridge, MA, 1996, pp. 580-589, MIT Press.
49. T. Savage, "Shaping: The link between rats and robots," Connect. Sci., vol.10, pp. 321-340, 1998.
50. R. E. Smith, S. Forrest, and A. S. Perelson, "Searching for diverse, cooperative populations with genetic algorithms," Evol. Comput., vol.1, pp. 127-149, 1993.
51. S. F. Smith, "A learning system based on genetic adaptive algorithms," Ph.D. dissertation, Dept. Comput. Sci., Univ. Pittsburgh, Pittsburgh, PA, 1980.
52. K. O. Stanley and R. Miikkulainen, "Evolving neural networks through augmenting topologies," Evol. Comput., vol.10, pp. 99-127, 2002.
53. K. O. Stanley and R. Miikkulainen, "Competitive coevolution through evolutionary complexification," J. Artif. Intell. Res., vol.21, pp. 63-100, 2004.
54. L. Steels, "Self-organising vocabularies," in Proc. 5th Int. Workshop Artif. Life, C. G. Langton and K. Shimohara, Eds., Cambridge, MA, 1996, pp. 179-184, MIT Press.
55. R. S. Sutton, D. McAllester, S. Sing, and Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation," in Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, 2001, pp. 1057-1063.
56. J. Urzelai, D. Floreano, M. Dorigo, and M. Colombetti, "Incremental robot shaping," Connect. Sci., vol.10, pp. 341-360, 1998.
57. K. Wagner, "Cooperative strategies and the evolution of communication," Artif. Life, vol.6, pp. 149-179, 2000.
58. C. J. C. H. Watkins and P. Dayan, "Q-learning," Mach. Learn., vol.8, no.3, pp. 279-292, 1992.
59. G. M. Werner and M. G. Dyer, "Evolution of communication in artificial organisms," in Proc. Workshop Artif. Life, C. G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen, Eds., Reading, MA, 1991, pp. 659-687, Addison-Wesley.
60. D. Whitley, S. Dominic, R. Das, and C. W. Anderson, "Genetic reinforcement learning for neurocontrol problems," Mach. Learn., vol.13, pp. 259-284, 1993.
61. D. Whitley, F. Gruau, and L. Pyeatt, "Cellular encoding applied to neurocontrol," in Proc. 6th Int. Conf. Genet. Algor., L. J. Eshelman, Ed., San Francisco, CA, 1995, pp. 460-469, Kaufmann.
62. D. Whitley, K. Mathias, and P. Fitzhorn, "Delta-Coding: An iterative search strategy for genetic algorithms," in Proc. 4th Int. Conf. Genet. Algor., R. K. Belew and L. B. Booker, Eds., San Francisco, CA, 1991, pp. 77-84, Kaufmann.
63. R. P.Wiegand, "An analysis of cooperative coevolutionary algorithms," PhD dissertation, George Mason Univ.,Washinton, D.C., 2003.
64. M. Wooldridge, An Introduction to Multiagent Systems. Hoboken, NJ: Wiley, 2002.
65. M. J. Wooldridge and N. R. Jennings, "Cooperative problem solving," J. Logic Comput., vol.9, pp. 563-592, 1999.