Incremental coevolution with competitive and cooperative tasks in a multirobot environment

Proceedings of the IEEE

Volumn 94, Issue 7, 2006, Pages 1412-1423

  Uchibe, Eiji ^b   Asada, Minoru ^a,c  

^a IEEE   (Japan)

^b OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

^c OSAKA UNIVERSITY   (Japan)

Author keywords

Arms race; Convolution; Fitness sharing; Importance sampling; Incremental evolution; Multiple schedules; RoboCup

Indexed keywords

COMPUTER SIMULATION; CONTROL THEORY; MULTI AGENT SYSTEMS; OPTIMAL CONTROL SYSTEMS; PROBLEM SOLVING;

FITNESS SHARING; IMPORTANCE SAMPLING; INCREMENTAL EVOLUTION; MULTIPLE SCHEDULES; ROBOCUP;

ROBOTICS;

EID: 33947193380     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2006.876918     Document Type: Article

References (34)

1. S. Nolfi and D. Floreano, Evolutionary Robotics, The Biology, Intelligence, and Technology of Self-Organizing Machines. Cambridge, MA: MIT Press, 2000.
2. J. Walker, S. Garrett, and W. Wilson, "Evolving controllers for real robots: A survey of the literature," Adapt. Behav., vol. 11, no. 3, pp. 179-203, 2003.
3. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
4. J. R. Koza, Genetic Programming I: On the Programming of Computers by Means of Natural Selection. Cambridge, MA: MIT Press, 1992.
5. S. Nolfi and D. Floreano, "Co-evolving predator and prey robots: Do 'arm races' arise in artificial evolution?" Artif. Life, vol. 4, pp. 311-335, 1998.
6. E. Uchibe, M. Nakamura, and M. Asada, "Cooperative and competitive behavior acquisition for mobile robots through co-evolution," in Proc. Genetic and Evolutionary Computation Conference, 1999, pp. 1406-1413.
7. M. Asada, K. F. MacDorman, H. Ishiguro, and Y. Kuniyoshi, "Cognitive developmental robotics as a new paradigm for the design of humanoid robots," Robot. Auton. Syst., vol. 37, pp. 185-193, 2001.
8. M. Asada, S. Noda, S. Tawaratsumida, and K. Hosoda, "Purposive behavior acquisition for a real robot by vision-based reinforcement learning," Mach. Learn., vol. 23, pp. 279-303, 1996.
9. T. Omata, "Learning with assistance based on evolutionary computation," in Proc. IEEE Int. Conf. Robotics and Automation, 1998, pp. 2180-2186.
10. G. Baldassarre, S. Nolfi, and D. Parisi, "Evolving mobile robots able to display collective behavior," Artif. Life, vol. 9, pp. 255-267, 2003.
11. G. Baldassarre, D. Parisi, and S. Nolfi, "Coordination and behavior integration in cooperating simulated robots," in Proc. 8th Int. Conf. Simulation of Adaptive Behavior: From Animals to Animats 8, S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, and J.-A. Meyer, Eds., 2004, pp. 385-394.
12. M. Quinn, "Evolving communication without dedicated communication channels," in Advances in Artificial Life: Proc. 6th Eur. Conf. Artificial Life (ECAL 2001), J. Kelemen and P. Sosik, Eds., pp. 357-366.
13. M. Quinn, L. Smith, G. Mayley, and P. Husband, "Evolving controllers for a homogeneous system of physical robots: Structured cooperation with minimal sensors," Philos. Trans. R. Soc. Lond. A, Math., Phys. Eng. Sci., vol 361, pp. 2321-2344, 2003.
14. D. Cliff and G. F. Miller, "Co-evolution of pursuit and evasion II: Simulation methods and results," in Proc. 4th Int. Conf. Simulation of Adaptive Behavior: From Animals to Animals 4, 1996, pp. 506-515.
15. G. S. Hornby and B. Mirtich, "Diffuse versus true coevolution in a physics-based world," in Proc. Genetic and Evolutionary Computation Conf., 1999, pp. 1305-1312.
16. D. Floreano and S. Nolfi, "Adaptive behavior in competing co-evolving species," in Proc. 4th Eur. Conf. Artificial Life, 1997, pp. 378-387.
17. C. D. Rosin and R. K. Belew, "New methods for competitive coevolution," Evol. Comput., vol. 5, no. 1, pp. 1-29, 1997.
18. K. O. Stanley and R. Miikklainen, "Competitive coevolution through evolutionary complexification," J. Artif. Intell. Res., vol. 21, pp. 63-100, 2004.
19. M. Asada, H. Kitano, I. Noda, and M. Veloso, "Robocup: Today and tomorrow - what we have learned," Artif. Intell., 1999.
20. M. Asada, E. Uchibe, and K. Hosoda, "Cooperative behavior acquisition for mobile robots in dynamically changing real worlds via vision-based reinforcement learning and development," Artif. Intell., vol. 110, pp. 275-292, 1999.
21. A. Bajurnow and V. Ciesielski, "Layered learning for evolving goal scoring behavior in soccer players," in Proc. 2004 Congr. Evolutionary Computation, 2004, pp. 1828-1835.
22. V. Ciesielski, D. Mawhinney, and P. Wilson, "Genetic programming for robot soccer," in Proc. RoboCup 2001 Int. Symp., 2001, pp. 319-324.
23. W. H. Hsu and S. M. Gustafson, "Genetic programming and multi-agent layered learning by reinforcements," in Proc. Genetic and Evolutionary Computation Conf., 2002, pp. 764-771.
24. W. H. Hsu, S. J. Harmon, E. Rodriguez, and C. Zhong, "Emprical comparison of incremental reuse strategies in genetic programming for keep-away soccer (late breaking paper)," presented at the 2004 Genetic and Evolutionary Computation Conf., Seattle, WA, 2004.
25. S. Luke, C. Hohn, J. Farris, G. Jackson, and J. Hendler, "Co-evolving soccer softbot team coordination with genetic programming," in Proc. 1st RoboCup-97 Workshop at IJCAI'97, pp. 115-118.
26. P. Stone, Layered Learning in Multiagent Systems. Cambridge, MA: MIT Press, 2000.
27. P. Stone, R. Sutton, and G. Kuhlmann, "Reinforcement learning for RoboCup-soccer keepaway," Adapt. Behav., vol. 13, no. 3, pp. 165-188, 2005.
28. S. Whiteson, N. Kohl, R. Miikkulainen, and P. Stone, "Evolving soccer keepaway players through task decomposition," Mach. Learn., vol. 59, no. 1, pp. 5-30, 2005.
29. M. Wiering, R. P. Salustowicz, and J. Schmidhüber, "Reinforcement learning soccer teams with incomplete world models," Auton. Robots, vol. 7, no. 1, pp. 77-88, 1999.
30. R. Rubinstein, Simulation and the Monte Carlo Method. New York: Wiley, 1981.
31. R. S. Sutton and A. G. Barto, Reinforcement Learning. Cambridge, MA: MIT Press/Bradford Books, 1998.
32. D. Precup, R. S. Sutton, and S. Dasgupta, "Off-policy temporal-difference learning with function approximation," in Proc. 18th Int. Conf. Machine Learning, 2001, pp. 417-424.
33. E. Uchibe and K. Doya, "Competitive-cooperative-concurrent reinforcement learning with importance sampling," in Proc. 8th Int. Conf. Simulation of Adaptive Behavior: From Animals to Animats 8, S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, and J.-A. Meyer, Eds., 2004, pp. 287-296.
34. E. Uchibe, M. Yanase, and M. Asada, "Behavior generation for a mobile robot based on the adaptive fitness function," Robot. Auton. Syst., vol. 40, no. 2-3, pp. 69-77, 2002.