Adaptation technique for integrating genetic programming and reinforcement learning for real robots

IEEE Transactions on Evolutionary Computation

Volumn 9, Issue 3, 2005, Pages 318-333

  Kamio, Shotaro ^a   Iba, Hitoshi ^a  

^a UNIVERSITY OF TOKYO   (Japan)

Author keywords

Adaptation evolutionary computation; Box moving; Genetic programming (GP); Real robot; Reinforcement learning (RL)

Indexed keywords

EVOLUTIONARY ALGORITHMS; GENETIC ALGORITHMS; LEARNING SYSTEMS; ROBOTS;

ADAPTATION EVOLUTIONARY COMPUTATION; BOX MOVING; GENETIC PROGRAMMING (GP); REAL ROBOT; REINFORCEMENT LEARNING (RL);

ROBOTICS;

EID: 21044439703     PISSN: 1089778X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TEVC.2005.850290     Document Type: Article

References (31)

1. J. R. Koza, Genetic Programming, On the Programming of Computers by Means of Natural Selection. Cambridge, MA: MIT Press, 1992.
2. D. P. Muni, N. R. Pal, and J. Das, "A novel approach to design classifiers using genetic programming," IEEE Trans. Evol. Comput., vol. 8, no. 2, pp. 183-196, Apr. 2004.
3. S. Nolfi and D. Floreano, Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines. Cambridge, MA: MIT Press, 2000.
4. M. V. Butz, T. Kovacs, P. L. Lanzi, and S. W. Wilson, "Toward a theory of generalization and learning in XCS," IEEE Trans. Evol. Comput., vol. 8, no. 1, pp. 28-46, Feb. 2004.
5. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
6. B. Andersson, P. Svesson, M. Nordahl, and P. Nordin, "On-line evolution of control for a four-legged robot using genetic programming," in Real-World Applications of Evolutionary Computing. ser. Lecture Notes in Computer Science, S. Cagnoni et al., Eds. Berlin, Germany: Springer-Verlag, 2000, vol. 1803, pp. 319-326.
7. K. Yanai and H. Iba, "Multi-agent robot learning by means of genetic programming: solving an escape problem," in Proc. 4th Int. Conf. Evolvable Syst. ICES'2001, Evolvable Syst. From Biol. Hardware., Y. Liu et al., Eds., Tokyo, 2001, pp. 192-203.
8. D. Floreano and F. Mondada, "Evolution of homing navigation in a real mobile robot," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 26, no. 3, pp. 396-407, 1996.
9. H. Kimura, T. Yamashita, and S. Kobayashi, "Reinforcement learning of walking behavior for a four-legged robot," in Proc. 40th IEEE Conf. Decision Control, 2001, pp. 411-416.
10. M. Asada, S. Noda, S. Tawaratsumida, and K. Hosoda, "Purposive behavior acquisition for a real robot by vision-based reinforcement learning," Mach. Learn., vol. 23, pp. 279-303, 1996.
11. N. Jacobi, P. Husbands, and I. Harvey, "Noise and the reality gap: The use of simulation in evolutionary robotics," in Lecture Notes in Artificial Intelligence, vol. 929, Proc. ECAL'95, 1995, pp. 704-720.
12. [Online]. Available: http://www.sony.net/Products/aibo/
13. [Online]. Available: http://www.automation.fujitsu.com/en/products/ products07.html
14. H. Iba, "Multi-agent reinforcement learning with genetic programming," in Proc. 3rd Annu. Genetic Program. Conf., 1998, pp. 167-175.
15. K. L. Downing, "Adaptive genetic programs via reinforcement learning," in Proc. Genetic Evol. Comput. Conf. (GECCO-2001), L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, Eds., San Francisco, CA, 2001, pp. 19-26.
16. W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming - An Introduction. San Mateo, CA: Morgan Kaufmann, 1998.
17. OPEN-R Programming Special Interest Group, "Introduction to OPEN-R programming," (in Japanese), Impress Corporation, LOCATION, 2002.
18. T. Minato and M. Asada, "Environmental change adaptation for mobile robot navigation," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 1998, pp. 1859-1864.
19. D. E. Moriarty, A. C. Schultz, and J. J. Grefenstette, "Evolutionary algorithms for reinforcement learning," J. Artif. Intell. Res., vol. 11, pp. 199-229, 1999.
20. M. Dorigo and M. Colombetti, Robot Shaping: An Experiment in Behavior Engineering. Cambridge, MA: MIT Press, 1998.
21. A. C. Schultz, C. L. Ramsey, and J. J. Grefenstette, "Simulation-assisted learning by competition: Effects of noise differences between training model and target environment," in Proc. 7th Int. Conf. Mach. Learn., 1990, pp. 211-215.
22. Y. Takahashi, M. Asada, S. Noda, and K. Hosoda, "Sensor space segmentation for mobile robot learning," in Proc. ICMAS'96 Workshop Learn. Interaction Organ. Multiagent Environ., 1996.
23. J. Ziegler, J. Barnholt, J. Busch, and W. Banzhaf, "Automatic evolution of control programs for a small humanoid walking robot," in Proc. 5th Int. Conf. Climbing Walking Robots ( CLAWAR), 2002, pp. 109-116.
24. K. Endo, F. Yamasaki, T. Maeno, and H. Kitano, "Co-evolution of morphology and controller for biped humanoid robot," in Lecture Notes in Computer Science, vol. 2752, Proc. RoboCup 2002, G. A. Kaminka, P. U. Lima, and R. Rojas, Eds., 2003, pp. 327-341.
25. G. S. Hornby, M. Fujita, S. Takamura, T. Yamamoto, and O. Hanagata, "Autonomous evolution of gaits with the Sony quadruped robot," in Proc. Genetic Evol. Comput. Conf., vol. 2, Orlando, FL, Jul. 1999, pp. 1297-1304.
26. J. P. Nordin and M. Nordahl, "An evolutionary architecture for a humanoid robot," presented at the 4th Int. Symp. Artif. Life Robotics, Oita, Japan, 1999.
27. C. T. Graae, P. Nordin, and M. Nordahl, "Stereoscopic vision for a humanoid robot using genetic programming," in Real-World Applications of Evolutionary Computing. ser. Lecture Notes in Computer Science, S. Cagnoni et al., Eds. Berlin, Germany: Springer-Verlag, 2000, vol. 1803, pp. 12-21.
28. W. B. Langdon and P. Nordin, "Evolving hand-eye coordination for a humanoid robot with machine code genetic programming," in Lecture Notes in Computer Science, vol. 2038, Proc. 4th Eur. Conf. EuroGP 2001, J. Miller et al., Eds., Lake Como, Italy, Apr. 18-20, 2001, pp. 313-324.
29. E. Uchibe, M. Nakamura, and M. Asada, "Cooperative behavior acquisition in a multiple mobile robot environment by co-evolution," in RoboCup-98: Robot Soccer World Cup II. ser. Lecture Notes in Computer Science, H. K. M. Asada, Ed. Berlin, Germany: Springer-Verlag, 1999, vol. 1604, pp. 273-285.
30. Y. Inoue, T. Tohge, and H. Iba, "Cooperative transportation by humanoid robots - learning to correct positioning," in Proc. Hybrid Intell. Syst., 2003, pp. 1124-1133.
31. S. Kamio and H. Iba, "Evolutionary construction of a simulator for real robots," in Proc. Congr. Evol. Comput., 2004, pp. 2202-2209.