Darwinian embodied evolution of the learning ability for survival

Adaptive Behavior

Volumn 19, Issue 2, 2011, Pages 101-120

  Elfwing, Stefan ^a,b   Uchibe, Eiji ^b   Doya, Kenji ^b   Christensen, Henrik I ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^b JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

Embodied evolution; evolutionary robotics; meta learning; metaparameters; reinforcement learning; shaping rewards

Indexed keywords

EID: 79955445257     PISSN: 10597123     EISSN: 17412633     Source Type: Journal    
DOI: 10.1177/1059712310397633     Document Type: Article

References (25)

1. Baldwin, J. (1896). A new factor in evolution. American Naturalist, 30, 441-451.
2. Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4), 485-506.
3. Doya, K., & Uchibe, E. (2005). The Cyber Rodent project: Exploration of adaptive mechanisms for self-preservation and self-reproduction. Adaptive Behavior, 13(2), 149-160.
4. Elfwing, S., Uchibe, U., Doya, K., & Christensen, H.I. (2005). Biologically inspired embodied evolution of survival. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC2005) (Vol. 3)IEEE Press, Piscataway, NJ (pp. 2210-2216).
5. Elfwing, S., Uchibe, U., Doya, K., & Christensen, H. (2007). Evolutionary development of hierarchical learning structures. IEEE Transactions on Evolutionary Computation, 11(2), 249-264.
6. Elfwing, S., Uchibe, U., Doya, K., & Christensen, H. (2008). Coevolution of shaping rewards and meta-parameters in reinforcement learning. Adaptive Behavior, 16(6), 400-412.
7. Eriksson, A., Capi, G., & Doya, K. (2003). Evolution of meta-parameters in reinforcement learning algorithm. In C. S. G. Lee & J. Yuh (Eds.), Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems (IROS2003) (pp. 412-417). Piscataway, NJ: IEEE Press.
8. Floreano, D., & Mondada, F. (1996). Evolution of plastic neurocontrollers for situated agents. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack & S. Wilson (Eds.), Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB1996) (pp. 401-410).
9. Hinton, G., & Nowlan, S. (1987). How learning can guide evolution. Complex Systems, 1, 495-502.
10. Laud, A., & DeJong, G. (2003). The influence of reward on the speed of reinforcement learning: An analysis of shaping. In T. Fawcett & N. Mishra (Eds.), Proceedings of the International Conference on Machine learning (ICML2003) (pp. 440-447).
11. Nehmzow, U. (2002). Physically embedded genetic algorithm learning in multi-robot scenarios: The PEGA algorithm. In C. G. Prince, Y. Demiris, Y. Marom, H. Kozima & C. Balkenius (Eds.), Proceedings of the International Workshop on Epigenetic Robotics and Robotics (EPIROB2002) (pp. 115-123).
12. Ng, A.Y., Harada, D., & Russell, S.J. (1999). Policy invariance under reward transformations: theory and application to reward shaping. In I. Bratko & S. Dzeroski (Eds.), Proceedings of the International Conference on Machine learning (ICML1999) (pp. 278-287). San Francisco, CA: Morgan Kaufmann Publishers Inc.
13. Niv, Y., Joel, D., Meilijson, I., & Ruppin, E. (2002). Evolution of reinforcement learning in uncertain environments: A simple explanation for complex foraging behaviors. Adaptive Behavior, 10(1), 5-24.
14. Nolfi, S., & Floreano, D. (2000). Evolutionary robotics. The biology, intelligence, and technology of self-organizing machines. Cambridge, MA: MIT Press.
15. Rummery, G.A., & Niranjan, M. (1994). On-line Q-learning using connectionist systems. (Tech. Rep. CUED/F-INFENG/TR 166). Cambridge University Engineering Department.
16. Ruppin, E. (2002). Evolutionary autonomous agents: A neuroscience perspective. Nature Review Neuroscience, 3, 132-141.
17. Singh, S.P., & Sutton, R.S. (1996). Reinforcement learning with replacing eligibility traces. Machine Learning, 22(1-3), 123-158.
18. Sutton, R.S. (1996). Generalization in reinforcement learning: Successful examples using sparse coarse coding. In D. S. Touretzky, M. C. Mozer & M. E. Hasselmo (Eds.), Advances in neural information processing systems 8 (pp. 1038-1044). Cambridge, MA: MIT Press.
19. Sutton, R.S., & Barto, A.G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
20. Turney, P., Whitley, D., & Anderson, R. (1996). Introduction to the special issue: Evolution, learning, and instinct: 100 years of the Baldwin effect. Evolutionary Computation, 4(3), iv-viii.
21. Urzelai, J., & Floreano, D. (2000). Evolutionary robotics: Coping with environmental change. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO2000) (pp. 941-948).
22. Usui, Y., & Arita, T. (2003). Situated and embodied evolution in collective evolutionary robotics. In M. Sugisaka & H. Tanaka (Eds.), Proceedings of the International Symposium on Artificial Life and Robotics (AROB2003) (pp. 212-215).
23. Watson, R., Ficici, S., & Pollack, J. (2002). Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robotics and Autonomous Systems, 39(1), 1-18.
24. Whiteson, S., & Stone, P. (2006). Evolutionary function approximation for reinforcement learning. Journal of Machine Learning Research, 7, 877-917.
25. Wiewiora, E. (2003). Potential-based shaping and Q-value initialization are equivalent. Journal of Artificial Intelligence Research, 19, 205-208.