Comparing evolutionary and temporal difference methods in a reinforcement learning domain

GECCO 2006 - Genetic and Evolutionary Computation Conference

Volumn 2, Issue , 2006, Pages 1321-1328

  Taylor, Matthew E ^a   Whiteson, Shimon ^a   Stone, Peter ^a  

^a University of Texas at Austin   (United States)

Author keywords

Empirical study; Genetic algorithms; Machine learning; Performance analysis

Indexed keywords

GENETIC ALGORITHMS; NEURAL NETWORKS; PUBLIC POLICY;

EMPIRICAL STUDY; PERFORMANCE ANALYSIS; TEMPORAL DIFFERENCE (TD);

LEARNING SYSTEMS;

EID: 33750259111     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1143997.1144202     Document Type: Conference Paper

References (28)

1. T. Beielstein and S. Markon. Threshold selection, hypothesis tests and DOE methods. In 2002 Congresss on Evolutionary Computation, pages 777-782, 2002.
2. S. J. Bradtke and M. O. Duff. Reinforcement learning methods for continuous-time Markov decision problems. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems 7, pages 393-400, San Mateo, CA, 1995. Morgan Kaufmann.
3. D. E. Goldberg and J. Richardson. Genetic algorithms with sharing for multimodal function optimization. In Proceedings of the Second International Conference on Genetic Algorithms, pages 148-154, 1987.
4. F. Gomez and R. Miikkulainen. Robust nonlinear control through neuroevolution. Technical Report AI02-292, Department of Computer Sciences, University of Texas at Austin, 2002.
5. F. Gomez and J. Schmidhuber. Co-evolving recurrent neurons learn deep memory POMDPs. In GECCO-05: Proceedings of the Genetic and Evolutionary Computation Conference, pages 491-498, 2005.
6. F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 81-89. MIT Press, 1996.
7. W. H. Hsu and S. M. Gustafson. Genetic programming and multi-agent layered learning by reinforcements. In Genetic and Evolutionary Computation Conference, pages 764-771, New York,NY, July 2002. Morgan Kaufmann.
8. K. Kostiadis and H. Hu. KaBaGe-RL: Kanerva-based generalisation and reinforcement learning for possession football. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2001), October 2001.
9. D. E. Moriarty and R. Miikkulainen. Efficient reinforcement learning through symbiotic evolution. Machine Learning, 22:11-32, 1996.
10. I. Noda, H. Matsubara, K. Hiraki, and I. Frank. Soccer server: A tool for research on multiagent systems. Applied Artificial Intelligence, 12:233-250, 1998.
11. J. B. Pollack, A. D. Blair, and M. Land. Coevolution of a backgammon player. In C. G. Langton and K. Shimohara, editors, Artificial Life V: Proc. of the Fifth Int. Workshop on the Synthesis and Simulation of Living Systems, pages 92-98, Cambridge, MA, 1997. The MIT Press.
12. M. A. Potter and K. A. De Jong. Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evolutionary Computation, 8:1-29, 2000.
13. M. J. D. Powell. Radial basis functions for multivariate interpolation: A review. In J. C. Mason and M. G. Cox, editors, Algorithms for Approximation, pages 143-167, Oxford, 1987. Clarendon Press.
14. M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., 1994.
15. N. J. Radcliffe. Genetic set recombination and its application to neural network topology optimization. Neural computing and applications, 1(1):67-90, 1993.
16. G. A. Rummery and M. Niranjan. On-line Q-learning using connectionist systems. Technical Report CUED/F-INFENG-RT 116, Engineering Department, Cambridge University, 1994.
17. S. P. Singh and R. S. Sutton. Reinforcement learning with replaceing eligibility traces. Machine Learning, 22:123-158, 1996.
18. P. Stagge. Averaging efficiently in the presense of noise. In PPSN V: Proceedings of the 5th Internation Conference on Parallel Problem Solving from Nature, pages 188-200, London, UK, 1998. Springer-Verlag.
19. K. O. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10(2):99-127, 2002.
20. K. O. Stanley and R. Miikkulainen. Competitive coevolution through evolutionary complexification. Journal of Artificial Intelligence Research, 21:63-100, 2004.
21. K. O. Stanley and R. Miikkulainen. Evolving a roving eye for go. In Proceedinngs of the Genetic and Evolutionary Computation Conference, 2004.
22. P. Stone, G. Kuhlmann, M. E. Taylor, and Y. Liu. Keepaway soccer: From machine learning testbed to benchmark. In I. Noda, A. Jacoff, A. Bredenfeld, and Y. Takahashi, editors, RoboCup-2005: Robot Soccer World Cup IX. Springer Verlag, Berlin, 2006. To appear.
23. P. Stone, R. S. Sutton, and G. Kuhlmann. Reinforcement learning for RoboCup-soccer keepaway. Adaptive Behavior, 13(3): 165-188, 2005.
24. R. S. Sutton. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In Advances in Neural Information Processing Systems 8, pages 1038-1044, 1996.
25. R. S. Sutton and A. G. Barto. Introduction to Reinforcement Learning. MIT Press, 1998.
26. S. Whiteson, N. Kohl, R. Miikkulainen, and P. Stone. Evolving keepaway soccer players through task decomposition. Machine Learning, 59(1):5-30, 2005.
27. S. Whiteson and P. Stone. Evolutionary function approximation for reinforcement learning. Journal of Machine Learning Research, 2006. To appear.
28. X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423-1447, 1999.