XCS with eligibility traces

GECCO 2005 - Genetic and Evolutionary Computation Conference

Volumn , Issue , 2005, Pages 1851-1858

  Drugowitsch, Jan ^a   Barry, Alwyn M ^a  

^a University of Bath   (United Kingdom)

Author keywords

Eligibility Traces; LCS; Q Learning; Temporal Difference Learning; XCS

Indexed keywords

COMPUTATION THEORY; COMPUTATIONAL METHODS; COMPUTER SCIENCE; GENETIC ALGORITHMS; ROBUSTNESS (CONTROL SYSTEMS);

OPTIMAL SOLUTIONS; REINFORCEMENT LEARNING; SYSTEM PERFORMANCE; XCS LEARNING CLASSIFIER SYSTEMS;

LEARNING SYSTEMS;

EID: 32444435131     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1068009.1068322     Document Type: Conference Paper

References (35)

1. L. C. Baird. Residual algorithms: Reinforcement learning with function approximation. In Machine Learning: Proceeding of the Twelfth International Conference. Morgan Kaufmann Publishers, July 1995.
2. L. C. Baird. Reinforcement Learning Through Gradient Descent. PhD thesis, School of Computer Science. Carnigie Mellon University, Pittsburgh, PA 15213, 1999.
3. A. Barry. Limits in Long Path Learning with XCS. In E. Cantú-Paz, J. A. Foster, K. Deb, D. Davis, R. Roy, U.-M. O'Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. Dowsland, N. Jonoska, and J. Miller, editors, Genetic and Evolutionary Computation - GECCO-2003, volume 2724 of LNCS, pages 1832-1843. Springer-Verlag, 2003.
4. A. M. Barry. XCS Performance and Population Structure within Multiple-Step Environments. PhD thesis, Queens University Belfast, September 2000.
5. A. M. Barry. The Stability of Long Action Chains in XCS. Journal of Soft Computing, 6(3-4):183-199, 2002.
6. E. Bernadó, X. Llorà, and J. M. Garrell. XCS and GALE: a Comparative Study of Two Learning Classifier Systems with Six Other Learning Algorithms on Classification Tasks. In Proceedings of the 4th International Workshop on Learning Classifier Systems (IWLCS-2001), pages 337-341, 2001.
7. M. V. Butz, D. E. Goldberg, and P. L. Lanzi. Gradient Descent Methods in Learning Classifier Systems: Improving XCS Performance in Multistep Problems. Technical Report 2003028, Illinois Genetic Algorithms Laboratory, December 2004.
8. M. V. Butz, T. Kovacs, P. L. Lanzi, and S. W. Wilson. How XCS Evolves Accurate Classifiers. In L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, editors, GECCO-2001: Proceedings of the Genetic and Evolutionary Computation Conference, pages 927-934. Morgan Kaufmann, 2001.
9. M. V. Butz and S. W. Wilson. An Algorithmic Description of XCS. Technical Report 2000017, Illinois Genetic Algorithms Laboratory, 2000.
10. P. Cichosz and J. J. Mulawka. Fast and Efficient Reinforcement Learning with Truncated Temporal Differences. In International Conference on Machine Learning, 1995.
11. P. Cichosz and J. J. Mulawka. Faster Temporal Credit Assignment in Learning Classifier Systems. In Proceedings of the First Polish Conference on Evolutionary Algorithms, 1996.
12. M. Compiani, D. Montanari, R. Serra, and P. Simonini. Learning and Bucket Brigade Dynamics in Classifier Systems. Special issue of Physica D (Vol. 42), 42:202-212, 1990.
13. L. Davis, C. Fu, and S. W. Wilson. An Incremental Multiplexer Problem and Its Uses in Classifier System Research. In P. L. Lanzi, W. Stolzmann, and S. W. Wilson, editors, Advances in Learning Classifier Systems, volume 2321 of LNAI, pages 23-31. Springer-Verlag, Berlin, 2002.
14. M. Dorigo and M. Colombetti. Robot Shaping: An Experiment in Behavior Engineering. MIT Press/Bradford Books, 1998.
15. J. H. Holland. Properties of the Bucket Brigade. In J. J. Grefenstette, editor, Proceedings of the 1st International Conference on Genetic Algorithms and their Applications (ICGA85), pages 1-7. Lawrence Erlbaum Associates: Pittsburgh, PA, July 1985.
16. J. H. Holland, K. J. Holyoak, R. E. Nisbett, and P, R. Thagard. Induction: Processes of Inference, Learning, and Discovery. MIT Press, Cambridge, 1986.
17. J. H. Holland and J. S. Reitman. Cognitive systems based on adaptive algorithms. In D. A. Waterman and F. Hayes-Roth, editors, Pattern-directed Inference Systems. New York: Academic Press, 1978.
18. J. H. Holmes. A genetics-based machine learning approach to knowledge discovery in clinical data. Journal of the American Medical Informatics Association Supplement, 1996.
19. T. Kovacs. Evolving Optimal Populations with XCS Classifier Systems. Master's thesis, School of Computer Science, University of Birmingham, Birmingham, U.K., 1996.
20. P. L. Lanzi. A Study of the Generalization Capabilities of XCS. In T. Back, editor, Proceedings of the 7th International Conference on Genetic Algorithms (ICGA97), pages 418-425. Morgan Kaufmann, 1997.
21. P. L. Lanzi. Learning Classifier Systems from a Reinforcement Learning Perspective. Technical Report 00-03, Dipartimento di Elettronica e Informazione, Politecnico di Milano, 2000.
22. A. Parodi and P. Bonelli. The Animat and the Physician. In J. A. Meyer and S. W. Wilson, editors, From Animals to Animais 1. Proceedings of the First International Conference on Simulation of Adaptive Behavior (SAB90), pages 50-57. A Bradford Book. MIT Press, 1990.
23. R. L. Riolo. Bucket Brigade Performance: I. Long Sequences of Classifiers. In J. J. Grefenstette, editor, Proceedings of the 2nd International Conference on Genetic Algorithms (ICGA87), pages 184-195, Cambridge, MA, July 1987. Lawrence Erlbaum Associates.
24. S. P. Singh, T. Jaakkola, and M. I. Jordan. Reinforcement Learning with Soft State Aggregation. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems, volume 7, pages 361-368. The MIT Press, 1995.
25. S. P. Singh and R. S. Sutton. Reinforcement Learning with Replacing Eligibility Traces. Machine Learning, 22(1-3):123-158, 1996.
26. R. E. Smith, B. A. Dike, R. K. Mehra, B. Ravichandran, and A. El-Fallah. Classifier Systems in Combat: Two-sided Learning of Maneuvers for Advanced Fighter Aircraft. Computer Methods in Applied Mechanics and Engineering, 186(2-4):421-437, 2000.
27. R. Sutton. Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding. Advances in Neural Information Processing Systems, 8:1038-1044, 1996.
28. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998. A Bradford Book.
29. A. Tomlinson. Corporate Classifier Systems. PhD thesis, University of the West of England, 1999.
30. G. Venturini. Apprentissage Adaptatif et Apprentissage Supervisé par Algorithme Génétique. PhD thesis, Université de Paris-Sud, 1994.
31. C. J. Watkins. Learning from delayed rewards. PhD thesis, University of Cambridge, Psychology Department, 1989.
32. C. J. Watkins and P. Dayan. Q-learning. Machine Learning, 8(3):279-292, 1992.
33. S. W. Wilson. Classifier Fitness Based on Accuracy. Evolutionary Computation, 3(2):149-175, 1995.
34. S. W. Wilson. Generalization in the XCS Classifier System. In J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo, editors, Genetic Programming 1998: Proceedings of the Third Annual Conference, pages 665-674. Morgan Kaufmann, 1998.
35. S. W. Wilson. Mining Oblique Data with XCS. Technical Report 2000028, University of Illinois at Urbana-Champaign, 2000.