Gradient descent methods in learning classifier systems: Improving XCS performance in multistep problems

IEEE Transactions on Evolutionary Computation

Volumn 9, Issue 5, 2005, Pages 452-473

  Butz, Martin V ^a   Goldberg, David E ^a   Lanzi, Pier Luca ^a,b  

^a University of Illinois at Urbana Champaign   (United States)

^b POLITECNICO DI MILANO   (Italy)

Author keywords

Function approximation; Gradient descent; Learning classifier systems (LCSs); Multistop problems; Q learning; Reinforcement learning; XCS

Indexed keywords

APPROXIMATION THEORY; DATA MINING; EVOLUTIONARY ALGORITHMS; GRADIENT METHODS; KNOWLEDGE BASED SYSTEMS; MARKOV PROCESSES;

FUNCTION APPROXIMATION; GRADIENT DESCENT METHOD; LEARNING CLASSIFIER SYSTEMS (LCSS); MULTISTOP PROBLEMS; Q-LEARNING; REINFORCEMENT LEARNING;

LEARNING SYSTEMS;

EID: 27344437190     PISSN: 1089778X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TEVC.2005.850265     Document Type: Article

References (31)

1. L. C. Baird, "Residual algorithms: Reinforcement learning with function approximation," in Proc. 12th Int. Conf. Mach. Learn., Jul. 1995, pp. 30-37.
2. L. C. Baird, "Reinforcement learning through gradient descent," Ph.D. dissertation, School of Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, 1999.
3. A. M. Barry, "The stability of long action chains in XCS," Soft Comput. - A Fusion Foundations, Methodologies, Applicat., vol. 6, no. 3-4, pp. 183-199, 2002.
4. E. Bernadó, X. Llorà, and J. M. Garrell, "XCS and GALE: A comparative study of two learning classifier systems with six other learning algorithms on classification tasks," in Proc. 4th Int. Workshop Learn. Classifier Syst., 2001, pp. 337-341.
5. L. Bull and J. Hurst, "ZCS redux," Evol. Comput., vol. 10, no. 2, pp. 185-205, 2003.
6. M. V. Butz, "XCS (+ tournament selection) classifier system implementation in C, Version 1.2," Illinois Genetic Algorithms Lab., Univ. Illinois, Urbana, IL, Tech. Rep. 2 003 023, 2003.
7. M. V. Butz and D. E. Goldberg, "Generalized state values in an anticipatory learning classifier system," in Anticipatory Behavior in Adaptive Learning Systems: Foundations, Theories, and Systems, M.V. Butz, O. Sigaud, and P. Gérard, Eds. Berlin, Germany: Springer-Verlag, 2003, pp. 282-301.
8. M. V. Butz, T. Kovacs, P. L. Lanzi, and S. W. Wilson, "Toward a theory of generalization and learning in XCS," IEEE Trans. Evol. Computat., vol. 8, no. 1, pp. 28-46, Feb. 2004.
9. M. V. Butz, K. Sastry, and D. E. Goldberg, "Tournament selection: Stable fitness pressure in XCS," in Lecture Notes in Computer Science, E. Cantü-Paz, J. A. Foster, K. Deb, D. Davis, R. Roy, U.-M. O'Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. Dowsland, N. Jonoska, and J. Miller, Eds. Chicago, IL, Jul. 12-16, 2003, vol. 2724, Proc. Genetic and Evol. Comput., pp. 1857-1869.
10. M. V. Butz and S. W. Wilson, "An algorithmic description of XCS," Soft Comput. - A Fusion of Foundations, Methodologies, Applicat., vol. 6, no. 3-4, pp. 144-153, 2002.
11. P. W. Dixon, D. W. Corne, and M. J. Oates, "A preliminary investigation of modified XCS as a generic data mining tool," in Advances in Learning Classifier Systems, P. L. Lanzi, W. Stolzmann, and S. W. Wilson, Eds. Berlin, Germany: Springer-Verlag, 2002, vol. 2321, LNAI, pp. 133-150.
12. S. A. Glantz and B. K. Slinker, Primer of Applied Regression & Analysis of Variance, 2nd ed. New York: McGraw-Hill, 2001.
13. D. E. Goldberg, "Computer-aided gas pipeline operation using genetic algorithms and rule learning," Ph.D. dissertation, Univ. Michigan, Ann Arbor, MI, 1983.
14. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: a survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, 1996.
15. T. Kovacs, Strength or Accuracy: Credit Assignment in Learning Classifier Systems. Berlin, Germany: Springer-Verlag, 2003.
16. P. L. Lanzi, "An analysis of generalization in the XCS classifier system," Evol. Comput., vol. 7, no. 2, pp. 125-149, 1999a.
17. P. L. Lanzi, "Extending the representation of classifier conditions, Part I: From binary to Messy coding," in Proc. Genetic Evol. Comput. Conf., W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith, Eds., 1999b, pp. 337-344.
18. P. L. Lanzi, "Mining interesting knowledge from data with the XCS classifier system," in Proc. Genetic Evol. Comput. Conf., L. Spector, E. D. Goodman, A. Wu, W. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, Eds., Jul. 7-11, 2001, pp. 958-965.
19. P. L. Lanzi, "Learning classifier systems from a reinforcement learning perspective," Soft Comput. - A Fusion of Foundations, Methodologies, Applicat., vol. 6, no. 3-4, pp. 162-170, 2002a.
20. P. L. Lanzi, (2002b) The XCS library. [Online]. Available: http://xcslib. sourceforge.net
21. P. L. Lanzi and M. Colombetti, "An extension to the XCS classifier system for stochastic environments," in Proc. Genetic Evol. Comput. Conf., W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith, Eds., Orlando, FL, Jul. 1999, pp. 353-360.
22. R. S. Sutton, "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming," in Proc. 7th Int. Conf. Mach. Learn., 1990, pp. 216-224.
23. R. S. Sutton and A. G. Barto, Reinforcement Learning - An Introduction. Cambridge, MA: MIT Press, 1998.
24. C. J. C. H. Watkins, "Learning from delayed rewards," Ph.D. dissertation, King's College, Cambridge, U.K., 1989.
25. S. W. Wilson, "ZCS: A zeroth level classifier system," Evol. Comput., vol. 2, no. 1, pp. 1-18, 1994. [Online]. Available: http://prediction-dynamics.com/.
26. S. W. Wilson, "Classifier fitness based on accuracy," Evol. Comput., vol. 3, no. 2, pp. 149-175, 1995. [Online]. Available: http://prediction-dynamics.com/.
27. S. W. Wilson, "Generalization in the XCS classifier system," in Proc. 3rd Ann. Conf. Genetic Program., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo, Eds., 1998, pp. 665-674. [Online]. Available: http://predictiondynamics.com/.
28. S. W. Wilson, "Get real! XCS with continuous-valued inputs," in Learning Classifier Systems. From Foundations to Applications, P. L. Lanzi, W. Stolzmann, and S. W. Wilson, Eds. Berlin, Germany: Springer-Verlag, 2000, vol. 1813, LNAI, pp. 209-219.
29. S. W. Wilson, "Function approximation with a classifier system," in Proc. Genetic Evol. Comput. Conf., L. Spector, E. D. Goodman, A. Wu, W. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, Eds., Jul. 7-11, 2001a, pp. 974-981.
30. S. W. Wilson, "Mining oblique data with XCS," in Lecture Notes in Computer Science, P. L. Lanzi, W. Stolzmann, and S. W. Wilson, Eds. Berlin, Germany: Springer, 2001b, vol. 1996, Proc. Int. Workshop Learn. Classifier Syst., pp. 158-176.
31. S. W. Wilson, "Classifiers that approximate functions," Natural Comput., vol. 1, pp. 211-234, 2002.