Learning classifier systems: A survey

Soft Computing

Volumn 11, Issue 11, 2007, Pages 1065-1078

  Sigaud, Olivier ^a   Wilson, Stewart W ^b  

^a SORBONNE UNIVERSITÉ   (France)

^b Prediction Dynamics   (United States)

Author keywords

Generalization; Learning classifier systems; Reinforcement learning

Indexed keywords

COGNITIVE SYSTEMS; EVOLUTIONARY ALGORITHMS; KNOWLEDGE BASED SYSTEMS; REINFORCEMENT LEARNING;

AUTOMATIC CLASSIFICATION; LEARNING CLASSIFIER SYSTEM; SEQUENTIAL DECISION PROBLEM-SOLVING SYSTEM;

SOFT COMPUTING;

EID: 34249950269     PISSN: 14327643     EISSN: 14337479     Source Type: Journal    
DOI: 10.1007/s00500-007-0164-0     Document Type: Article

References (91)

1. Ahluwalia M, Bull L (1999) A genetic programming-based classifier system. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) (1999) Proceedings of the 1999 genetic and evolutionary computation conference workshop program. Morgan Kaufmann, San Francisco, pp 11-18
2. Bacardit J, Garrell JM (2003) Evolving multiple discretizations with adaptive intervals for a Pittsburgh rule-based learning classifier system. In: Cantú-Paz E, Foster JA, Deb K, Davis D, Roy R, O'Reilly U-M, Beyer H-G, Standish R, Kendall G, Wilson S, Harman M, Wegener J, Dasgupta D, Potter MA, Schultz AC, Dowsland K, Jonoska N, Miller J (eds). Genetic and evolutionary computation-GECCO-2003. Springer, Berlin, pp. 1818-1831
3. Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) (1999) Proceedings of the 1999 genetic and evolutionary computation conference workshop program. Morgan Kaufmann, San Francisco
4. Bellman RE (1957) Dynamic programming. Princeton University Press, Princeton
5. Bellman RE (1961) Adaptive control processes: A guided tour. Princeton University Press, Princeton
6. Bernadó E, Llorà X, Garrel JM (2001) XCS and GALE: A comparative study of two learning classifer systems with six other learning algorithms on classification tasks. In: Lanzi P-L, Stolzmann W, Wilson SW (eds) Proceedings of the fourth international workshop on Learning Classifer Systems
7. Bertsekas DP (1995) Dynamic programming and optimal control. Athena Scientific, Belmont
8. Beyer H-G, O'Reilly U-M, Arnold D, Banzhaf W, Blum C, Bonabeau E, Cantú Paz E, Dasgupta D, Deb K, Foster J, de Jong E, Lipson H, Llora X, Mancoridis S, Pelikan M, Raidl G, Soule T, Tyrrell A, Watson J-P, Zitzler E (eds) (2005) Proceedings of the genetic and evolutionary computation conference, GECCO-2005. ACM, Washington
9. Booker L, Goldberg DE, Holland JH (1989) Classifier systems and genetic algorithms. Artif Intell 40(1-3):235-282
10. Booker LB (2000) Do we really need to estimate rule utilities in classifier systems?. In: Lanzi P-L, Stolzmann W, Wilson SW (eds). Learning classifier systems. From foundations to applications, vol. 1813 of Lecture Notes in Artificial Intelligence. Springer, Berlin, pp. 125-142
11. Boutilier C, Dearden R, Goldszmidt M (1995) Exploiting structure in policy construction. In: Proceedings of the fourteenth international joint conference on artificial intelligence (IJCAI-95), Montreal, pp 1104-1111
12. Boutilier C, Dearden R, Goldszmidt M (2000) Stochastic dynamic programming with factored representations. Artif Intell 121(1): 49-107
13. Bull L (1998) On ZCS in multi-agent environments. Lect Notes Comput Sci 1498:471-479
14. Bull L (1999) On using ZCS in a simulated continuous double-auction market. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) Proceedings of the 1999 genetic and evolutionary computation conference workshop program. Morgan Kaufmann, San Francisco, pp. 83-90
15. Bull L (2004a) A simple payoff-based learning classifier system. In: Yao X, Burke E, Lozano JA, Smith J, Merelo-Guervós JJ, Bullinaria JA, Rowe J, Tino P, Kabán A, Schwefel H-P (eds). Proceedings of parallel problem solving from nature (PPSN VIII). Springer, Heidelberg
16. Bull L (ed) (2004b) Applications of learning classifier systems. Springer, Heidelberg
17. Bull L (2005) Two simple learning classifier systems. In: Bull L, Kovacs T (eds) Foundations of learning classifier systems. Springer, Heidelberg
18. Bull L, Hurst J (2002) ZCS redux. Evol Comput 10(2):185-205
19. Butz M, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in xcs. IEEE Trans Evol Comput 8(1):28-46
20. Butz M, Pelikan M, Llorà X, Goldberg DE (2006) Automated global structure extraction for eeffective local building block processing in XCS. J Evol Comput (to appear)
21. Butz MV (2002) An algorithmic description of ACS2. In: Lanzi P-L, Stolzmann W, Wilson SW (eds) Advances in learning classifier systems, vol 2321 of Lecture Notes in Artificial Intelligence. Springer, Berlin, pp 211-229
22. Butz MV, Goldberg DE (2003) Generalized state values in an anticipatory Learning Classifier System. In Butz MV, Sigaud O, Gérard P (eds) LNAI 2684: Anticipatory behavior in adaptive learning systems. Springer, Heidelberg, pp 281-301
23. Butz MV, Goldberg DE, Lanzi PL (2003a) Gradient descent methods in learning classifier systems: Improving XCS performance in multistep problem. Technical report 2003028, IlliGAL
24. Butz MV, Goldberg DE, Stolzmann W (2000) Introducing a genetic generalization pressure to the Anticipatory Classifier Systems part I: Theoretical approach. In: Whitley LD, Goldberg DE, Cantú-Paz E, Spector L, Parmee IC, Beyer H-G (eds) (2000) Proceedings of the genetic and evolutionary computation conference (GECCO00). Morgan Kaufmann, San Francisco, pp 34-41. Also Technical Report 2000005 of the Illinois Genetic Algorithms Laboratory
25. Butz MV, Sastry K, Goldberg DE (2003b) Tournament selection: Stable fitness pressure in XCS. In: Cantú-Paz E, Foster JA, Deb K, Davis D, Roy R, O'Reilly U-M, Beyer H-G, Standish R, Kendall G, Wilson S, Harman M, Wegener J, Dasgupta D, Potter MA, Schultz AC, Dowsland K, Jonoska N, Miller J (eds) Genetic and evolutionary computation-GECCO-2003, vol 2724 of LNCS. Springer, Berlin, pp 1857-1869
26. Butz MV, Wilson SW (2002) An algorithmic description of XCS. J Soft Comput 6(3-4):144-153
27. Cantu-Paz E, Foster JA, Deb K, O'Reilly U-M, Beyer H-G, Standish R, Kendall G, Wilson S, Harman M, Weneger J, Dasgupta D, Potter MA, Schultz AC, Dowsland KA, Jonoska N, Miller J (eds) (2003) Proceedings of the 2003 genetic and evolutionary computation conference workshop program, Chicago. Springer, Berlin
28. Cao YJ, Ireson N, Bull L, Miles R (1999) Design of a traffic junction controller using a classifier system and fuzzy logic. In Reusch B (ed) Proceedings of the sixth international conference on computational intelligence, theory and applications (6th fuzzy days), vol 1625 of LNCS. Springer, Berlin, pp 342-353
29. Cao YJ, Ireson N, Bull L, Miles R (2001) An evolutionary intelligent agents approach to traffic signal control. Int J Knowl based Intell Eng Syst 5(4):279-289
30. Cliff D, Ross S (1994) Adding memory to ZCS. Adapt Behav 3(2):101-150
31. Deb K, Poli R, Banzhaf W, Beyer H-G, Burke E, Darwen P, Dasgupta D, Floreano D, Foster JA, Harman M, Holland O, Lanzi P-L, Spector L, Tettamanzi A, Thierens D, Tyrrell A (eds) (2004) Proceedings of the 2004 genetic and evolutionary computation conference workshop program, Seattle. Springer, Berlin
32. Degris T, Sigaud O, Wuillemin P-H (2006a) Chi-square tests driven method for learning the structure of factored mdps. In: Proceedings of the 22nd uncertainty in artificial intelligence conference (UAI'2006). MIT, Massachussetts, pp 122-129
33. Degris T, Sigaud O, Wuillemin P-H (2006b) Learning the structure of factored markov decision processes in reinforcement learning problems. In: Proceedings of the 23rd international conference on machine learning (ICML'2006). CMU, Pensylvania, pp 257-264
34. Dorigo M, Bersini H (1994) A comparison of Q-learning and classifier systems. In: Cliff D, Husbands P, Meyer J-A, Wilson SW (eds) From animals to animats 3 MIT, Cambridge, pp 248-255
35. Drugowitsch J, Barry A (2006) Towards convergence of learning classifier systems value iteration. Technical report CSBU-2006-03, Department of Computer Science, University of Bath
36. Gérard P, Meyer J-A, Sigaud O (2005) Combining latent learning with dynamic programming in MACS. Euro J Oper Res 160: 614-637
37. Gérard P, Sigaud O (2003) Designing efficient exploration with MACS: Modules and function approximation. In: Proceedings of the genetic and evolutionary computation conference 2003 (GECCO'03), Chicago. Springer, Berlin, pp 1882-1893
38. Gérard P, Stolzmann W, Sigaud O (2002) YACS: A new learning classifier system with anticipation. J Soft Comput Spl Issue Learn Classif Syst 6(3-4):216-228
39. Golberg DE, Holland JH (1988) Guest editorial: Genetic algorithms and machine learning. Mach Learn 3:95-99
40. Goldberg DE (1989) Genetic algorithms in search, optimization, machine learning. Addison Wesley, Reading
41. Herbart JF (1825) Psychologie als Wissenschaft neu gegründet auf Erfahrung, Metaphysik und Mathematik. Zweiter, analytischer Teil. August Wilhem Unzer, Koenigsberg, Germany
42. Hoffmann J (1993) Vorhersage und Erkenntnis [Anticipation and Cognition]. Hogrefe, Göttingen
43. Holland JH (1975) Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, artificial intelligence. University of Michigan Press, Ann Arbor
44. Holland JH (1986) Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In: Machine learning, an artificial intelligence approach (vol II). Morgan Kaufmann, San Francisco
45. Holland JH, Holyoak KJ, Nisbett RE, Thagard PR (1986) Induction. MIT, Cambridge
46. Holland JH, Reitman JS (1978) Cognitive systems based on adaptive algorithms. Pattern Direct Infer Syst 7(2):125-149
47. Holmes JH (2002) A new representation for assessing classifier performance in mining large databases. In: Stolzmann W, Lanzi P-L, Wilson SW (eds) IWLCS-02. Proceedings of the international workshop on learning classifier systems, LNAI, Granada. Springer, Berlin
48. Kovacs T (2002) Learning classifier systems resources. J Soft Comput 6(3-4):240-243
49. Kovacs T (2004) Strength or accuracy: Credit assignment in learning classifier systems. Springer, Berlin
50. Landau S, Sigaud O (2006) A comparison between ATNoSFERES and LCSs on non-Markov problems. Inform Sci (to appear)
51. Landau S, Sigaud O, Schoenauer M (2005) ATNoSFERES revisited. In: Beyer H-G, O'Reilly U-M, Arnold D, Banzhaf W, Blum C, Bonabeau E, Cantú Paz E, Dasgupta D, Deb K, Foster J, de Jong E, Lipson H, Llora X, Mancoridis S, Pelikan M, Raidl G, Soule T, Tyrrell A, Watson J-P, Zitzler E (eds) Proceedings of the genetic and evolutionary computation conference, GECCO-2005, Washington. ACM, pp 1867-1874
52. Langdon WB, Cantu-Paz E, Mathias K, Roy R, Davis D, Poli R, Balakrishnan K, Honavar V, Rudolph G, Wegener J, Bull L, Potter MA, Schultz AC, Miller JF, Burke E, Jonoska N (eds) (2002) Proceedings of the genetic and evolutionary computation conference, GECCO 2002, New York. Morgan Kaufmann, San Francisco
53. Lanzi P-L (1998) Adding memory to XCS. In: Proceedings of the IEEE conference on evolutionary computation (ICEC98). IEEE Press
54. Lanzi P-L (2002) Learning classifier systems from a reinforcement learning perspective. J Soft Comput 6(3-4):162-170
55. Lanzi P-L, Loiacono D, Wilson S, Goldberg DE (2006) Classifier prediction based on tile coding. In: Proceedings of the 2006 genetic and evolutionary computation conference workshop program (GECCO 2006). Seattle, Washington, pp 1497-1504
56. Lanzi P-L, Perrucci A (1999) Extending the representation of classifier conditions part ii: From messy coding to s-expressions. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) Proceedings of the genetic and evolutionary computation conference (GECCO 99), Orlando. Morgan Kaufmann, San Francisco, pp 345-352
57. Lanzi P-L, Riolo RL (2000) A roadmap to the last decade of learning classifier systems research (from 1989 to 1999). In: Lanzi P-L, Stolzmann W, Wilson SW (eds) Learning classifier systems: From foundations to applications. Springer, Heidelberg, pp 33-62
58. Lanzi P-L, Stolzmann W, Wilson SW, (eds) (2000) Learning classifier systems. From Foundations to Applications, vol 1813 of Lecture Notes in Artificial Intelligence. Springer, Berlin
59. Lanzi P-L, Stolzmann W, Wilson SW (eds). (2001) Advances in learning classifier systems, vol 1996 of Lecture Notes in Artificial Intelligence. Springer, Berlin
60. Lanzi P-L, Stolzmann W, Wilson SW (eds) (2002a) Advances in learning classifier systems, vol 2321 of Lecture Notes in Artificial Intelligence. Springer, Berlin
61. Lanzi P-L, Stolzmann W, Wilson SW (eds) (2002b) Proceedings of the international workshop on learning classifier systems (IWLCS2002), Granada
62. Llorà X, Garrell JM (2002) Co-evolving different knowledge representations with fine-grained parallel learning classifier systems. In: Langdon WB, Cantu-Paz E, Mathias K, Roy R, Davis D, Poli R, Balakrishnan K, Honavar V, Rudolph G, Wegener J, Bull L, Potter MA, Schultz AC, Miller JF, Burke E, Jonoska N (eds) Proceeding of the genetic and evolutionary computation conference (GECCO2002). Morgan Kaufmann, San Francisco
63. MiramontesHercog L, Fogarty TC (2002) Co-evolutionary classifier systems for multi-agent simulation. In: Fogel DB, El-Sharkawi MA, Yao X, Greenwood G, Iba H, Marrow P, Shackleton M (eds) Proceedings of the 2002 congress on evolutionary computation CEC2002. IEEE Press, pp 1798-1803
64. Poli R, Langdon WB (1998) Schema theory for genetic programming with one-point crossover and point mutation. Evol Comput J 6(3):231-252
65. Puterman ML, Shin MC (1978) Modified policy iteration algorithms for discounted markov decision problems. Manage Sci 24:1127-1137
66. Riolo RL (1991) Lookahead planning and latent learning in a Classifier System. In: Meyer J-A, Wilson SW (eds) From animals to animats: proceedings of the first international conference on simulation of adaptative behavior. MIT, Cambridge, pp 316-326
67. Robert G (2005) MHiCS, une architecture de Sélection de l'Action Motivationnelle et Hiérarchique à Systèmes de Classeurs pour Personnages Non Joueurs Adaptatifs. PhD thesis, Laboratoire d'Informatique de Paris 6
68. Sanchez S (2004) Mécanismes évolutionnistes pour la simulation comportementale d'acteurs virtuels. PhD thesis, Université de Siences Sociales Toulouse 1, Toulouse
69. Sanza C (2001) Evolution d'entités vituelles cooperatives par systèmes de classifieurs. PhD thesis, Université Paul Sabatier, Toulouse
70. Seward JP (1949) An experimental analysis of latent learning. J Exp Psychol 39:177-186
71. Sigaud O, Gourdin T, Wuillemin P-H (2004) Improving MACS thanks to a comparison with 2TBNs. In: Proceedings of the genetic and evolutionary computation conference, GECCO'04. Springer, Berlin, pp 810-823
72. Smith SF (1980) A learning system based on genetic algorithms. PhD thesis, Department of Computer Science, University of Pittsburg, Pittsburg
73. Spector LD, GE, Wu A, Langdon WB, Voigt HM, Gen M (eds) (2001) Proceedings of the genetic and evolutionary computation conference (GECCO01). Morgan Kaufmann, San Francisco
74. Stolzmann W (1998) Anticipatory classifier systems. In Koza J, Banzhaf W, Chellapilla K, Deb K, Dorigo M, Fogel DB, Garzon MH, Goldberg DE, Iba H, Riolo R (eds) Genetic programming. Morgan Kaufmann, San Francisco, pp 658-664
75. Stolzmann W, Lanzi P-L, Wilson SW (eds) (2002) Proceedings of the international workshop on learning classifier systems (IWLCS2003), Chicago
76. Stone C, Bull L (2003) Towards learning classifier systems for continuous-valued online environments. In: Cantú-Paz E, Foster JA, Deb K, Davis D, Roy R, O'Reilly U-M, Beyer H-G, Standish R, Kendall G, Wilson S, Harman M, Wegener J, Dasgupta D, Potter MA, Schultz AC, Dowsland K, Jonoska N, Miller J (eds) Genetic and evolutionary computation-GECCO-2003. Springer, Berlin, pp 1924-1925
77. Sutton RS (1990) Planning by incremental dynamic programming. In: Proceedings of the eighth international conference on machine learning, San Mateo. Morgan Kaufmann, San Francisco pp 353-357
78. Sutton RS (1996) Generalization in reinforcement learning: Successul examples using sparse coarse coding. In: Touretzky DS, Mozer MC, Hasselmo ME (eds) Advances in neural information processing systems: proceedings of the 1995 conference. MIT, Cambridge, pp. 1038-1044
79. Sutton RS, Barto AG (1998) Reinforcement learning: An introduction. MIT, Cambridge
80. Tolman EC (1932) Purposive behavior in animals and men. Appletown, New York
81. Tomlinson A, Bull L (1998) A corporate classifier system. In: Eiben AE, Bäck T, Shoenauer M, Schwefel H-P (eds) Proceedings of the fifth international conference on parallel problem solving from nature - PPSN V, no. 1498 in LNCS, Springer, Berlin pp 550-559
82. Tomlinson A, Bull L (2000) CXCS. In: Lanzi P-L, Stolzmann W, Wilson SW (eds) Learning classifier systems: From foundations to applications. Springer, Heidelberg, pp 194-208
83. Watkins CJCH (1989) Learning with delayed rewards. PhD thesis, Psychology Department, University of Cambridge, England
84. Whitley LD, Goldberg DE, Cantú-Paz E, Spector L, Parmee IC, Beyer H-G (eds) (2000) Proceedings of the genetic and evolutionary computation conference (GECCO00). Morgan Kaufmann, San Francisco
85. Wilson SW (1985) Knowledge growth in an artificial animat. In: Grefenstette JJ (Ed.) Proceedings of the 1st international conference on genetic algorithms and their applications (ICGA85), pp 16-23. LE. Associates
86. Wilson SW (1994) ZCS, a zeroth level classifier system. Evol Comput 2(1):1-18
87. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149-175
88. Wilson SW (2000) Get real! XCS with continuous-valued inputs. In: Lanzi P-L, Stolzmann W, Wilson SW (eds) LNAI 1813: From foundations to applications. Springer, Berlin, pp 209-220
89. Wilson SW (2001) Function approximation with a classifier system. In: Spector L, D GE, Wu A, Langdon WB, Voigt HM, Gen M (eds) Proceedings of the genetic and evolutionary computation conference (GECCO01). Morgan Kaufmann, San Francisco, pp 974-981
90. Wilson SW (2004) Classifier systems for continuous payoff environments. In: LLCS 3103: learning classifier systems. Springer, Heidelberg, pp 824-835.
91. Wilson SW, Goldberg DE (1989) A critical review of classifier systems. In: Proceedings of the third international conference on genetic algorithms, Los Altos, California. Morgan Kaufmann, San Francisco, pp 244-255