A formal framework and extensions for function approximation in learning classifier systems

Machine Learning

Volumn 70, Issue 1, 2008, Pages 45-88

  Drugowitsch, Jan ^a   Barry, Alwyn M ^b  

^a UNIVERSITY OF ROCHESTER   (United States)

^b University of Bath   (United Kingdom)

Author keywords

Function approximation; Kalman filter; Learning classifier systems

Indexed keywords

APPROXIMATION THEORY; CLASSIFICATION (OF INFORMATION); FUNCTION EVALUATION; KALMAN FILTERS; PROBLEM SOLVING;

FUNCTION APPROXIMATION; LEARNING CLASSIFIER SYSTEMS (LCS); PROBLEM DEFINITION;

LEARNING SYSTEMS;

EID: 36849052375     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-007-5024-8     Document Type: Article

References (68)

1. Anderson, B. D. O., & Moore, J. B. (1979). Information and system sciences series. Optimal filtering. Englewood Cliffs: Prentice-Hall.
2. Barry, A. M. (2002). The stability of long action chains in XCS. Journal of Soft Computing, 6(3-4), 183-199.
3. Barry, A. (2003). Limits in long path learning with XCS. In E. Cantú-Paz, J. A. Foster, K. Deb, D. Davis, R. Roy, U.-M. O'Reilly, & H.-G. Beyer et al. (Eds.), Lecture notes in computer science : Vol. 2724. Genetic and evolutionary computation (GECCO-2003) (pp. 1832-1843). Berlin: Springer.
4. Barry, A., Holmes, J., & Llora, X. (2004). Data mining using learning classifier systems. In L. Bull (Ed.), Applications of learning classifier systems. Berlin: Springer.
5. Bernadó, E., Llorà, X., & Garrell, J. M. (2001). XCS and GALE: a comparative study of two learning classifier systems with six other learning algorithms on classification tasks. In Proceedings of the 4th international workshop on learning classifier systems (IWLCS-2001) (pp. 337-341).
6. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Belmont: Athena Scientific.
7. Bishop, C. M. (2006). Pattern recognition and machine learning, information science and statistics. Berlin: Springer.
8. Booker, L. (2006). Personal communication, Mai 2006.
9. Bull, L. (2002). On accuracy-based fitness. Journal of Soft Computing, 6(3-4), 154-161.
10. Bull, L. (Ed.). (2004). Applications of learning classifier systems. Berlin: Springer.
11. Bull, L. (2005). Two simple learning classifier systems. In L. Bull & T. Kovacs (Eds.), Studies fuzziness and soft computing : Vol. 183. Foundations of learning classifier systems. Berlin: Springer.
12. Butz, M. (2004). Rule-based evolutionary online learning systems: learning bounds, classification, and prediction. PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL.
13. Butz, M. (2005). Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system. In H.-G. Beyer, et al. (Eds.), Proceedings of the genetic and evolutionary computation conference (GECCO-2005) (Vol. 2, pp. 1835-1842). New York: ACM.
14. Butz, M. V. (2006). Studies in fuzziness and soft computing : Vol. 191. Rule-based evolutionary online learning systems: a principled approach to LCS analysis and design. Berlin: Springer.
15. Butz, M. V., & Pelikan, M. (2001). Analyzing the evolutionary pressures in XCS. In L. Spector, et al. (Eds.), Proceedings of the genetic and evolutionary computation conference (GECCO-2001) (pp. 935-942). Los Altos: Kaufmann.
16. Butz, M., Kovacs, T., Lanzi, P.-L., & Wilson, S. W. (2004). Toward a theory of generalization and learning in XCS. In IEEE transactions on evolutionary computation.
17. Compiani, M., Montanari, D., Serra, R., & Simonini, P. (1990). Learning and bucket brigade dynamics in classifier systems. In S. Forrest, et al. (Eds.), Emergent computation, proceedings of the ninth annual international conference of the center for nonlinear studies on self-organizing, collective, and cooperative phenomena in natural and artificial computing networks. A special issue of Physica D (Vol. 42, pp. 202-212). Amsterdam: Elsevier Science.
18. Dixon, P. W., Corne, D. W., & Oates, M. J. (2002). A preliminary investigation of modified XCS as a generic data mining tool. In P. L. Lanzi, W. Stolzmann, & S. W. Wilson (Eds.), Lecture notes in artificial intelligence : Vol. 2321. Advances in learning classifier systems (pp. 133-150). Berlin: Springer.
19. Douglas, S. C. (1994). A family of normalized LMS algorithms. IEEE Signal Processing Letters, SPL-1(3), 49-51.
20. Drugowitsch, J., & Barry, A. M. (2006a). A formal framework and extensions for function approximation in learning classifier systems (Technical Report 2006-01). University of Bath, UK, January 2006.
21. Drugowitsch, J., & Barry, A. M. (2006b). A formal framework for reinforcement learning with function approximation in learning classifier systems (Technical Report 2006-02). University of Bath, UK, January 2006.
22. Drugowitsch, J., & Barry, A. M. (2006c). Towards convergence of learning classifier systems value iteration (Technical Report 2006-03). University of Bath, UK, April 2006.
23. Drugowitsch, J., & Barry, A. M. (2006d). Mixing independent classifiers (Technical Report 2006-13). University of Bath, UK, November 2006.
24. Drugowitsch, J., & Barry, A. M. (2007a). Mixing independent classifiers. In Proceedings of the genetic and evolutionary computation conference (GECCO-2007), July 2007. New York: ACM.
25. Drugowitsch, J., & Barry, A. M. (2007b). Generalized mixtures of experts, independent expert training, and learning classifier systems (Technical Report 2007-12). University of Bath, UK, April 2007.
26. Drugowitsch, J., & Barry, A. M. (2007c). A principled approach to LCS. In Proceedings of the tenth international workshop on learning classifier systems (IWLCS 2007), July 2007.
27. Eweda, E., & Macchi, O. (1987). Convergence of the RLS and LMS adaptive filter. IEEE Transactions on Circuits and Systems, CAS-34(7), 799-803.
28. Forrest, S., & Miller, J. H. (1990). Emergent behavior in classifier systems. In S. Forrest (Ed.), Proceedings of the ninth annual international conference of the center for nonlinear studies on self-organizing, collective, and cooperative phenomena in natural and artificial computing networks. A special issue of Physica D (Vol. 42, pp. 213-217). Amsterdam: Elsevier Science.
29. Graybill, F. A. (1961). An introduction to linear statistical models (Vol. 1). New York: McGraw-Hill.
30. Greenyer., A. (2000). The use of a learning classifier system JXCS. In: P. van der Putten & M. van Someren (Eds.), CoIL challenge 2000: the insurance company case (Technical report 2000-09). Leiden: Leiden Institute of Advanced Computer Science, June 2000.
31. Haykin, S. (1999). Neural networks. A comprehensive foundation (2nd ed). Upper Saddle River: Prentice Hall.
32. Haykin, S. (2002). Information and system sciences series. Adaptive filter theory (4th ed.). Upper Saddle River: Prentice Hall.
33. Hettich, S., & Bay, S. D. (1999). The UCI KDD Archive, 1999. http://kdd.ics.uci.edu.
34. Holland, J. H. (1985). Properties of the bucket brigade. In J. J. Grefenstette (Ed.), Proceedings of the 1st international conference on genetic algorithms and their applications (ICGA85) (pp. 1-7), July 1985. Pittsburgh: Lawrence Erlbaum Associates
35. Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering series D, 82, 35-45.
36. Kalman, R. E., & Bucy, R. S. (1961). New results in linear filtering and prediction theory. Transactions ASME, Part D (J. Basic Engineering), 83, 95-108.
37. Koller, D., & Parr, R. (2000). Policy iteration for factored MDPs. In UAI '00: proceedings of the 16th conference on uncertainty in artificial intelligence (pp. 326-334). San Francisco: Kaufmann Publishers.
38. Kovacs, T. (2002). A comparison and strength and accuracy-based fitness in learning classifier systems. PhD thesis, University of Birmingham.
39. Lanzi, P. L., Loiacono, D., Wilson, S. W., & Goldberg, D. E. (2005a). Extending XCSF beyond linear approximation. In H.-G. Beyer, U.-M. O'Reilly, D. V. Arnold, W. Banzhaf, C. Blum, & E. W. Bonabeau (Eds.) Proceedings of the genetic and evolutionary computation conference (GECCO-2005) (Vol. 2, pp. 1827-1834). New York: ACM.
40. Lanzi, P. L., Loiacono D., Wilson, S. W., Goldberg, D. E. (2005b). Generalization in the XCSF classifier systems: analysis, improvement, and extension (Technical Report 2005012). Illinois Genetic Algorithms Laboratory, March 2005.
41. Lanzi, P. L., Loiacono D., Wilson, S. W., Goldberg, D. E. (2005c). XCS with computed prediction in continuous multistep environments (Technical Report 2005018). Illinois Genetic Algorithms Laboratory, May 2005.
42. Lanzi, P. L., Loiacono, D., Wilson, S. W., & Goldberg, D. E. (2005d). XCS with computed predictions in multistep environments. In Beyer, H.-G., O'Reilly, U.-M., Arnold, D. V., Banzhaf, W., Blum, C., & Bonabeau, E. W. (Eds.), Proceedings of the genetic and evolutionary computation conference (GECCO-2005) (Vol. 2, pp. 1859-1866). New York: ACM.
43. Littman, M. (2005). Neural information processing systems workshop: RL comparisons. http://www.cs.rutgers.edu/~mlittman/topics/nips05-mdp/.
44. Loiacono, D., Drugowitsch, J., Barry, A. M., & Lanzi, P. L. (2006). Improving classifier error estimate in XCSF. In Proceedings of the 9th international workshop on learning classifier systems.
45. MacKay, D. J. C. (1991). Bayesian interpolation. Neural Computation, 4(3), 415-447.
46. Maybeck, P. S. (1979). Mathematics in science and engineering: Vol. 141. Stochastic models, estimation, and control, Vol. 1. New York: Academic Press.
47. Mook, D. J., & Junkins, J. L. (1988). Minimum model error estimation for poorly modeled dynamic systems. Journal of Guidance, Control and Dynamics, 11(3), 256-261.
48. Nedić, A., & Bertsekas, D. P. (2003). Least squares policy evaluation algorithms with linear function approximation. Discrete Event Dynamic Systems, 13(1-2), 79-110.
49. Piater, J. H., Cohen, P. R., Zhang, X., & Atighetchi, M. (1998). A randomized ANOVA procedure for comparing performance curves. In ICML '98: proceedings of the fifteenth international conference on machine learning (pp. 430-438). San Francisco: Kaufmann.
50. Riolo, R. L. (1987a). Bucket brigade performance, I: long sequences of classifiers. In J. J. Grefenstette (Ed.), Proceedings of the 2nd international conference on genetic algorithms (ICGA87) (pp. 184-195). Cambridge: Lawrence Erlbaum Associates.
51. Riolo, R. L. (1987b). Bucket brigade performance, II: default hierarchies. In J. J. Grefenstette (Ed.), Proceedings of the 2nd international conference on genetic algorithms (ICGA87) (pp. 196-201). Cambridge: Lawrence Erlbaum Associates.
52. Rummery, G., & Niranja, M. (1994). On-line q-learning using connectionist systems (Technical Report 166). Engineering Department, University of Cambridge.
53. Saxon, S., & Barry, A. (2000). XCS and the Monk's problems In P. L. Lanzi, W. Stolzmann, & S. W. Wilson (Eds.), Leture notes in artificial intelligence : Vol. 1813. Learning classifier systems. From foundations to applications (pp. 223-242). Berlin: Springer.
54. Smith, R. E. (1994). Memory exploitation in learning classifier systems. Evolutionary Computation, 2(3), 199-220.
55. Sutton, R. S. (1996). Generalization in reinforcement learning: successful examples using sparse coarse coding. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems (Vol. 8, pp. 1038-1044). Cambridge: MIT Press.
56. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge: MIT Press.
57. Tsitsiklis, J., & Van Roy, B. (1997). An analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control, 42(5), 674-690.
58. Wada, A., Takadama, K., Shimohara, K., & Katai, O. (2004). Is gradient descent method effective for XCS? Analysis of reinforcement process in XCSG? In W. Stolzmann, et al. (Eds.), Lecture notes in artificial intelligence. Proceedings of the seventh international workshop on learning lassifier systems. Seattle, WA, June 2004. Berlin: Springer.
59. Wada, A., Takadama, K., Shimohara, K., & Katai, O. (2005). Learning classifier system with convergence and generalization. In L. Bull & T. Kovacs (Eds.) Foundations of learning classifier systems : Vol. 183. Studies in fuzziness and soft computing. Berlin: Springer.
60. Watkins, C. J. C. H. (1989). Learning from delayed rewards. PhD thesis, University of Cambridge, Psychology Department.
61. Welch, G., & Bishop, G. (2004). An introduction to the Kalman filter (Technical Report TR 95-401). University of North Carolina at Chapel Hill, Department of Computer Science, April 2004.
62. Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. In IRE WESCON convention revord part IV (pp. 96-104).
63. Williams, G. (2005). Linear algebra with applications (5th ed.). Boston: Jones and Bartlett.
64. S. W. Wilson 1994 ZCS: a zeroth level classifier system Evolutionary Computation 2 1 1 18 Wilson, S. W. (1994). ZCS: a zeroth level classifier system. Evolutionary Computation, 2(1), 1-18. http://prediction-dynamics.com/.
65. Wilson S. W. (1995). Classifier fitness based on accuracy. Evolutionary Computation, 3(2), 149-175.
66. Wilson, S. W. (2001). Function approximation with a classifier system. In Spector, et al. (Eds.). Proceedings of the genetic and evolutionary computation conference (GECCO-2001) (pp. 974-981). Los Altos: Kaufmann.
67. Wilson, S. W. (2002). Classifiers that approximate functions. Neural Computing, 1(2-3), 211-234.
68. Wilson, S. W. (2004). Classifier systems for continuous payoff environments. In K. Deb, R. Poli, W. Banzhaf, H. G. Beyer, E. K. Burke, P. J. Darwen (Eds.), Lecture notes in computer science : Vol. 3103. Proceedings of the genetic and evolutionary computation conference (GECCO-2004) (pp. 824-835). Berlin: Springer.