Maximizing fun by creating data with easily reducible subjective complexity

Intrinsically Motivated Learning in Natural and Artificial Systems

Volumn 9783642323751, Issue , 2013, Pages 95-128

  Schmidhuber, Jürgen ^a,b  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

^b UNIVERSITY OF LUGANO   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ENCODING (SYMBOLS);

ART AND MUSICS; COMPUTATIONAL EFFORT; CREATIVES; FORMAL THEORY; INTRINSIC MOTIVATION; INTRINSIC REWARDS; LEARN+; LEARNING PROGRESS; SENSORY DATA; SIMPLE++;

SIGNAL ENCODING; EDUCATION;

EID: 84886639645     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-3-642-32375-1_5     Document Type: Chapter

References (128)

1. Bakker, B., Schmidhuber, J.: Hierarchical reinforcement learning based on subgoal discovery and subpolicy specialization. In: Groen, F., Amato, N., Bonarini, A., Yoshida, E., and Kröse, B. (eds.) Proceedings of the 8th Conference on Intelligent Autonomous Systems IAS-8, pp. 438-445. IOS Press, Amsterdam (2004)
2. Balcan, M., Beygelzimer, A., Langford, J.: Agnostic active learning. J. Comput. Syst. Sci. 75(1), 78-89 (2009)
3. Barto, A.G., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: Proceedings of International Conference on Developmental Learning (ICDL). MIT, Cambridge (2004)
4. Bense, M.: Einführung in die informationstheoretische Ästhetik. Grundlegung und Anwendung in der Texttheorie (Introduction to Information-Theoretical Aesthetics. Foundation and Application to Text Theory). Rowohlt Taschenbuch Verlag (1969)
5. Berlyne, D.E.: Novelty and curiosity as determinants of exploratory behavior. Br. J. Psychol. 41, 68-80 (1950)
6. Berlyne, D.E.: Conflict, Arousal, and Curiosity. McGraw Hill, New York (1960)
7. Birkhoff, G.D.: Aesthetic Measure. Harvard University Press, Cambridge (1933)
8. Blank, D., Meeden, L.: Introduction to the special issue on developmental robotics. Connect. Sci. 18(2) (2006)
9. Brafman, R.I., Tennenholtz, M.: R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning. J. Mach. Learn. Res. 3, 213-231 (2002)
10. Cohn, D.A.: Neural network exploration using optimal experiment design. In: Cowan, J., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems 6, pp. 679-686. Morgan Kaufmann, San Francisco (1994)
11. Collingwood, R.G.: The Principles of Art. Oxford University Press, London (1938)
12. Cuccu, G., Luciw, M., Schmidhuber, J., Gomez, F.: Intrinsically motivated evolutionary search for vision-based reinforcement learning. In: Proceedings of the 2011 IEEE Conference on Development and Learning and Epigenetic Robotics IEEE-ICDL-EPIROB. IEEE (2011)
13. Dayan, P., Hinton, G.: Feudal reinforcement learning. In: Lippman, D.S., Moody, J.E., Touretzky, D.S. (eds.) Advances in Neural Information Processing Systems 5, pp. 271-278. Morgan Kaufmann, San Francisco (1993)
14. Fedorov, V.V.: Theory of Optimal Experiments. Academic, New York (1972)
15. Frank, H.G.: Kybernetische Analysen subjektiver Sachverhalte. Verlag Schnelle, Quickborn (1964)
16. Frank, H.G., Franke, H.W.: Ästhetische Information. Estetika informacio. Eine Einführung in die kybernetische Åsthetik. Kopäd Verlag (2002)
17. Franke, H.W.: Kybernetische ä sthetik. Phänomen Kunst, 3rd edn. Ernst Reinhardt Verlag, Munich (1979)
18. Friston, K.J., Daunizeau, J., Kilner, J., Kiebel, S.J.: Action and behavior: A free-energy formulation. Biol. Cybern. 102(3), 227-260 (2010)
19. Gold, K., Scassellati, B.: Learning acceptable windows of contingency. Connect. Sci. 18(2) (2006)
20. Gomez, F.J., Schmidhuber, J., Miikkulainen, R.: Efficient non-linear control through neuroevolu-tion. J. Mach. Learn. Res. 9, 937-965 (2008)
21. Goodman, N.: Languages of Art: An Approach to a Theory of Symbols. The Bobbs-Merrill Company, Indianapolis (1968)
22. Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for improved unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5) (2009)
23. Harlow, H.F., Harlow, M.K., Meyer, D.R.: Novelty and curiosity as determinants of exploratory behavior. J. Exp. Psychol. 41, 68-80 (1950)
24. Hart, S.: The Development of Hierarchical Knowledge in Robot Systems. Ph.D. Thesis, Department of Computer Science, University of Massachusetts Amherst (2009)
25. Hart, S., Sen, S., Grupen, R.: Intrinsically motivated hierarchical manipulation. In: Proceedings of the 2008 IEEE Conference on Robots and Automation (ICRA), Pasadena (2008)
26. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735-1780 (1997)
27. Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor (1975)
28. Holland, J.H.: Properties of the bucket brigade. In: Proceedings of an International Conference on Genetic Algorithms. Lawrence Erlbaum, Hillsdale (1985)
29. Huffman, D.A.: A method for construction of minimum-redundancy codes. Proc. IRE 40, 1098-1101 (1952)
30. Hutter, M.: Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability. Springer, Berlin (2005). (On J. Schmidhuber's SNF grant 20-61847)
31. Hwang, J., Choi, J., Oh, S., II, R.J.M.: Query-based learning applied to partially trained multilayer perceptrons. IEEE Trans. Neural Netw. 2(1), 131-136 (1991)
32. Itti, L., Baldi, P.F.: Bayesian surprise attracts human attention. In: Advances in Neural Information Processing Systems 19, pp. 547-554. MIT, Cambridge (2005)
33. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. J. AI Res. 4, 237-285 (1996)
34. Kant, I.: Critik der reinen Vernunft (1781)
35. Klyubin, A.S., Polani, D., Nehaniv, C.L.: Empowerment: A universal agent-centric measure of control. In: Congress on Evolutionary Ccomputation (CEC-05), IEEE (2005)
36. Kolmogorov, A.N.: Three approaches to the quantitative definition of information. Prob. Inform. Transm. 1, 1-11 (1965)
37. Kuhn, T.: The Structure of Scientific Revolutions. University of Chicago Press, Chicago (1962)
38. Kuipers, B., Beeson, P., Modayil, J., Provost, J.: Bootstrap learning of foundational representations. Connect. Sci. 18(2) (2006)
39. Kullback, S.: Statistics and Information Theory. Wiley, New York (1959)
40. Lenat, D.B.: Theory formation by heuristic search. Mach. Learn. 21 (1983)
41. Levin, L.A.: Universal sequential search problems. Prob. Inform. Transm. 9(3), 265-266 (1973)
42. Li, L., Littman, M.L., Walsh, T.J.: Knows what it knows: A framework for self-aware learning. In: Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08) (2008)
43. Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications, 2nd edn. Springer, Berlin (1997)
44. Littman, M.L., Ackley, D.H.: Adaptation in constant utility non-stationary environments. In: Belew, R.K., Booker, L. (eds.) Proceedings of the Fourth International Conference on Genetic Algorithms, pp. 136-142. Morgan Kaufmann, San Mateo (1991)
45. Luciw, M., Graziano, V., Ring, M., Schmidhuber, J.: Artificial curiosity with planning for autonomous perceptual and cognitive development. In: Proceedings of the First Joint Conference on Development Learning and on Epigenetic Robotics ICDL-EPIROB, Frankfurt (2011)
46. MacKay, D.J.C.: Information-based objective functions for active data selection. Neural Comput. 4(2), 550-604 (1992)
47. Moles, A.: Information Theory and Esthetic Perception. University of Illinois Press, Illinois (1968)
48. Nake, F.: Ästhetik als Informationsverarbeitung. Springer, Berlin (1974)
49. Ngo, H., Ring, M., Schmidhuber, J.: Compression progress-based curiosity drive for developmental learning. In: Proceedings of the 2011 IEEE Conference on Development and Learning and Epigenetic Robotics IEEE-ICDL-EPIROB. IEEE (2011)
50. Olsson, L., Nehaniv, C.L., Polani, D.: From unknown sensors and actuators to actions grounded in sensorimotor perceptions. Connect. Sci. 18(2) (2006)
51. Oudeyer, P.-Y., Kaplan, F.: What is intrinsic motivation? a typology of computational approaches. Front. Neurorobot. 1 (2006)
52. Oudeyer, P.-Y., Kaplan, F., Hafner, V.F.: Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11(2), 265-286 (2007)
53. Piaget, J.: The Child's Construction of Reality. Routledge and Kegan Paul, London (1955)
54. Plutowski, M., Cottrell, G., White, H.: Learning Mackey-Glass from 25 examples, plus or minus 2. In: Cowan, J., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, Vol. 6, pp. 1135-1142. Morgan Kaufmann, San Francisco (1994)
55. Poland, J., Hutter, M.: Strong asymptotic assertions for discrete MDL in regression and classification. In: Annual Machine Learning Conference of Belgium and the Netherlands (Benelearn-2005), Enschede (2005)
56. Popper, K.R.: All Life Is Problem Solving. Routledge, London (1999)
57. Provost, J., Kuipers, B.J., Miikkulainen, R.: Developing navigation behavior through self-organizing distinctive state abstraction. Connect. Sci. 18(2) (2006)
58. Raskin, V.: Semantic Mechanisms of Humor. Dordrecht/Boston/Lancaster (1985)
59. Rechenberg, I.: Evolutionsstrategie - Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Dissertation. Published 1973 by Fromman-Holzboog (1971)
60. Ring, M.B.: Incremental development of complex behaviors through automatic construction of sensory-motor hierarchies. In: Birnbaum, L., Collins, G. (eds.) Machine Learning: Proceedings of the Eighth International Workshop, pp. 343-347. Morgan Kaufmann, San Francisco (1991)
61. Ring, M.B.: Continual Learning in Reinforcement Environments. Ph.D. Thesis, University of Texas at Austin, Austin (1994)
62. Rissanen, J.: Modeling by shortest data description. Automatica 14, 465-471 (1978)
63. Rückstieß, T., Sehnke, F., Schaul, T., Wierstra, D., Yi, S., Schmidhuber, J.: Exploring parameter space in reinforcement learning. Paladyn J. Behav. Robot. 1(1), 14-24 (2010)
64. Schaul, T., Pape, L., Glasmachers, T., Graziano, V., Schmidhuber, J.: Coherence Progress: A Measure of Interestingness Based on Fixed Compressors. In: Fourth Conference on Artificial General Intelligence (AGI) (2011a)
65. Schaul, T., Sun, Y., Wierstra, D., Gomez, F., Schmidhuber, J.: Curiosity-Driven Optimization. In: IEEE Congress on Evolutionary Computation (CEC), New Orleans (2011b)
66. Schembri, M., Mirolli, M., Baldassarre, G.: Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot. In: Demiris, Y., Scassellati, B., Mareschal, D. (eds.) The 6th IEEE International Conference on Development and Learning (ICDL2007), pp. 282-287. Imperial College, London. IEEE Catalog Number: 07EX1740C, Library of Congress: 2007922394 (2007)
67. Schlesinger, M.: Decomposing infants' object representations: A dual-route processing account. Connect. Sci. 18(2) (2006)
68. Schmidhuber, J.: A local learning algorithm for dynamic feedforward and recurrent networks. Connect. Sci. 1(4), 403-412 (1989a)
69. Schmidhuber, J.: The neural bucket brigade. In: Pfeifer, R., Schreter, Z., Fogelman, Z., Steels, L. (eds.) Connectionism in Perspective, pp. 439-446. Elsevier, North-Holland, Amsterdam (1989b)
70. Schmidhuber, J.: Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem. Dissertation, Institut für Informatik, Technische Universität München (1990a)
71. Schmidhuber, J.: An on-line algorithm for dynamic reinforcement learning and planning in reactive environments. In: Proceedings of the IEEE/INNS International Joint Conference on Neural Networks, San Diego, Vol. 2, pp. 253-258 (1990b)
72. Schmidhuber, J.: Adaptive curiosity and adaptive confidence. Technical Report FKI-149-91, Institut für Informatik, Technische Universität München. See also Schmidhuber (1991b) (1991a)
73. Schmidhuber, J.: Curious model-building control systems. In: Proceedings of the International Joint Conference on Neural Networks, Singapore, Vol. 2, pp. 1458-1463. IEEE (1991b)
74. Schmidhuber, J.: A possibility for implementing curiosity and boredom in model-building neural controllers. In: Meyer, J.A., Wilson, S.W. (eds.) Proceedings of the of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pp. 222-227. MIT/Bradford Books (1991c)
75. Schmidhuber, J.: Reinforcement learning in Markovian and non-Markovian environments. In: Lippman, D.S., Moody, J.E., Touretzky, D.S. (eds.) Advances in Neural Information Processing Systems 3 (NIPS 3), pp. 500-506. Morgan Kaufmann, San Francisco (1991d)
76. Schmidhuber, J.: Learning complex, extended sequences using the principle of history compression. Neural Comput. 4(2), 234-242 (1992)
77. Schmidhuber, J.: Femmes fractales. Report IDSIA-99-97, IDSIA, Switzerland, (1997)
78. Schmidhuber, J.: Low-complexity art. Leonardo, J. Int. Soc. Arts Sci. Technol. 30(2), 97-103 (1997b)
79. Schmidhuber, J.: What's interesting? Technical Report IDSIA-35-97, IDSIA (1997c). ftp://ftp.idsia.ch/pub/juergen/interest.ps.gz
80. extended abstract in Proc. Snowbird'98, Utah, 1998; see also Schmidhuber (2002a)
81. Schmidhuber, J.: Exploring the predictable. In: Ghosh, A., Tsuitsui, S. (eds.) Advances in Evolutionary Computing, pp. 579-612. Springer, Berlin (2002a)
82. Schmidhuber, J.: Hierarchies of generalized Kolmogorov complexities and nonenumerable universal measures computable in the limit. Int. J. Found. Comput. Sci. 13(4), 587-612 (2002b)
83. Schmidhuber, J.: The Speed Prior: a new simplicity measure yielding near-optimal computable predictions. In: Kivinen, J., Sloan, R.H. (eds.) Proceedings of the 15th Annual Conference on Computational Learning Theory (COLT 2002), Lecture Notes in Artificial Intelligence, pp. 216-228. Springer, Sydney (2002c)
84. Schmidhuber, J.: Optimal ordered problem solver. Mach. Learn. 54, 211-254 (2004)
85. Schmidhuber, J.: Completely self-referential optimal reinforcement learners. In: Duch, W., Kacprzyk, J., Oja, E., Zadrozny, S. (eds.) Artificial Neural Networks: Biological Inspirations - ICANN 2005, LNCS 3697, pp. 223-233. Springer (2005a). Plenary talk
86. Schmidhuber, J.: Gödel machines: Towards a technical justification of consciousness. In: Kudenko, D., Kazakov, D., Alonso, E. (eds.) Adaptive Agents and Multi-Agent Systems III (LNCS 3394), pp. 1-23. Springer, Berlin (2005b)
87. Schmidhuber, J.: Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connect. Sci. 18(2), 173-187 (2006a)
88. Schmidhuber, J.: Gödel machines: Fully self-referential optimal universal self-improvers. In: Goertzel, B., Pennachin, C. (eds.) Artificial General Intelligence, pp. 199-226. Springer, Berlin (2006b). Variant available as arXiv:cs.LO/0309048
89. Schmidhuber, J.: The new AI: General & sound & relevant for physics. In: Goertzel, B., Pennachin, C. (eds.) Artificial General Intelligence, pp. 175-198. Springer, Berlin (2006c). Also available as TR IDSIA-04-03, arXiv:cs.AI/0302012
90. Schmidhuber, J.: Randomness in physics. Nature 439(3), 392 (2006d). Correspondence
91. Schmidhuber, J.: 2006: Celebrating 75 years of AI - history and outlook: The next 25 years. In: Lungarella, M., Iida, F., Bongard, J., Pfeifer, R. (eds.) 50 Years of Artificial Intelligence, LNAI, Vol. 4850, pp. 29-41. Springer, Berlin (2007a). Preprint available as arXiv:0708.4311
92. Schmidhuber, J.: New millennium AI and the convergence of history. In: Duch, W., Mandziuk, J. (eds.) Challenges to Computational Intelligence, Vol. 63, pp. 15-36. Studies in Computational Intelligence, Springer, Berlin (2007b). Also available as arXiv:cs.AI/0606081
93. Schmidhuber, J.: Simple algorithmic principles of discovery, subjective beauty, selective attention, curiosity & creativity. In: Proceedings of the 10th International Conf. on Discovery Science (DS 2007), LNAI, Vol. 4755, pp. 26-38. Springer, Berlin. Joint invited lecture for ALT 2007 and DS 2007, Sendai, Japan, 2007 (2007c)
94. Schmidhuber, J.: Driven by compression progress. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds.) Knowledge-Based Intelligent Information and Engineering Systems KES-2008, Lecture Notes in Computer Science, Vol. 5177, Part I, p. 11. Springer, Berlin. Abstract of invited keynote (2008)
95. Schmidhuber, J.: Art & science as by-products of the search for novel patterns, or data compressible in unknown yet learnable ways. In: Botta, M. (ed.) Multiple ways to design research. Research cases that reshape the design discipline, Swiss Design Network - Et al. Edizioni, pp. 98-112. Springer, Berelin (2009a)
96. Schmidhuber, J.: Compression progress: The algorithmic principle behind curiosity and creativity (with applications of the theory of humor). 40 min video of invited talk at Singularity Summit 2009, New York City (2009b). http://www.vimeo.com/7441291.
97. 10 min excerpts: http://www.youtube.com/watch?v=Ipomu0MLFaI.
98. Schmidhuber, J.: Driven by compression progress: A simple principle explains essential aspects of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. In: Pezzulo, G., Butz, M.V., Sigaud, O., Baldassarre, G. (eds.) Anticipatory Behavior in Adaptive Learning Systems. From Psychological Theories to Artificial Cognitive Systems, LNCS, Vol. 5499, pp. 48-76. Springer, Berlin (2009c)
99. Schmidhuber, J.: Simple algorithmic theory of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. J. Soc. Instrum. Control Eng. 48(1), 21-32 (2009d)
100. Schmidhuber, J.: Ultimate cognition à la Gödel. Cogn. Comput. 1(2), 177-193 (2009e)
101. Schmidhuber, J.: Artificial scientists & artists based on the formal theory of creativity. In: Eric, B., Marcus, H., Emanuel, K. (eds.) Artificial General Intelligence. Springer, Berlin (2010a)
102. Schmidhuber, J.: Formal theory of creativity, fun, and intrinsic motivation (1990-2010): IEEE Trans. Auton. Mental Dev. 2(3), 230-247 (2010b)
103. Schmidhuber, J.: A formal theory of creativity to model the creation of art. In: McCormack, J., d'Inverno, M. (eds.) Computational Creativity. MIT, Cambridge (2012)
104. Schmidhuber, J., Gomez, F., Graves, A.: Sequence Learning with Artificial Recurrent Neural Networks. Cambridge University Press, Cambridge (2012) (in preparation)
105. Schmidhuber, J., Heil, S.: Sequential neural text compression. IEEE Trans. Neural Netw. 7(1), 142-146 (1996)
106. Schmidhuber, J., Wahnsiedler, R.: Planning simple trajectories using neural subgoal generators. In: Meyer, J.A., Roitblat, H.L., Wilson, S.W. (eds.) Proc. of the 2nd International Conference on Simulation of Adaptive Behavior, pp. 196-202. MIT, Cambridge (1992)
107. Schmidhuber, J., Zhao, J., Wiering, M.: Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement. Mach. Learn. 28, 105-130 (1997)
108. Schwefel, H.P.: Numerische Optimierung von Computer-Modellen. Dissertation. Published 1977 by Birkhäuser, Basel (1974)
109. Sehnke, F., Osendorfer, C, Rückstieß, T., Graves, A., Peters, J., Schmidhuber, J.: Parameter-exploring policy gradients. Neural Netw. 23(4), 551-559 (2010)
110. Seung, H.S., Opper, M., Sompolinsky, H.: Query by committee. In: COLT '92: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 287-294. ACM, New York (1992)
111. Shannon, C.E.: A mathematical theory of communication (parts I and II): Bell Syst. Techn. J. XXVII, 379-423 (1948)
112. Singh, S., Barto, A.G., Chentanez, N: Intrinsically motivated reinforcement learning. In: Advances in Neural Information Processing Systems 17 (NIPS). MIT, Cambridge (2005)
113. Singh, S., Lewis, R.L., Barto, A.G.: Where do rewards come from? In: Taatgen, N, van Rijn, H. (eds.) Proceedings of the 31st Annual Conference of the Cognitive Science Society. Austin (2009)
114. Solomonoff, R.J.: A formal theory of inductive inference. Part I. Inform. Control 7, 1-22 (1964)
115. Solomonoff, R.J.: Complexity-based induction systems. IEEE Trans. Inform. Theory IT-24(5), 422-432 (1978)
116. Storck, J., Hochreiter, S., Schmidhuber, J.: Reinforcement driven information acquisition in non-deterministic environments. In: Proceedings of the International Conference on Artificial Neural Networks, Paris, Vol. 2, pp. 159-164. EC2 & Cie (1995)
117. Strehl, A., Langford, J., Kakade, S.: Learning from logged implicit exploration data. Technical Report arXiv:1003.0120 (2010)
118. Stronger, D., Stone, P.: Towards autonomous sensor and actuator model induction on a mobile robot. Connect. Sci. 18(2) (2006)
119. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT, Cambridge (1998)
120. Sutton, R.S., McAllester, D.A., Singh, S.P, Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. ü In: Solla, S.A., Leen, T.K., Müller, K.-R. (eds.) Advances in Neural Information Processing Systems 12 [NIPS Conference, Denver, Colorado, USA, November 29 - December 4 (1999)], pp. 1057-1063. MIT, Cambridge (1999)
121. Wallace, C.S., Boulton, D.M.: An information theoretic measure for classification. Comput. J. 11(2), 185-194 (1968)
122. Wallace, C.S., Freeman, PR.: Estimation and inference by compact coding. J. R. Stat. Soc. B 49(3), 240-265 (1987)
123. Watkins, C, Dayan, P.: Q-learning. Mach. Learn. 8(3/4), 279-292 (1992)
124. Wiering, M., Schmidhuber, J.: HQ-learning. Adap. Behav. 6(2), 219-246 (1998a)
125. Wiering, M.A., Schmidhuber, J.: Fast online Q(X): Mach. Learn. 33(1), 105-116 (1998b)
126. Wierstra, D., Schaul, T, Peters, J., Schmidhuber, J.: Fitness expectation maximization. In: Proceedings of Parallel Problem Solving from Nature (PPSN 2008) (2008)
127. Wilson, S.: ZCS: A zeroth level classifier system. Evol. Comput. 2, 1-18 (1994)
128. Wundt, WM.: Grundzüge der Phvsiologischen Psychologie. Engelmann, Leipzig (1874)