Evolutionary computation versus reinforcement learning

IECON Proceedings (Industrial Electronics Conference)

Volumn 1, Issue , 2000, Pages 2992-2997

  Schmidhuber, J ^a  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

DYNAMIC PROGRAMMING; EVOLUTIONARY ALGORITHMS; INDUSTRIAL ELECTRONICS; STOCHASTIC SYSTEMS;

CREDIT ASSIGNMENT; FITNESS FUNCTIONS; PARTIAL OBSERVABILITY; STOCHASTIC ENVIRONMENT; STOCHASTIC POLICY; UNKNOWN ENVIRONMENTS; VALUE FUNCTIONS;

REINFORCEMENT LEARNING;

EID: 84969172313     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IECON.2000.972474     Document Type: Conference Paper

References (52)

1. W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone. Genetic Programming An Introduction. Morgan Kaufmann Publishers, San Francisco, CA, USA, 1998.
2. R. Bellman. Adaptive Control Processes. Princeton University Press, 1961.
3. G.J. Chaitin. On the length of programs for computing finite binary sequences: statistical considerations. Journal of the ACM, 16:145-159, 1969.
4. N. L. Cramer. A representation for the adaptive generation of simple sequential programs. In J.J. Grefenstette, editor, Proceedings of an International Conference on Genetic Algorithms and Their Applications, Hillsdale NJ, 1985. Lawrence Erlbaum Associates.
5. P. Dayan and G. Hinton. Feudal reinforcement learning. In D. S. Lippman, J. E. Moody, and D. S. Touretzky, editors, Advances in Neural Information Processing Systems 5, pages 271-278. San Mateo, CA: Morgan Kaufmann, 1993.
6. D. Dickmanns, J. Schmidhuber, and A. Winklhofer. Der genetische Algorithmus: Eine Implementierung in Prolog. Fortgeschrittenenpraktikum, Institut für Informatik, Lehrstuhl Prof. Radig, Technische Universität München, 1987.
7. B.L. Digney. Emergent hierarchical control structures: Learning reactive/hierarchical relationships in reinforcement environments. In Pattie Maes, Maja Mataric, Jean-Arcady Meyer, Jordan Pollack, and Stewart W. Wilson, editors, From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, Cambridge, MA, pages 363-372. MIT Press, Bradford Books, 1996.
8. M. Eldracher and B. Baginski. Neural subgoal generation using backpropagation. In George G. Lendaris, Stephen Grossberg, and Bart Kosko, editors, World Congress on Neural Networks, pages III-145-III-148. Lawrence Erlbaum Associates, Inc., Publishers, Hillsdale, July 1993.
9. J. H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, 1975.
10. M. Humphrys. Action selection methods using reinforcement learning. In Pattie Maes, Maja Mataric, Jean-Arcady Meyer, Jordan Pollack, and Stewart W. Wilson, editors, From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, Cambridge, MA, pages 135-144. MIT Press, Bradford Books, 1996.
11. T. Jaakkola, S. P. Singh, and M. I. Jordan. Reinforcement learning algorithm for partially observable Markov decision problems. In G. Tesauro, D. S. Touretzky, and T. K. Leen, editors, Advances in Neural Information Processing Systems 7, pages 345-352. MIT Press, Cambridge MA, 1995.
12. L.P. Kaelbling, M.L. Littman, and A.R. Cassandra. Planning and acting in partially observable stochastic domains. Technical report, Brown University, Providence RI, 1995.
13. M. Kearns and S. Singh. Finite-sample convergence rates for Q-learning and indirect algorithms. In M. Kearns, S. A. Solla, and D. Cohn, editors, Advances in Neural Information Processing Systems 12. MIT Press, Cambridge MA, 1999.
14. A.N. Kolmogorov. Three approaches to the quantitative definition of information. Problems of Information Transmission, 1:1-11, 1965.
15. D. Lenat. Theory formation by heuristic search. Machine Learning, 21, 1983.
16. M. Li and P. M. B. Vitányi. An Introduction to Kolmogorov Complexity and its Applications. Springer, 1993.
17. L.J. Lin. Reinforcement Learning for Robots Using Neural Networks. PhD thesis, Carnegie Mellon University, Pittsburgh, January 1993.
18. M.L. Littman. Algorithms for Sequential Decision Making. PhD thesis, Brown University, March 1996.
19. M.L. Littman, A.R. Cassandra, and L.P. Kaelbling. Learning policies for partially observable environments: Scaling up. In A. Prieditis and S. Russell, editors, Machine Learning: Proceedings of the Twelfth International Conference, pages 362-370. Morgan Kaufmann Publishers, San Francisco, CA, 1995.
20. R. A. McCallum. Learning to use selective attention and short-term memory in sequential tasks. In Pattie Maes, Maja Mataric, Jean-Arcady Meyer, Jordan Pollack, and Stewart W. Wilson, editors, From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, Cambridge, MA, pages 315-324. MIT Press, Bradford Books, 1996.
21. A. Moore and C. G. Atkeson. Prioritized sweeping: Reinforcement learning with less data and less time. Machine Learning, 13:103-130, 1993.
22. T. S. Ray. An approach to the synthesis of life. In C.G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen, editors, Artificial Life II, pages 371-408. Addison Wesley Publishing Company, 1992.
23. I. Rechenberg. Evolutionsstrategie - Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Dissertation, 1971. Published 1973 by Fromman-Holzboog.
24. M. B. Ring. Incremental development of complex behaviors through automatic construction of sensory-motor hierarchies. In L. Birnbaum and G. Collins, editors, Machine Learning: Proceedings of the Eighth International Workshop, pages 343-347. Morgan Kaufmann, 1991.
25. M. B. Ring. Learning sequential tasks by incrementally adding higher orders. In J. D. Cowan S. J. Hanson and C. L. Giles, editors, Advances in Neural Information Processing Systems 5, pages 115-122. Morgan Kaufmann, 1993.
26. M. B. Ring. Continual Learning in Reinforcement Environments. PhD thesis, University of Texas at Austin, Austin, Texas 78712, August 1994.
27. R. P. Salustowicz and J. Schmidhuber. Probabilistic incremental program evolution. Evolutionary Computation, 5(2):123-141, 1997.
28. A. L. Samuel. Some studies in machine learning using the game of checkers. IBM Journal on Research and Development, 3:210-229, 1959.
29. J. Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. Institut für Informatik, Technische Universität München, 1987.
30. J. Schmidhuber. Learning to generate sub-goals for action sequences. In T. Kohonen, K. Mäkisara, O. Simula, and J. Kangas, editors, Artificial Neural Networks, pages 967-972. Elsevier Science Publishers B.V., North-Holland, 1991.
31. J. Schmidhuber. Reinforcement learning in Markovian and non-Markovian environments. In D. S. Lippman, J. E. Moody, and D. S. Touretzky, editors, Advances in Neural Information Processing Systems 3, pages 500-506. San Mateo, CA: Morgan Kaufmann, 1991.
32. J. Schmidhuber. Discovering solutions with low Kolmogorov complexity and high generalization capability. In A. Prieditis and S. Russell, editors, Machine Learning: Proceedings of the Twelfth International Conference, pages 488-496. Morgan Kaufmann Publishers, San Francisco, CA, 1995.
33. J. Schmidhuber. A general method for incremental self-improvement and multi-agent learning. In X. Yao, editor, Evolutionary Computation: Theory and Applications, pages 81-123. Scientific Publ. Co., Singapore, 1999.
34. J. Schmidhuber and J. Zhao. Direct policy search and uncertain policy evaluation. In AAAI Spring Symposium on Search under Uncertain and Incomplete Information, Stanford Univ., pages 119-124. American Association for Artificial Intelligence, Menlo Park, Calif., 1999.
35. J. Schmidhuber, J. Zhao, and N. Schraudolph. Reinforcement learning with self-modifying policies. In S. Thrun and L. Pratt, editors, Learning to learn, pages 293-309. Kluwer, 1997.
36. J. Schmidhuber, J. Zhao, and M. Wiering. Simple principles of metalearning. Technical Report IDSIA-69-96, IDSIA, 1996.
37. J. Schmidhuber, J. Zhao, and M. Wiering. Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement. Machine Learning, 28:105-130, 1997.
38. H. P. Schwefel. Numerische Optimierung von Computer-Modellen. Dissertation, 1974. Published 1977 by Birkhäuser, Basel.
39. S.P. Singh. The efficient learning of multiple task sequences. In J.E. Moody, S.J. Hanson, and R.P. Lippman, editors, Advances in Neural Information Processing Systems 4, pages 251-258, San Mateo, CA, 1992. Morgan Kaufmann.
40. R.J. Solomonoff. A formal theory of inductive inference. Part I. Information and Control, 7:1-22, 1964.
41. R.J. Solomonoff. An application of algorithmic probability to problems in artificial intelligence. In L. N. Kanal and J. F. Lemmer, editors, Uncertainty in Artificial Intelligence, pages 473-491. Elsevier Science Publishers, 1986.
42. R. Sun and C. Sessions. Self-segmentation of sequences: automatic formation of hierarchies of sequential behaviors. IEEE Transactions on Systems, Man, and Cybernetics: Part B Cybernetics, 30(3), 2000.
43. R. S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3:9-44, 1988.
44. R. S. Sutton. TD models: Modeling the world at a mixture of time scales. In A. Prieditis and S. Russell, editors, Machine Learning: Proceedings of the Twelfth International Conference, pages 531-539. Morgan Kaufmann Publishers, San Francisco, CA, 1995.
45. R. S. Sutton, S. Singh, D. Precup, and B. Ravindran. Improved switching among temporally abstract actions. In Advances in Neural Information Processing Systems 11. MIT Press, 1999. To appear.
46. A. Teller. The evolution of mental models. In Jr. Kenneth E. Kinnear, editor, Advances in Genetic Programming, pages 199-219. MIT Press, 1994.
47. G. Tesauro. TD-gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation, 6(2):215-219, 1994.
48. C.K. Tham. Reinforcement learning of multiple tasks using a hierarchical CMAC architecture. Robotics and Autonomous Systems, 15(4):247-274, 1995.
49. C.J.C.H Watkins. Learning from Delayed Rewards. PhD thesis, King's College, Oxford, 1989.
50. M. Wiering and J. Schmidhuber. HQ-learning. Adaptive Behavior, 6(2):219-246, 1998.
51. M.A. Wiering and J. Schmidhuber. Solving POMDPs with Levin search and EIRA. In L. Saitta, editor, Machine Learning: Proceedings of the Thirteenth International Conference, pages 534-542. Morgan Kaufmann Publishers, San Francisco, CA, 1996.
52. R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.