Optimal ordered problem solver

Machine Learning

Volumn 54, Issue 3, 2004, Pages 211-254

  Schmidhuber, Jürgen ^a  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

Author keywords

Bias optimality; Efficient planning and backtracking in program space; Incremental optimal universal search; Metasearching and metalearning; OOPS; Self improvement

Indexed keywords

COMPUTATIONAL METHODS; COMPUTER SOFTWARE SELECTION AND EVALUATION; LEARNING ALGORITHMS; OPTIMIZATION; PROBLEM SOLVING; TURING MACHINES;

METALEARNING; METASEARCHING; OPTIMAL ORDERED PROBLEM SOLVER; PROGRAMS TESTING;

LEARNING SYSTEMS;

EID: 1642328943     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/B:MACH.0000015880.99707.b2     Document Type: Article

References (84)

1. Anderson, C. W. (1986). Learning and problem solving with multilayer connectionist systems. Ph.D. thesis, University of Massachusetts, Dept. of Comp. and Inf. Sci.
2. Banzhaf, W., Nordin, P., Keller, R. E., & Francone, F. D. (1998). Genetic Programming - An Introduction. San Francisco, CA, USA: Morgan Kaufmann Publishers.
3. Baum, E. B., & Durdanovic, I. (1999). Toward a model of mind as an economy of agents. Machine Learning, 35:2, 155-185.
4. Bennett, C. H. (1982). The thermodynamics of computation, a review. International Journal of Theoretical Physics, 21:12, 905-940.
5. Bishop, C. M. (1995). Neural Networks for Pattern Recognition. Oxford University Press.
6. Bremermann, H. J. (1982). Minimum energy requirements of information transfer and computing. International Journal of Theoretical Physics, 21, 203-217.
7. Chaitin, G. (1975). A theory of program size formally identical to information theory. Journal of the ACM, 22, 329-340.
8. Cramer, N. L. (1985). A representation for the adaptive generation of simple sequential programs. In J. Grefenstette (Ed.), Proceedings of an International Conference on Genetic Algorithms and Their Applications, Carnegie-Mellon University, July 24-26, 1985. Hillsdale NJ: Lawrence Erlbaum Associates.
9. Deville, Y., & Lau, K. K. (1994). Logic program synthesis. Journal of Logic Programming, 19:20, 321-350.
10. Dickmanns, D., Schmidhuber, J., & Winklhofer, A. (1987). Der genetische Algorithmus: Eine Implementierung in Prolog. Fortgeschrittenenpraktikum, Institut für Informatik, Lehrstuhl Prof. Radig, Technische Universität München.
11. Dorigo, M., Di Caro, G., and Gambardella, L. M. (1999). Ant algorithms for discrete optimization. Artificial Life, 5:2, 137-172.
12. Fredkin, E. F., & Toffoli, T. (1982). Conservative logic. International Journal of Theoretical Physics, 21:3/4, 219-253.
13. Gambardella, L. M. & Dorigo, M. (2000). An ant colony system hybridized with a new local search for the sequential ordering problem. INFORMS Journal on Computing, 12:3, 237-255.
14. Gödel, K. (1931). Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I. Monatshefte für Mathematik und Physik, 38, 173-198.
15. Green, C. C. (1969). Application of theorem proving to problem solving. In D. E. Walker & L. M. Norton (Eds.), Proceedings of the 1st International Joint Conference on Artificial Intelligence, IJCAI (pp. 219-240), Morgan Kaufmann.
16. Hochreiter, S., Younger, A. S., & Conwell, P. R. (2001). Learning to learn using gradient descent. In Lecture Notes on Comp. Sci. 2130, Proc. Intl. Conf. on Artificial Neural Networks (ICANN-2001) (pp. 87-94). Berlin, Heidelberg: Springer.
17. Holland, J. H. (1975). Adaptation in Natural and Artificial Systems. Ann Arbor. University of Michigan Press.
18. Holland, J. H. (1985). Properties of the bucket brigade. In Proceedings of an International Conference on Genetic Algorithms. Hillsdale, NJ: Lawrence Erlbaum.
19. Hutter, M. (2001). Towards a universal theory of artificial intelligence based on algorithmic probability and sequential decisions. In Proceedings of the 12th European Conference on Machine Learning (ECML-2001) (pp. 226-238). (On J. Schmidhuber's SNF grant 20-61847).
20. Hutter, M. (2002a). The fastest and shortest algorithm for all well-defined problems. International Journal of Foundations of Computer Science, 13:3, 431-443. (On J. Schmidhuber's SNF grant 20-61847).
21. Hutter, M. (2002b). Self-optimizing and pareto-optimal policies in general environments based on Bayes-mixtures. In J. Kivinen & R. H. Sloan (Eds.), Proceedings of the 15th Annual Conference on Computational Learning Theory (COLT 2002) (pp. 364-379). Sydney, Australia, Springer. (On J. Schmidhuber's SNF grant 20-61847).
22. Jordan, M. I. & Rumelhart, D. E. (1990). Supervised learning with a distal teacher. Technical Report Occasional Paper #40, Center for Cog. Sci., Massachusetts Institute of Technology.
23. Koopman, Jr., P. J. (1989). Stack Computers: The New Wave. http://www-2.cs.cmu.edu/~koopman/stack_computers/index.html.
24. Kaelbling, L., Littman, M., & Moore, A. (1996). Reinforcement learning: A survey. Journal of AI research, 4:237-285.
25. Koehler, J., Nebel, B., Hoffman, J., & Dimopoulos, Y. (1997). Extending planning graphs to an adl subset. In S. Steel (Ed.), Proceedings of the 4th European Conference on Planning, Vol. 1348 of LNAI (pp. 273-285). Springer.
26. Kolmogorov, A. (1965). Three approaches to the quantitative definition of information. Problems of Information Transmission, 1, 1-11.
27. Kwee, I., Hunter, M., & Schmidhuber, J. (2001). Market-based reinforcement learning in partially observable worlds. In Proceedings of the International Conference on Artificial Neural Networks (ICANN-2001) (IDSIA-10-01, cs.AI/0105025).
28. Langley, P. (1985). Learning to search: From weak methods to domain-specific heuristics. Cognitive Science, 9, 217-260.
29. Lenat, D. (1983). Theory formation by heuristic search. Machine Learning, 21.
30. Levin, L. A. (1973). Universal sequential search problems. Problems of Information Transmission, 9:3, 265-266.
31. Levin, L. A. (1974). Laws of information (nongrowth) and aspects of the foundation of probability theory. Problems of Information Transmission, 10:3, 206-210.
32. Levin, L. A. (1984). Randomness conservation inequalities: Information and independence in mathematical theories. Information and Control, 61, 15-37.
33. Li, M., & Vitányi, P. M. B. (1997). An Introduction to Kolmogorov Complexity and its Applications (2nd edition). Springer.
34. Lloyd, S. (2000). Ultimate physical limits to computation. Nature, 406, 1047-1054.
35. Mitchell, T. (1997). Machine Learning. McGraw Hill.
36. Moore, C. H., & Leach, G. C. (1970). Forth - A language for interactive computing. http;//www.ultratechnology.com.
37. Newell, A. & Simon, H. (1963). GPS, a program that simulates human thought. In E. Feigenbaum & J. Feldman (Eds.) Computers and Thought (pp. 279-293). New York: McGraw-Hill.
38. Nguyen, & Widrow, B. (1989). The truck backer-upper: An example of self learning in neural networks. In Proceedings of the International Joint Conference on Neural Networks (pp. 357-363). IEEE Press.
39. Olsson, J. R. (1995). Inductive functional programming using incremental program transformation. Artificial Intelligence, 74:1, 55-83.
40. Rechenberg, I. (1971). Evolutionsstrategie - Optimierung technischer systeme nach prinzipien der biologischen evolution. Dissertation. Published 1973 by Fromman-Holzboog.
41. Rosenbloom, P. S., Laird, J. E., & Newell, A. (1993). The SOAR Papers. MIT Press.
42. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel Distributed Processing (Vol. 1, pp. 318-362). MIT Press.
43. Russell, S. & Norvig, P. (1994). Artificial Intelligence: A Modern Approach. Englewood Cliffs, NJ: Prentice Hall.
44. Salustowicz, R. P. & Schmidhuber, J. (1997). Probabilistic incremental program evolution. Evolutionary Computation, 5:2, 123-141.
45. Salustowicz, R. P. & Schmidhuber, J. (1998). Evolving structured programs with hierarchical instructions and skip nodes. In. J. Shavlik (Ed.), Machine Learning: Proceedings of the Fifeteenth International Conference (ICML'98) (pp. 488-496). San Francisco: Morgan Kaufmann Publishers.
46. Salustowicz, R. P., Wiering, M. A., & Schmidhuber, J. (1998). Learning team strategies: soccer case studies. Machine Learning, 33:2/3, 263-282.
47. Schmidhuber, J. (1987). Evolutionary principles in self-referential learning. Diploma thesis, Institut für Informatik, Technische Universität München.
48. Schmidhuber, J. (1991). Reinforcement learning in Markovian and non-Markovian environments. In D. S. Lippman, J. E. Moody, & D. S. Touretzky (Eds.), Advances in Neural Information Processing Systems 3 (pp. 500-506). Morgan Kaufmann.
49. Schmidhuber, J. (1993a). An introspective network that can learn to run its own weight change algorithm. In Proc. of the Intl. Conf. on Artificial Neural Networks (pp. 191-195). Brighton: IEE.
50. Schmidhuber, J. (1993b). A self-referential weight matrix. In Proceedings of the International Conference on Artificial Neural Networks (pp. 446-451). Springer. Amsterdam.
51. Schmidhuber, J. (1994). On learning how to learn learning strategies. Technical Report FKI-198-94, Fakultät für Informatik, Technische Universität München. See (Schmidhuber, Zhao, & Wiering, 1997b; Schmidhuber, Zhao, & Schraudolph, 1997a)
52. Schmidhuber, J. (1995). Discovering solutions with low Kolmogorov complexity and high generalization capability. In A. Prieditis & S. Russell (Eds.), Machine Learning: Proceedings of the Twelfth International Conference (pp. 488-496). San Francisco, CA: Morgan Kaufman Publishers.
53. Schmidhuber, J. (1997). Discovering neural nets with low Kolmogorov complexity and high generalization capability. Neural Networks, 10:5, 857-873.
54. Schmidhuber, J. (2000). Algorithmic theories of everything. Technical report IDSIA-20-00, quant-ph/0011122, IDSIA, Manno (Lugano), Switzerland. Sections 1-5: see (Schmidhuber, 2002b); Section 6: see (Schmidhuber, 2002d).
55. Schmidhuber, J. (2001). Sequential decision making based on direct search. In. R. Sun & C. L. Giles (Eds.), Sequence Learning: Paradigms, Algorithms, and Applications. Springer. Lecture Notes on AI 1828.
56. Schmidhuber, J. (2002a). Exploring the predictable. In A. Ghosh & S. Tsuitsui (Eds.), Advances in Evolutionary Computing (pp. 579-612). Springer.
57. Schmidhuber, J. (2002b). Hierarchies of generalized Kolmogorov complexities and nonenumerable universal measures computable in the limit. International Journal of Foundations of Computer Science, 13:4, 587-612.
58. Schmidhuber, J. (2002c), Optimal ordered problem solver. Technical Report IDSIA-12-02, arXiv:cs.AI/0207097, IDSIA, Manno-Lugano, Switzerland.
59. Schmidhuber, J. (2002d). The speed prior: A new simplicity measure yielding near-optimal computable predictions. In J. Kivinen & R. H. Sloan (Eds.), Proceedings of the 15th Annual Conference on Computational Learning Theory (COLT 2002) (pp. 216-228). Lecture Notes in Artificial Intelligence. Sydney, Australia: Springer.
60. Schmidhuber, J. (2003a). Bias-optimal incremental problem solving. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in Neural Information Processing Systems 15 (pp. 1571-1578). Cambridge, MA: MIT Press.
61. Schmidhuber, J. (2003b). Gödel machines: Self-referential universal problem solvers making probably optimal self-improvements. Technical Report IDSIA-19-03, arXiv:cs.LO/0309048 v2, IDSIA, Manno-Lugano, Switzerland.
62. Schmidhuber, J. (2003c), The new AI: General & sound & relevant for physics. Technical Report TR IDSIA-04-03, Version 1.0, cs.AI/0302012 v1.
63. Schmidhuber, J. (2003d). The new AI: General & sound & relevant for physics. In B. Goertzel & C. Pennachin (Eds.) Real AI: New Approaches to Artificial General Intelligence. Plenum Press, New York. To appear. Also available as TR IDSIA-04-03, cs.AI/0302012.
64. Schmidhuber, J. (2003e). Towards solving the grand problem of AI. In P. Quaresma, A. Dourado, E. Costa, & J. F. Costa (Eds.), Soft Computing and complex systems. Centro International de Mathematica, Coimbra, Portugal (pp. 77-97). Based on (Schmidhuber, 2003c).
65. Schmidhuber, J., Zhao, J., & Schraudolph, N. (1997a). Reinforcement learning with self-modifying policies. In S. Thrun & L. Pratt (Eds.), Learning to learn (pp. 293-309). Kluwer.
66. Schmidhuber, J., Zhao, J., & Wiering, M. (1996). Simple principles of metalearning. Technical Report IDSIA-69-96, IDSIA. See (Schmidhuber, Zhao, & Schraudolph, 1997a, 1997b).
67. Schmidhuber, J., Zhao, J., & Wiering, M. (1997b). Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement. Machine Learning, 28, 105-130.
68. Schmidhuber, J., Zhumatiy, V., & Gagliolo, M. (2004). Bias-optimal incremental learning of control sequences for virtual robots. In Proc. 8th Conference on Intelligent Autonomous Systems IAS-8. Amsterdam, NL.
69. Schwefel, H. P. (1974). Numerische optimierung von computer-modellen. Dissertation. Published 1977 by Birkhäuser, Basel.
70. Solomonoff, R. (1964). A formal theory of inductive inference. Part I. Information and Control, 7, 1-22.
71. Solomonoff, R. (1986). An application of algorithmic probability to problems in artificial intelligence. In L. N. Kanal & J. F. Lemmer (Eds.), Uncertainty in Artificial Intelligence (pp. 473-491). Elsevier Science Publishers.
72. Solomonoff, R. (1989). A system for incremental learning based on algorithmic probability. In Proceedings of the Sixth Israeli Conference on Artificial Intelligence, Computer Vision and Pattern Recognition (pp. 515-527). Tel Aviv, Israel.
73. Tsukamoto, M. (1977). Program stacking technique. Information Processing in Japan (Information Processing Society of Japan), 17:1, 114-120.
74. Turing, A. M. (1936). On computable numbers, with an application to the entscheidungsproblem. In Proceedings of the London Mathematical Society, Series 2, 41, 230-267.
75. Ulam, S. (1950). Random processes and transformations. In Proceedings of the International Congress on Mathematics (Vol. 2, pp. 264-275).
76. Utgoff, P. (1986). Shift of bias for inductive concept learning. In. R. Michalski, J. Carbonell, & T. Mitchell (Eds.), Machine Learning (pp. 163-190). Vol. 2, Morgan Kaufmann, Los Altos, CA.
77. Vapnik, V. (1992). Principles of risk minimization for learning theory. In D. S. Lippman, J. E Moody & D. S. Touretzky (Eds.) Advances in Neural Information Processing Systems 4 (pp. 831-838). Morgan Kaufmann.
78. von Neumann, J. (1966). Theory of Self-Reproducing Automata. Champain, IL: University of Illionis Press.
79. Waldinger, R. J., & Lee, C. T. (1969). Prow: A step toward automatic program writing. In. D. E. Walker & L. M. Norton (Eds.), Proceedings of the 1st International Joint Conference on Artificial Intelligence, IJCAI (pp. 241-252). Morgan Kaufmann.
80. Werbos, P. J. (1974). Beyond Regression: New tools for prediction and analysis in the behavioral sciences. Ph.D. thesis, Harvard University.
81. Werbos, P. J. (1987). Learning how the world works: Specifications for predictive networks in robots and brains. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, NY.
82. Wiering, M., & Schmidhuber, J. (1996). Solving POMDPs with levin search and EIRA. In L. Saitta (Ed.) Machine Learning: Proceedings of the Thirteenth International Conference (pp. 534-542). San Francisco, CA: Morgan Kaufmann Publishers.
83. Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for search. IEEE Transactions on Evolutionary Computation, 1.
84. Zuse, K. (1969). Rechnender Raum. Braunschweig: Friedrich Vieweg & Sohn. English translation: Calculating Space, MIT Technical Translation AZT-70-164-GEMIT, Massachusetts Institute of Technology (Prof. MAC), Cambridge, MA 02139, Feb. 1970.