Learning to learn using gradient descent

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 2130, Issue , 2001, Pages 87-94

  Hochreiter, Sepp ^a   Younger, A Steven ^a   Conwell, Peter R ^b  

^a UCB 450   (United States)

^b WESTMINSTER COLLEGE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPLEX NETWORKS; LEARNING SYSTEMS; NEURAL NETWORKS; RECURRENT NEURAL NETWORKS;

EVOLUTIONARY METHOD; FREE PARAMETERS; GRADIENT DESCENT; GRADIENT DESCENT METHOD; LEARNING TO LEARN; META-LEARNING APPROACH; META-LEARNING SYSTEM; NON-STATIONARY TIME SERIES;

LEARNING ALGORITHMS;

EID: 84958985283     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/3-540-44668-0_13     Document Type: Conference Paper

References (19)

1. R. Caruana. Learning many related tasks at the same time with backpropagation. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems 7, pages 657-664. The MIT Press, 1995.
2. D. Chalmers. The evolution of learning: An experiment in genetic connectionism. In D. S. Touretzky, J. L. Elman, T. J. Sejnowski, and G. E. Hinton, editors, Proc. of the 1990 Con. Models Summer School, pages 81-90. Morgan Kaufmann, 1990.
3. N. E. Cotter and P. R. Conwell. Fixed-weight networks can learn. In Int. Joint Conference on Neural Networks, volume 2, pages 553-559. IEEE, NY, 1990.
4. H. Ellis. Transfer of Learning. MacMillan, New York, NY, 1965.
5. J. L. Elman. Finding structure in time. Technical Report CRL 8801, Center for Researchin Language, University of California, San Diego, 1988.
6. D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant’s learning framework. Artificial Intelligence, 36:177-221, 1988.
7. S. Hochreiter and J. Schmidhuber. Flat minima. Neural Comp., 9(1):1-42, 1997.
8. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.
9. A. J. Robinson and F. Fallside. The utility driven dynamic error propagation network. Technical Report CUED/F-INFENG/TR.1, Camb. Uni. Eng. Dep., 1987.
10. T. P. Runarsson and M. T. Jonsson. Evolution and design of distributed learning rules. In 2000 IEEE Symposium of Combinations of Evolutionary Computing and Neural Networks, San Antonio, Texas, USA, page 59. 2000.
11. J. Schmidhuber, J. Zhao, and M. Wiering. Simple principles of metalearning. Technical Report IDSIA-69-96, IDSIA, 1996.
12. J. Schmidhuber, J. Zhao, and M. Wiering. Shifting inductive bias with successstory algorithm, adaptive levin search, and incremental self-improvement. Machine Learning, 28:105-130, 1997.
13. J. Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-… hook. Inst. für Inf., Tech. Univ. München, 1987.
14. S. Thrun and L. Pratt, editors. Learning To Learn. Kluwer Academic Pub., 1997.
15. P. Utgoff. Shift of bias for inductive concept learning. In R. Michalski, J. Carbonell, and T. Mitchell, editors, Machine Learning, volume 2. Morgan Kaufmann, 1986.
16. P. J. Werbos. Generalization of backpropagation withapplication to a recurrent gas market model. Neural Networks, 1, 1988.
17. R. J. Williams and D. Zipser. A learning algorithm for continually running fully recurrent networks. Technical Report ICS 8805, Univ. of Cal., La Jolla, 1988.
18. R. J. Williams and D. Zipser. Gradient-based learning algorithms for recurrent networks and their computational complexity. In Y. Chauvin and D. E. Rumelhart, editors, Back-propagation: Theory, Architectures and Applications. Hillsdale, 1992.
19. A. S. Younger, P. R. Conwell, and N. E. Cotter. Fixed-weight on-line learning. IEEE-Transactions on Neural Networks, 10(2):272-283, 1999.