TRTRL: A localized resource-efficient learning algorithm for recurrent neural networks

Midwest Symposium on Circuits and Systems

Volumn 1, Issue , 2006, Pages 371-374

  Budik, Danny ^a,b   Elhanany, Itamar ^a,b  

^a IEEE

^b UNIVERSITY OF TENNESSEE   (United States)

Author keywords

Constraint optimization; Real time recurrent learning (RTRL); Recurrent neural networks

Indexed keywords

COMPUTATIONAL COMPLEXITY; COMPUTER HARDWARE; COMPUTER SIMULATION; CONSTRAINT THEORY; NEURAL NETWORKS; NEURONS; OPTIMIZATION; OXYGEN; PARALLEL ALGORITHMS; REAL TIME SYSTEMS; SENSITIVITY ANALYSIS;

CONSTRAINT OPTIMIZATION; LOCALIZED RESOURCE-EFFICIENT LEARNING ALGORITHM; REAL-TIME RECURRENT LEARNING (RTRL) ALGORITHM;

LEARNING ALGORITHMS;

EID: 34748822254     PISSN: 15483746     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MWSCAS.2006.382075     Document Type: Conference Paper

References (11)

1. R. J. Williams and J. Peng, An efficient gradient-based algorithm for on-line training of recurrent network trajectories, Neural Computation, vol. 2, pp. 490-501, 1990.
2. P. Werbos, Backpropagation through time: what it does and how to do it," in Special issue on neural networks, Proceedings of IEEE, vol. 78, pp. 1550-1560, October 1990.
3. R. J. Williams and D. Zipser, A learning algorithm for continually naming fully recurrent neural networks," Neural Computation, no. 1, pp. 270-280, 1989.
4. D. Zipser, A subgrouping strategy that reduces complexity and speeds up learning in recurrent networks," Neural Computation, no. 1, pp. 552-558, 1989.
5. N. Euliano and J. Principe, Dynamic subgrouping in RTRL provides a faster o(n2) algorithm," in Acoustics, Speech, and Signal Processing, 2000. ICASSP '00. Proceedings. 2000 IEEE International Conference on, vol. 6, (Istanbul), pp. 3418-3421, 2000.
6. J. L. Elman, Finding structure in time," Cogniti e Science, no. 14, pp. 179-211, 1990.
7. J. Schmidhuber, A fixed size storage o(n3) time complexity learning algorithm for fully recurrent continually running networks," Neural Computation, vol. 4, pp. 243-248, 1992.
8. G.-Z. Sun, H.-H. Chen, and Y.-C. Lee, Green's function method for fast on-line learning algorithm of recurrent neural networks.," in NIPS, pp. 333-340, 1991.
9. M. Mackey and L. Glass, Oscillation and chaos in physiological control systems," Science, vol. 197, pp. 287-289, July 1977.
10. R. C. III, Predicting the mackey-glass timeseries with cascade-correlation learning," in D. Touret k , G. Hinton and T. Sejnowski eds., Connectionist Models Summer School Proceedings, pp. 117-123, 1990. Carnegie Mellon University.
11. D. S. V.P. Plagianakos and M. Vrahatis, An improved backpropagation method with adaptive learning rate," in 2nd International Conference on Circuits, S stems, and Computers, 1998. Piraeus, Greece.