Stabilizing and improving the learning speed of 2-layered LSTM network

Proceedings - 2008 IEEE 11th International Conference on Computational Science and Engineering, CSE 2008

Volumn , Issue , 2008, Pages 293-300

  Corrêa, Débora C ^a   Levada, Alexandre L M ^b   Saito, José Hiroki ^b  

^a FEDERAL UNIVERSITY OF SÃO CARLOS   (Brazil)

^b UNIVERSITY OF SÃO PAULO   (Brazil)

Author keywords

[No Author keywords available]

Indexed keywords

SPEED; TWO DIMENSIONAL;

COMPUTATIONAL SCIENCE AND ENGINEERING; HIDDEN LAYERS; INITIALIZATION METHODS; INTERNATIONAL CONFERENCES; LEARNING SPEED; MEMORY CELLS; MLP NETWORKS; NETWORK WEIGHTS; NON-LINEAR FUNCTIONS; NOVEL METHODS; TRAINING PROCESSES; WEIGHT INITIALIZATION;

TECHNOLOGY;

EID: 51849149899     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CSE.2008.32     Document Type: Conference Paper

References (14)

1. A. C. Kak, Malcolm Slaney, Principles of Computerized Tomographic Imaging, Society of Industrial and Applied Mathematics, 2001.
2. A. Gaves, J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures." Neural Networks, Vol. 18, Issues 5-6, p. 602-610, 2005.
3. B. A. Pearlmutter, "Gradient calculations for dynamic recurrent neural networks: A survey." IEEE Transactions on Neural Network, 6(5), 1212-1228, 1995.
4. B. Irie and S. Miyake, "Capabilities of three-layerd perceptrons." In proceedings of the IEEE International Conference on Neural Networks, pp. I-641, 1988.
5. D. Nguyen, B. Widrow, "Improving the learning speed of 2-layer neural networks by choosing initial values of adaptive weights," in Proc. IJCNN, vol. 3, pp. 21-26, 1990.
6. F. Gers: Long Short-Term Memory in Recurrent Neural Networks. PhD thesis (2001)
7. Haykin, S., Neural Networks: A comprehensive Foundation, Prentice Hall; 2nd edition (July 6, 1998).
8. J. A. Franklin, "Recurrent Neural Networks for Music Computation", Informs Journal on Computing, Vol.18, No.3, pp.321-338, 2006.
9. J. A. Pérez -Ortiz, F. A. Gers, D. Eck, J.Schmidhuber , "Kalman Filters improve LSTM network performance in problems unsolvable by traditional recurrent nets." Neural Networks, vol. 16, issue 2, p. 241-250, 2003.
10. J. Schmidhuber, D. Wierstra, M. Gagliolo, F. Gomez, "Training Recurrent Networks by Evolino." Neural Computation, MIT Press Journal, 19: 757-779, 2007.
11. P. J. Werbos, "Generalization of backpropagation with application to a recurrent gas market model." Neural Networks, 1, 339-356, 1988.
12. R. J. Williams and D. Zipser, "Gradient-Based Learning Algorithms for Recurrent Networks and Their Computational Complexity." In Back-propagation:Theory, Architectures and Applications. Hillsdale, NJ: Erlbaum, 1992.
13. S. Hochreiter, J. Schmidhuber, Long Short-Term Memory. Neural Computation, 9(8):1735-1780, 1997.
14. S. Hochreiter,, Y. Bengio, P. Frasconi, J. Schmidhuber, "Gradient flow in recurrent nets: The difficulty of learning long-term dependencies." A Field Guide to Dynamical Recurrent Networks, IEEE Press, New Your, 2001.