Learning long-term dependencies in segmented memory recurrent neural networks

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 3173, Issue , 2004, Pages 362-369

  Chen, Jinmiao ^a   Chaudhari, Narendra S ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; RECURRENT NEURAL NETWORKS;

GRADIENT BASED; GRADIENT DESCENT; GRADIENT DESCENT LEARNING ALGORITHM; LONG-TERM DEPENDENCIES; NOVEL ARCHITECTURE; REAL TIME RECURRENT LEARNING ALGORITHMS; RECURRENT NEURAL NETWORK (RNNS); STANDARD PROBLEMS;

LEARNING ALGORITHMS;

EID: 35048871535     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-28647-9_61     Document Type: Article

References (11)

1. Bengio, Y., Simard, P., Frasconi, P.: Learning Long-Term Dependencies with Gradient Descent is Difficult, IEEE Transactions on Neural Networks, vol. 5, no. 2, (1994) 157-166
2. Elman, J.L.: Distributed Representations, Simple Recurrent Networks, and Grammatical Structure, Machine Learning, vol. 7, (1991) 195-226
3. Hihi, S.E., Bengio, Y.: Hierarchical Recurrent Neural Networks for Long-Term Dependencies, Advances in Neural Information Processing Systems, Perrone, M., Mozer, M., Touretzky, D.D. ed., MIT Press, (1996) 493-499
4. Hochreiter, S., Schmidhuber, J.: Long Short-Term Memory, Neural Computation, vol. 9, no. 8, (1997) 1735-1780
5. Ku, K.W.C., Mak, M.W., Siu, W.C.: A Cellular Genetic Algorithm for Training Recurrent Neural Networks, Proc. Int. Conf. Neural Networks and Signal Processing, (1995) 140-143
6. Lin, T., Horne, B.G., Tino, P., Giles, C.L.: Learning Long-Term Dependencies in NARX Recurrent Neural Networks, IEEE Trans. on Neural Networks, vol. 7, (1996) 1329-1337
7. Lin, T., Horne, B.G., Giles, C.L.: How Embedded Memory in Recurrent Neural Network Architectures Helps Learning Long-Term Temporal Dependencies, Neural Networks, vol. 11, (1998) 861-868
8. Ma, S., Ji, C.: Fast Training of Recurrent Networks Based on the EM Algorithms, IEEE Trans. on Neural Networks, vol. 9(1), (1998) 11-26
9. Branko Šter: Latched Recurrent Neural Network, Elektrotehniški vestnik, vol. 70(1-2), (2003) 46-51
10. Werbos, P.J.: Backpropagation through Time: What It Does and How to Do It, Proc. IEEE, vol. 78(10), (1990) 1550-1560
11. Williams, R.J., Zipser, D.: Gradient Based Learning Algorithms for Recurrent Connectionist Networks, Northeastern University, College of Computer Science Technical Report, NU-CCS-90-9, (1990) 433-486