Recurrent Neural Networks

Intelligent Systems Reference Library

Volumn 49, Issue , 2013, Pages 29-65

  Marhon, Sajid A ^a   Cameron, Christopher J F ^a   Kremer, Stefan C ^a  

^a UNIVERSITY OF GUELPH   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84885651436     PISSN: 18684394     EISSN: 18684408     Source Type: Book Series    
DOI: 10.1007/978-3-642-36657-4_2     Document Type: Article

References (59)

1. Allen, R.B., Alspector, J.: Learning of stable states in stochastic asymmetric networks. Technical Report TM-ARH-015240, Bell Communications Research, Morristown, NJ (1989)
2. Atiya, A.F.: Learning on a general network. In: Neural Information Processing Systems, New York, pp. 22-30 (1988)
3. Back, A.D., Tsoi, A.C.: FIR and IIR synapses, a new neural network architecture for time series modeling. Neural Computation 3, 375-385 (1991)
4. Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gadient descent is difficult. IEEE Transactions on Neural Networks 5, 157-166 (1994)
5. Chen, L., Chua, H., Tan, P.: Grammatical inference using an adaptive recurrent neural network. Neural Processing Letters 8, 211-219 (1998)
6. Chen, S., Billings, S., Grant, P.: Nonlinear system identification using neural networks. International Journal of Control 51(6), 1191-1214 (1990)
7. Cohen, M.A., Grossberg, S.: Stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man and Cybernetics 13, 815-826 (1983)
8. Elman, J.L.: Finding structure in time. Cognitive Science 14, 179-211 (1990)
9. Elman, J.L.: Distributed representations, simple recurrent networks and grammatical structure. Machine Learning 7, 195-225 (1991)
10. Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. Technical Report CMU-CS-90-100, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA (February 1990)
11. Forcada, M.L., Ñeco, R.P.: Recursive Hetero-Associative Memories for Translation. In: Mira, J., Moreno-Díaz, R., Cabestany, J. (eds.) IWANN 1997. LNCS, vol. 1240, pp. 453-462. Springer, Heidelberg (1997)
12. Frasconi, P., Gori, M., Soda, G.: Local feedback multilayered networks. Neural Computation 4, 120-130 (1992)
13. Galland, C.C., Hinton, G.E.: Deterministic Boltzman learning in networks with asymmetric connectivity. Technical Report CRG-TR-89-6, University of Toronto Department of Computer Science (1989)
14. Ge, H., Du, W., Qian, F., Liang, Y.: Identification and control of nonlinear systems by a time-delay recurrent neural network. Neurocomputing 72, 2857-2864 (2009)
15. Giles, C., Kuhn, G., Williams, R.: Dynamic recurrent neural networks: theory and applications. IEEE Trans. Neural Netw. 5(2), 153-156 (1994)
16. Giles, C.L., Chen, D., Miller, C.B., Chen, H.H., Sun, G.Z., Lee, Y.C.: Second-order recurrent neural networks for grammatical inference. In: 1991 IEEE INNS International Joint Conference on Neural Networks, Seattle, Piscataway, NJ, vol. 2, pp. 271-281. IEEE Press (1991)
17. Giles, C.L., Horne, B.G., Lin, T.: Learning a class of large finite state machines with a recurrent neural network. Neural Networks 8, 1359-1365 (1995)
18. Giles, C.L., Miller, C.B., Chen, D., Chen, H.H., Sun, G.Z., Lee, Y.C.: Learning and extracting finite state automata with second-order recurrent neural networks. Neural Computation 4, 395-405 (1992)
19. Gori, M., Bengio, Y., Mori, R.D.: Bps: A learning algorithm for capturing the dynamic nature of speech. In: International Joint Conference on Neural Networks, vol. II, pp. 417-423 (1989)
20. Harigopal, U., Chen, H.C.: Grammatical inference using higher order recurrent neural networks. In: Proceedings of the Twenty-Fifth Southeastern Symposium on System Theory, SSST 1993, pp. 338-342 (1993)
21. Hinton, T.J., abd Sejnowski, G.E.: Optimal perceptual inference. In: Proceedines of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 448-453. IEEE Computer Society (1983)
22. Hochreiter, S.: Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis (1991)
23. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9, 1735-1780 (1997)
24. Hopcroft, J.E., Ullman, J.D.: Introduction to Automata Theory, Languages, and Computation. Addison-Wesley (1979)
25. Jordan, M.I.: Supervised learning and systems with excess degrees of freedom. Technical Report COINS Technical Report 88-27, Massachusetts Institute of Technology (1988)
26. Karakasoglu, A., Sudharsanan, S., Sundareshan, M.K.: Identification and decentralized adaptive control using dynamic neural networks with application to robotic manipulators. IEEE Trans. Neural Networks 4, 919-930 (1993)
27. Karray, F.O., Silva, C.: Soft Computing and Intelligent Systems Design. AddisonWesley (2004)
28. Kilian, J., Siegelmann, H.T.: On the power of sigmoid neural networks. In: Proceedings of the Sixth ACM Workshop on Computational Learning Theory, pp. 137-143. ACM Press (1993)
29. Kolen, J.F., Kremer, S.C. (eds.): A Field Guide to Dynamical Recurrent Networks. Wiley-IEEE Press (2001)
30. Kuo, J., Celebi, S.: Adaptation of memory depth in the gamma filter. In: Acoustics, Speech and Signal Processing IEEE Conference, pp. 1-4 (1994)
31. Kuroe, Y.: Representation and Identification of Finite State Automata by Recurrent Neural Networks. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 261-268. Springer, Heidelberg (2004)
32. Lippmann, R.P.:An introduction to computing with neural nets. IEEEASSP Magazine 4, 4-22 (1987)
33. Mozer, M.: A focused background algorithm for temporal pattern recognition. Complex Systems 3 (1989)
34. Mozer, M.C.: Induction of multiscale temporal structure. In: Advances in Neural Information Processing Systems 4, pp. 275-282. Morgan Kaufmann (1992)
35. Nguyen, M., Cottrell, G.: A technique for adapting to speech rate. In: Kamm, C., Kuhn, G., Yoon, B., Chellapa, R., Kung, S. (eds.) Neural Networks for Signal Processing 3. IEEE Press (1993)
36. Omlin, C.W., Giles, C.L.: Constructing deterministic finite-state automata in recurrent neural networks. Journal of the ACM 43(6), 937-972 (1996)
37. Patan, K.: Locally Recurrent Neural Networks. In: Patan, K. (ed.) Artificial. Neural Net. for the Model. & Fault Diagnosis. LNCIS, vol. 377, pp. 29-63. Springer, Heidelberg (2008)
38. Pollack, J.B.: On Connectionist Models of Natural Language Processing. PhD thesis, Computer Science Department of the University of Illinois at Urbana-Champaign, Urbana, Illinois, Available as TR MCCS-87-100, Computing Research Laboratory, New Mexico State University, Las Cruces, NM (1987)
39. Principe, J.C., de Vries, B., de Oliveira, P.G.: The gamma filter-a new class of adaptive IIR filter with restricted feedback. IEEE Transactions on Signal Processing 41, 649-656 (1993)
40. Renals, S., Rohwer, R.: A study of network dynamics. Journal of Statistical Physics 58, 825-848 (1990)
41. Robinson, A.J.: Dynamic Error Propagation Networks. Ph.d., Cambridge University Engineering Department (1989)
42. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Parallel Distributed Processing. MIT Press, Cambridge (1986)
43. Schmidhuber, J.H.: A fixed size storage o(n3) time complexity learning algorithm for fully recurrent continually running networks. Neural Computation 4(2), 243-248 (1992)
44. Sejnowski, T.J., Rosenberg, C.R.: Parallel networks that learn to pronounce english text. Complex Syst. 1, 145-168 (1987)
45. Shannon, C.E.: Communication in the presence of noise. Proc. Institute of Radio Engineers 37(1), 10-21 (1949); reprinted as classic paper in: Proc. IEEE 86(2) (February 1998)
46. Shearer, J.L., Murphy, A.T., Richardson, H.H.: Introduction to System Dynamics. Addison-Wesley, Reading (1971)
47. Siegelmann, H.T., Sontag, E.D.: Turing computability with neural nets. Applied Mathematics Letters 4(6), 77-80 (1991)
48. Silva, T.O.: Laguerre filters-an introduction. Revista do Detua 1(3) (1995)
49. Smith, J.O.: Delay lines. Physical Audio Signal Processing (2010), http://ccrma.stanford.edu/ jos/pasp/ Tapped_Delay_Line_TDL.htm (cited November 28, 2010)
50. Smith, S.W.: The scientist and engineer's guide to digital signal processing. California Technical Publishing (2006), http://www.dspguide.com/ch15.htm (cited November 29, 2010)
51. Tsoi, A.C., Back, A.D.: Locally recurrent globally feedforward networks: A critical review of architectures. IEEE Transactions on Neural Networks 5, 229-239 (1994)
52. Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., Lang, L.: Phonemic recognition using time delay neural networks. IEEE Trans. Acoustic Speech and Signal Processing 37(3), 328-339 (1989)
53. Werbos, P.: Beyond Regression: New Tools for Prediction and Analysis in the Behavioural Sciences. Phd thesis, Harvard University (1974)
54. Williams, R.J., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1, 270-289 (1989)
55. Won, S.H., Song, I., Lee, S.Y., Park, C.H.: Identification of finite state automata with a class of recurrent neural networks. IEEE Transactions on Neural Networks 21(9), 1408-1421 (2010)
56. Yan, P.F., Zhang, C.S.: Artificial Neural Network and Simulated Evolutionary Computation. Thinghua University Press, Beijing (2000)
57. Zamarreno, J.M., Vega, P.: State space neural network. Properties and application. Neural Networks 11, 1099-1112 (1998)
58. Zeng, Z., Goodman, R.M., Smyth, P.: Learning finite state machines with self-clustering recurrent networks. Neural Computation 5(6), 977-990 (1993)
59. Zeng, Z., Goodman, R.M., Smyth, P.: Discrete recurrent neural networks for grammatical inference. IEEE Transactions on Neural Networks 5(2), 320-330 (1994)