Context-free and context-sensitive dynamics in recurrent neural networks

Connection Science

Volumn 12, Issue 3-4, 2000, Pages 197-210

  Bodén, Mikael ^a   Wiles, Janet ^a  

^a UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

Context free grammar; Context sensitive grammar; Dynamical system; Language; Learning; Recurrent neural network; Recursive structure

Indexed keywords

EID: 0034345038     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/095400900750060122     Document Type: Article

References (28)

1. Blair, A. D., and Pollack, J. B. 1997, Analysis of dynamical recognizers. Neural Computation 9 (5): 1127-1142.
2. Bodén, M., Wiles, J., Tonkes, B., and Blair, A., 1999, Learning to predict a context-free language: analysis of dynamics in recurrent hidden units. In Proceedings of the International Conference on Artificial Neural Networks, Edinburgh, IEE, pp. 359-364.
3. Casey, M. 1996, The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation, 8 (6): 1135-1178.
4. n language. In Proceedings of the 6th International Conference on Neural Information Processing Perth, pp. 508-513.
5. Christiansen, M., and Chater, N., 1999, Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23: 157-205.
6. Clecremans, A., Servan-Schreiber, D., and McClelland, J. L., 1989, Finite state automata and simple recurrent networks. Neural Computation, 1 (3): 372-381.
7. Das, S., Giles, C. L., and Sun, G.-Z., 1993, Using prior knowledge in a NNPDA to learn context-free languages. In S. J. Hanson, J. D. Cowan and C. L. Giles (eds) Advances in Neural Information Processing Systems, Vol. 5 (San Mateo CA: Morgan Kaufmann), pp. 65-72.
8. Devaney, R. L., 1989, An Introduction to Chaotic Dynamical Systems (Reading MA: Addison-Wesley).
9. Elman, J. L., 1990, Finding structure in time. Cognitive Science, 14: 179-211.
10. Elman, J. L., 1993, Learning and development in neural networks: the importance of starting small. Cognition, 48: 71-99.
11. Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., and Lee, Y. C., 1992, Learning and extracted finite state automata with second-order recurrent neural networks. Neural Computation 4 (3): 393-405.
12. Gold, E. M., 1967, Language identification in the limit. Information and Control, 16: 447-474.
13. Hirsch, M. W., and Smale, S., 1974, Differential Equations, Dynamical Systems, and Linear Algebra (New York: Academic Press).
14. Hölldobler, S., Kalinke, Y., and Lehmann, H., 1997, Designing a counter: another case study of dynamics and activation landscapes in recurrent networks. In Proceedings of KI-97: Advances in Artificial Intelligence (Berlin: Springer), pp. 313-324.
15. Hopcraft, J. E., and Ullman, J. D., 1979, Introduction to Automata Theory, Languages, and Computation (Reading MA: Addison-Wesley).
16. Jacobsson, H., 1999, A comparison of simple recurrent and sequential cascaded networks for formal language recognition. Master's thesis, University of Skövde, Sweden.
17. Kolen, J. F., 1994, Fool's gold: extracting finite state machines from recurrent network dynamics. In J. D. Cowan. G. Tesauro and J. Alspector (eds) Advances in Neural Information Processing Systems, Vol. 6 (San Francisco: Morgan Kaufmann), pp. 501-508.
18. Moore, C., 1998, Dynamical recognizers: real-time language recognition by analog computers. Theoretical Computer Science, 201: 99-136.
19. Pollack, J. B., 1991, The induction of dynamical recognizers. Machine Learning, 7: 227.
20. Rodriguez, P., and Wiles, J., 1998, Recurrent neural networks can learn to implement symbol-sensitive counting. In M. I. Jordan, M. J. Kearns and S. A. Solla (eds) Advances in Neural Information Processing Systems, Vol. 10 (Cambridge, Mass: The MIT Press).
21. Rodriguez, P., Wiles, J., and Elman, J. L., 1999, A recurrent neural network that learns to count. Connection Science, 11 (1): 5-40.
22. Rohde, D. L. T., and Flaut, D. C., 1999, Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72: 67-109.
23. Siegelmann, H. T., 1999, Neural Networks and Analog Computation: Beyond the Turning Limit (Birkhäuser).
24. Steijvers, M., and Grünwald, P., 1996, A Recurrent Network that Performs a Context-sensitive Prediction Task. Technical Report NC-TR-96-035, NeuroCOLT, Royal Holloway, University of London.
25. Tabor, W., 2000, Fractal encoding of context free grammars in connectionist networks. Expert Systems: The International Journal of Knowledge Engineering and Neural Networks, 17 (1): 41-56.
26. Tino. P., Horne, B. G., Giles, C. L., and Collingwood, P. C., 1998, Finite state machines and recurrent neural networks-automata and dynamical systems approaches. In J. Dayhoff and O. Omidvar (eds) Neural Networks and Pattern Recognition (New York: Academic Press), pp. 171-220.
27. Wiles, J., and Elman, J. L., 1995, Learning to count without a counter: a case study of dynamics and activation landscapes in recurrent networks. In Proceedings of the Seventeenth Annual Meeting of the Cognitive Science Society (Mahwah, NJ: Lawrence Erlbaum), pp. 482-487.
28. Zeng, Z., Goodman, R. M., and Smyth, P., 1994, Discrete recurrent neural networks for grammatical inference. IEEE Transactions on Neural Networks, 5 (2): 320-330.