Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network

Neural Networks

Volumn 24, Issue 4, 2011, Pages 311-320

  Hinoshita, Wataru ^a   Arie, Hiroaki ^b   Tani, Jun ^b   Okuno, Hiroshi G ^a   Ogata, Tetsuya ^a  

^a KYOTO UNIVERSITY   (Japan)

^b RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

Hierarchical linguistic structure; Language acquisition; Multiple timescale recurrent neural network; Self organization

Indexed keywords

CORRECTION PERFORMANCE; HIERARCHICAL STRUCTURES; INITIAL STATE; LANGUAGE ACQUISITION; LINGUISTIC STRUCTURE; MULTIPLE TIMESCALE RECURRENT NEURAL NETWORK; NEURAL ACTIVATION; SELF ORGANIZING; SELF-ORGANIZATION; SELF-ORGANIZED; TIME-SCALES;

ERRORS; LINGUISTICS; RECURRENT NEURAL NETWORKS;

SPEECH RECOGNITION;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BRAIN FUNCTION; BRAIN NERVE CELL; CONTROLLED STUDY; EXPERIMENTAL DESIGN; GRAMMAR; LANGUAGE PROCESSING; LINGUISTICS; MULTIPLE TIMESCALE RECURRENT NEURAL NETWORK; PRIORITY JOURNAL; PROBABILITY;

COMPREHENSION; HUMANS; LANGUAGE; LEARNING; LINGUISTICS; NEURAL NETWORKS (COMPUTER); NEURONS; RECOGNITION (PSYCHOLOGY); REPRODUCIBILITY OF RESULTS;

EID: 79951944498     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2010.12.006     Document Type: Article

References (25)

1. Bot K.D., Lowie W., Verspoor M. A dynamic systems theory approach to second language acquisition. Bilingualism: Language and Cognition 2007, 10:7-21.
2. Chomsky N. Barrier 1986, MIT Press.
3. Chomsky N. Rules and representations 2005, Columbia University Press.
4. Christiansen M.H., Chater N. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science 1999, 23:157-205.
5. Cleeremans A., Servan-Schreiber D., McClelland J.L. Finite state automata and simple recurrent networks. Neural computation 1989, 1(3):372-381.
6. Devaney R.L. An introduction to chaotic dynamical systems 2003, Westview Press.
7. Elman J.L. Finding structure in time. Cognitive Science 1990, 14:179-211.
8. Elman J.L. Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning 1991, 7(2-3):195-225.
9. Elman J.L. Learning and development in neural networks: the importance of starting small. Cognition 1993, 48(1):71-99.
10. Elman J.L. Language as a dynamical system. Mind as motion: explorations in the dynamics of cognition 1995, 195-223. MIT Press, Cambridge, MA. R. port, T. van Gelder (Eds.).
11. Frasconi P., Gori M. Computational capabilities of local-feedback recurrent networks acting as finite-state machines. IEEE Transactions on Neural Networks 1996, 7(6).
12. Giles C.L., Miller C.B., Chen D., Chen H.H., Sun G.Z., Lee Y.C. Learning and extracting finite state automata with second-order recurrent neural networks. Neural Computation 1992, 4(3):393-405.
13. Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences 1982, 79:2554-2558.
14. Lawrence S., Giles C.L., Fong S. Natural language grammatical inference with recurrent neural networks. IEEE Transactions on Knowledge and Data Engineering 2000, 12(1):126-140.
15. Namikawa J., Tani J. Learning to imitate stochastic time series in a compositional way by chaos. Neural Networks 2010, 23:625-638.
16. Ogata, T., Murase, M., Tani, J., Komatani, K., & Okuno, H. G. (2007). Two-way translation of compound sentences and arm motions by recurrent neural networks. In IEEE/RSJ international conference on intelligent robots and systems. IROS-2007 (pp. 1858-1863).
17. Pollack J.B. The induction of dynamical recognizers. Machine Learning 1991, 7(2-3):227-252.
18. Rodriguez P. Simple recurrent networks learn context-free and context-sensitive languages by counting. Neural Computation 2001, 13(9).
19. Rumelhart D., Hinton G., Williams R. Learning internal representations by error propagation. Parallel distributed processing 1986, 318-362. MIT Press. D. Rumelhart, J. McClelland, R. Williams (Eds.).
20. Smolensky P. Information processing in dynamical systems: foundation of harmony theory. Parallel distributed processing 1986, 194-281. MIT Press, (Chapter 6). D. Rumelhart, J. McClelland, R. Williams (Eds.).
21. Steels L. The synthetic modeling of language origins. Evolution of Communication 1997, 1(1):1-34.
22. Sugita Y., Tani J. Learning semantic combinatoriality from the interaction between linguistic and behavioral processes. Adaptive Behavior 2005, 13(1):33-52.
23. Tani J., Ito M. Self-organization of behavioral primitives as multiple attractor dynamics: a robot experiment. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 2003, 33(4):481-488.
24. Weckerly, J., & Elman, J. L. (1992). A PDP approach to processing center-embedded sentences. In Fourteenth annual conference of the cognitive science society. Vol. 14. Routledge (pp. 414-419).
25. Yamashita Y., Tani J. Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Computational Biology 2008, 4.