Understanding state space organization in recurrent neural networks with iterative function systems dynamics

Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

Volumn 1778, Issue , 2000, Pages 255-269

  Tino, Peter ^a,b   Dorffner, Georg ^a,c   Schittenkopf, Christian ^a  

^a AUSTRIAN RESEARCH INSTITUTE FOR ARTIFICIAL INTELLIGENCE   (Austria)

^b SLOVAK UNIVERSITY OF TECHNOLOGY   (Slovakia)

^c UNIVERSITY OF VIENNA   (Austria)

Author keywords

[No Author keywords available]

Indexed keywords

DNA SEQUENCES; LOGIC CIRCUITS; MARKOV PROCESSES; NETWORK ARCHITECTURE;

CHAOS GAME REPRESENTATION; GENERALIZED DIMENSION; INPUT SEQUENCE; ITERATIVE FUNCTION SYSTEMS; MEMORY LENGTH; NETWORK STATE; STATISTICAL PROPERTIES; STATISTICAL STRUCTURES;

RECURRENT NEURAL NETWORKS;

EID: 84942856897     PISSN: 03029743     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1007/10719871_18     Document Type: Conference Paper

References (28)

1. M.F. Barnsley. Fractals everywhere. Academic Press, New York, 1988.
2. C. Beck and F. Schlogl. Thermodynamics of chaotic systems. Cambridge University Press, Cambridge, UK, 1995.
3. J. Bruske and G. Sommer. Dynamic cell structure learns perfectly topology preserving map. Neural Computation, 7(4):845-865, 1995.
4. M.P. Casey. The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation, 8(6):1135-1178, 1996.
5. J.P. Crutchfield and K. Young. Inferring statistical complexity. Physical Review Letters, 63:105-108, July 1989.
6. J.P. Crutchfield and K. Young. Computation at the onset of chaos. In W.H. Zurek, editor, Complexity, Entropy, and the physics of Information, SFI Studies in the Sciences of Complexity, vol 8, pages 223-269. Addison-Wesley, Reading, Massachusetts, 1990.
7. P. Frasconi, M. Gori, M. Maggini, and G. Soda. Insertion of finite state automata in recurrent radial basis function networks. Machine Learning, 23:5-32, 1996.
8. J. Freund, W. Ebeling, and K. Rateitschak. Self-similar sequences and universal scaling of dynamical entropies. Physical Review E, 54(5):5561-5566, 1996.
9. P. Grassberger. Information and complexity measures in dynamical systems. In H. Atmanspacher and H. Scheingraber, editors, Information Dynamics, pages 15-33. Plenum Press, New York, 1991.
10. J. Hertz, A. Krogh, and R.G. Palmer. Introduction to the Theory of Neural Computation. Addison-Wesley, Redwood City, CA, 1991.
11. J. Jeffrey. Chaos game representation of gene structure. Nucleic Acids Research, 18(8):2163-2170, 1990.
12. R. Kenyon and Y. Peres. Measures of full dimension on affine invariant sets. Ergodic Theory and Dynamical Systems, 16:307-323, 1996.
13. J.F. Kolen. Recurrent networks: state machines or iterated function systems? In M.C. Mozer, P. Smolensky, D.S. Touretzky, J.L. Elman, and A.S. Weigend, editors, Proceedings of the 1993 Connectionist Models Summer School, pages 203-210. Erlbaum Associates, Hillsdale, NJ, 1994.
14. P. Manolios and R. Fanelli. First order recurrent neural networks and deterministic finite state automata. Neural Computation, 6(6):1155-1173, 1994.
15. J.L. McCauley. Chaos, Dynamics and Fractals: an algorithmic approach to deterministic chaos. Cambridge University Press, 1994.
16. P. Tino. Spatial representation of symbolic sequences through iterative function system. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans, 29(4):386-393, 1999.
17. P. Tino and G. Dorffner. Recurrent neural networks with iterated function systems dynamics. In International ICSC/IFAC Symposium on Neural Computation, pages 526-532, 1998.
18. P. Tino, B.G. Horne, C.L. Giles, and P.C. Collingwood. Finite state machines and recurrent neural networks-automata and dynamical systems approaches. In J.E. Dayhoff and O. Omidvar, editors, Neural Networks and Pattern Recognition, pages 171-220. Academic Press, 1998.
19. P. Tino and M. Koteles. Extracting finite state representations from recurrent neural networks trained on chaotic symbolic sequences. IEEE Transactions on Neural Networks, 10(2):284-302, 1999.
20. P. Tino and J. Sajda. Learning and extracting initial mealy machines with a modular neural network model. Neural Computation, 7(4):822-844, 1995.
21. P. Tino and V. Vojtek. Modeling complex sequences with recurrent neural networks. In G.D. Smith, N.C. Steele, and R.F. Albrecht, editors, Artificial Neural Networks and Genetic Algorithms, pages 459-463. Springer Verlag Wien New York, 1998.
22. J.L. Oliver, P. Bernaola-Galvan, J. Guerrero-Garcia, and R. Roman Roldan. En-tropic profiles of dna sequences through chaos-game-derived images. Journal of Theor. Biology, (160):457-470, 1993.
23. C.W. Omlin and C.L. Giles. Extraction of rules from discrete-time recurrent neural networks. Neural Networks, 9(1):41-51, 1996.
24. A. Renyi. On the dimension and entropy of probability distributions. Acta Math. Hung., (10):193, 1959.
25. R. Roman-Roldan, P. Bernaola-Galvan, and J.L. Oliver. Entropic feature for sequence pattern through iteration function systems. Pattern Recognition Letters, 15:567-573, 1994.
26. D. Ron, Y. Singer, and N. Tishby. The power of amnesia. In Advances in Neural Information Processing Systems 6, pages 176-183. Morgan Kaufmann, 1994.
27. D. Ron, Y. Singer, and N. Tishby. The power of amnesia. Machine Learning, 25:117-150, 1996.
28. W. Tabor. Dynamical automata. Technical Report TR98-1694, Cornell University, Computer Science Department, 1998.