Effective learning in recurrent max-min neural networks

Neural Networks

Volumn 11, Issue 3, 1998, Pages 535-547

  Teow, Loo Nin ^a   Loe, Kia Fock ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Backpropagation; DFA extraction; Gradient descent; Grammatical inference; Learning; Max min functions; Neural network; Recurrent architecture

Indexed keywords

BACKPROPAGATION; COMPUTER ARCHITECTURE; CONVERGENCE OF NUMERICAL METHODS; FINITE AUTOMATA; FOURIER TRANSFORMS; FUNCTIONS; LEARNING SYSTEMS; MATHEMATICAL MODELS;

DETERMINISTIC FINITE AUTOMATON; FOURIER CONVERGENCE ANALYSIS; GRAMMATICAL INFERENCE; RECURRENT ARCHITECTURE;

RECURRENT NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; FOURIER ANALYSIS; LEARNING; MATHEMATICAL MODEL; PRIORITY JOURNAL;

EID: 0032052449     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(97)00151-2     Document Type: Article

References (30)

1. Aleksander, I. & Morton, H.B. (1993). Neurons and symbols. London: Chapman and Hall.
2. Bengio Y., Simard P., Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks. 5(2):1994;157-166.
3. Buckley J.J., Hayashi Y. Fuzzy neural networks: a survey. Fuzzy Sets and Systems. 66:1994;1-13.
4. Carroll, J. & Long, D. (1989). Theory of finite automata - with an introduction to formal languages. London: Prentice-Hall.
5. Casey M. The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation. 8(6):1996;1135-1178.
6. Cleeremans A., Servan-Schreiber D., McClelland J. Finite state automata and simple recurrent neural networks. Neural Computations. 1(3):1989;372-381.
7. Das, S. & Mozer, M. (1994). A unified gradient-descent/clustering architecture for finite state machine induction. In J. Cowan, G. Tesauro & J. Alspector (Eds.), Advances in neural information processing systems 6, pp. 19-26. San Mateo, CA: Morgan Kaufmann.
8. Forcada M.L., Carrasco R.C. Learning the initial state of a second-order recurrent neural network during regular-language inference. Neural Computation. 7:1995;923-930.
9. Fulks, W. (1981). Advanced calculus - an introduction to analysis, 3rd ed. New York: Wiley Trans-Edition.
10. Giles C.L., Horne B.G., Lin T. Learning a class of large finite state machines with a recurrent neural network. Neural Networks. 8(9):1995;1359-1365.
11. Gupta M.N., Rao D.H. On the principles of fuzzy neural networks. Fuzzy Sets and Systems. 61:1994;1-18.
12. Ishibuchi H., Fujioka R., Tanaka H. Neural networks that learn from fuzzy IF-THEN rules. IEEE Transactions on Fuzzy Systems. 1(2):1993;85-97.
13. Keller J.M., Yager R.R., Tahani H. Neural network implementation of fuzzy logic. Fuzzy Sets and Systems. 45:1992;1-12.
14. Khan, E. & Unal, F. (1995). Recurrent fuzzy logic using neural networks. In T. Furuhashi (Ed.), Advances in fuzzy logic, neural networks, and genetic algorithms - lecture notes in artificial intelligence. Berlin: Springer.
15. Klir, G.J. & Folger, T.A. (1988). Fuzzy sets, uncertainty, and information. London: Prentice-Hall.
16. Lang, K. (1992). Random DFAs can be approximately learned from sparse uniform examples. In D. Haussler (Ed.), Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory (pp. 45-52). New York: ACM Press.
17. Miller, S. & Giles, C.L. (1993). Experimental comparison of the effect of order in recurrent neural networks. In I. Guyon, and P.S.P. Wang (Eds.), Advances in pattern recognition systems using neural network technologies (pp. 205-228). Singapore: World Scientific.
18. Mitra S., Pal S.K. Logical operation based fuzzy MLP for classification and rule generation. Neural Networks. 7(2):1994;353-373.
19. Omlin C.W., Giles C.L. Extraction of rules from discrete-time recurrent neural network. Neural Networks. 9(1):1995;41-52.
20. Omlin C.W., Giles C.L. Constructing deterministic finite-state automata in recurrent neural networks. Journal of the ACM. 43(6):1996;937-972.
21. Omlin, C.W., Thornber, K.K. & Giles, C.L. (1996). Representation of fuzzy finite state automata in continuous recurrent neural networks. In B.J. Shen (Ed.), Proceedings of the IEEE International Conference on Neural networks (pp. 1023-1028). Piscataway, NJ: IEEE Press.
22. Omlin C.W., Thornber K.K., & Giles C.L. (1998). Fuzzy finite-state automata can be deterministically encoded into recurrent neural networks. IEEE Transactions on Fuzzy Systems (to be published).
23. Pearlmutter B.A. Learning state space trajectories in recurrent neural networks. Neural Computation. 1:1989;263-269.
24. Pedrycz W. Fuzzy neural networks and neurocomputations. Fuzzy Sets and Systems. 56:1993;1-28.
25. Rumelhart, D.E., Hinton, G.E. & Williams, R.J. (1986). Learning internal representations by error propagation. In D.E. Rumelhart & J.L. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations, pp. 318-362. Cambridge, MA: MIT Press.
26. Simpson P.K. Fuzzy min-max neural networks - Part 1: classifications. IEEE Transactions on Neural Networks. 3(5):1992;776-786.
27. Simpson P.K. Fuzzy min-max neural networks - Part 2: clustering. IEEE Transactions on Neural Networks. 4(1):1993;32-45.
28. Teow, L. & Loe, K. (1997). An effective learning method for max-min neural networks. In M.E. Pollack (Ed.), Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI-97), Vol. 2, pp. 1134-1139. Denver, CO: Professional Book Center.
29. Williams R., Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Computation. 1:1989;270-280.
30. Zeng Z., Goodman R.M., Smyth P. Learning finite state machines with self-clustering recurrent networks. Neural Computation. 5:1993;976-990.