Analysis and optimal design of continuous neural networks with applications to associative memory

Neural Networks

Volumn 12, Issue 2, 1999, Pages 259-271

  Zhenjiang, Miao ^a   Baozong, Yuan ^b  

^a NATIONAL RESEARCH COUNCIL CANADA   (Canada)

^b BEIJING JIAOTONG UNIVERSITY   (China)

Author keywords

Artificial intelligence; Associative memory; Neural computing; Neural networks; Optimal design; Pattern recognition; Stability; System analysis

Indexed keywords

ARTIFICIAL INTELLIGENCE; ASSOCIATIVE STORAGE; ASYMPTOTIC STABILITY; PATTERN RECOGNITION; SYSTEMS ANALYSIS;

CONTINUOUS NEURAL NETWORKS;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; ASSOCIATIVE MEMORY; IMAGE ANALYSIS; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; MEMORY CONSOLIDATION; PATTERN RECOGNITION; PRIORITY JOURNAL; RECALL; THEORY;

EID: 0033105281     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(98)00118-X     Document Type: Article

References (66)

1. Amari, S. (1972). Learning patterns and pattern sequences by self-organizing nets of threshold elements. IEEE Trans. Computers, 21, 1197-1206.
2. Amari, S. (1977). Neural theory of association and concept formation. Biol. Cybern., 26, 175-185.
3. Amari, S. (1987). On mathematical methods in the theory of neural networks. In Proceedings of IEEE International Conference on Neural Networks (Vol. III. pp. 3-10), San Diego, CA.
4. Amari, S. (1988). Statistical neurodynamics of various versions of correlation associative memory. In Proceedings of IEEE International Conference on Neural Networks (Vol. I, pp. 633-640), San Diego, CA.
5. Amari, S. et al. (1997). Asymptotic statistical theory of overtraining and cross-validation. IEEE Trans. Neural Networks, 8 (5), 985-996.
6. Bertsekas, D. P. (1995). Nonlinear programming (pp. 253-415). Belmont, MA: Athena Scientific.
7. Cao, Y. J., & Wu, Q. H. (1996). A note on stability of analog neural networks with time delays. IEEE Trans. Neural Networks, 7 (6), 1533-1535.
8. Carpenter, G. A., & Grossberg, S. (1988). The ART of adaptive pattern recognition by a self-organizing neural network. Computer, March, 77-88.
9. Chua, C. O., & Yang, L. (1988). Cellular neural networks: applications. IEEE Trans. Circuits and Systems, 35 (10), 1273-1290.
10. Cohen, M. A. (1987). Sustained oscillations in symmetric cooperative-competitive neural networks. In Proc. of IEEE International Conference on Neural Networks (Vol. II, pp. 127-129), San Diego, CA.
11. Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Systems, Man, and Cybernetics, 13, 815-826.
12. Dorffner, G. (1989). Replacing symbolic rule system with PDP Networks-Netzsprech: a German example. Applied Artificial Intelligence, 3, 45-67.
13. Fang, Y., & Kincaid, T. G. (1996). Stability analysis of dynamical neural networks. IEEE Trans. Neural Networks, 7 (4), 996-1006.
14. Farrell, J. A., & Michel, A. N. (1990). A synthesis procedure for Hopfield's continuous-time associative memory. IEEE Trans. CAS, 37, 877-884.
15. Franklin, J. N. (1968). Matrix theory (pp. 105, 141-146, 161). Englewood Cliffs, NJ: Prentice-Hall.
16. Fukushima, K. (1988). A neural network for visual pattern recognition. Computer, March, 65-74.
17. Gibson, J. E. (1963). Nonlinear automatic control. New York: McGraw-Hill.
18. Golub, G. H., & Loan, C. F. V. (1996). Matrix computations (3rd ed., pp. 140, 310). Baltimore, MD: The Johns Hopkins University Press.
19. Gopalsamy, K., & Leung, I. K. C. (1997). Convergence under dynamical thresholds with delays. IEEE Trans. Neural Networks, 8 (2), 341-348.
20. Grossberg, S. (1969a). Embedding field: a theory of learning with physiological implications. J. Math. Psych., 6, 209-239.
21. Grossberg, S. (1969b). On learning and energy-entropy dependence in recurrent and nonrecurrent signed networks. J. Stat. Phys., 1, 319-350.
22. Grossberg, S. (1982). Studies of mind and brain: neural principles of learning, perception, development, cognition and motor control. Boston, MA: Reidel.
23. Guez, A. et al. (1988). On the stability, storage capacity, and design of nonlinear continuous neural networks. IEEE Trans. SMC, 18 (1), 80-87.
24. Hinton, G. E., & Anderson, J. A. (1981). Parallel models of an associative memory. Hillsdale, NJ: Erlbaum.
25. Hirsch, M. W. (1987). Convergence in neural nets. In Proceedings of IEEE International Conference on Neural Networks (Vol. II), San Diego, CA.
26. Hirsch, M. W. (1989). Convergent activation dynamics in continuous time networks. IEEE Trans. Neural Networks, 2, 331-349.
27. Hopfield, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA, 81, 3088-3092.
28. Ingwerson, D. R. (1961). A modified Lyapunov method for nonlinear stability analysis. IRE Trans. Automatic Control, AC6, 199.
29. Jin, L., & Gupta, M. M. (1996). Globally asymptotical stability of discrete-time analog neural networks. IEEE Trans. Neural Networks, 7 (4), 1024-1031.
30. Khotanzad, A., & Lu, J.-H. (1990). Classification of invariant image representations using a neural network. IEEE Trans. ASSP, 18 (6), 1028-1038.
31. Kohonen, T. (1984). Self-organization and associative memory. Berlin: Springer.
32. Kosko, B. (1987). Adaptive bidirectional associative memories. Applied Optics, 26, 4947-4960.
33. Kosko, B. (1988). Bidirectional associative memories. IEEE Trans. System, Man, and Cybernetics, 18 (1), 49-60.
34. Krasovskii, N. N. (1954). On the global stability of a system of nonlinear differential equations (in Russian). Applied Mathematics and Mechanics (Prikladnaja Matematika i Mekanika), 18, 735-737.
35. Krasovskii, N. N. (1963). Stability of motion. Stanford, CA: Stanford University Press.
36. Lin, J. H., Michel, A. N., & Porod, W. (1988). Qualitative analysis and synthesis of a class of neural networks. IEEE Trans. CAS, 35, 976-985.
37. Lin, J. H., Michel, A. N., & Porod, W. (1989). Analysis and synthesis of a class of neural networks: linear systems operating on a closed hypercube. IEEE Trans. CAS, 36, 1405-1422.
38. Lin, W.-C. et al. (1991). A hierarchical multiple-view approach to three dimensional object recognition. IEEE Trans. Neural Networks, 2 (1), 84-92.
39. Luenberger, D. G. (1984). Linear and nonlinear programming (2nd ed). Reading, MA: Addison-Wesley.
40. Meer, P. et al. (1989). Edge detection by associative mapping. Pattern Recognition, 22 (5), 491-503.
41. Miao, Z.-J., & Yuan, B.-Z. (1993a). An extended BAM neural network model. In Proceedings of IJCNN (Vol. 3, pp. 2682-2685), Nagoya, Japan.
42. Miao, Z.-J., & Yuan, B.-Z. (1993b). Loop neural network model for associative memory. In Proceedings of IEEE TENCON'93 (Vol. 2, pp. 758-762), Beijing, China.
43. Miao, Z.-J., & Yuan, B.-Z. (1993c). A new analytical method for AM neural networks and its application to pattern recognition. In Proceedings of 1993 IEEE International Conference on Neural Networks, San Francisco, CA.
44. Miao, Z.-J., & Yuan, B.-Z. (1993d). A NN image understanding system for maps and animals recognition. In Proceedings of IEEE TENCON'93 (Vol. 2, pp. 902-905), Beijing, China.
45. Miao, Z.-J., & Yuan, B.-Z. (1994). MIPAS: a neural-network-based multimedia information processing and analyzing system. Engineering Applications of Artificial Intelligence, 7 (6), 593-606.
46. Michel, A. N. et al. (1989). Qualitative analysis of neural networks. IEEE Trans. CAS, 36, 229-243.
47. Michel, A. N. et al. (1990a). Analysis and synthesis techniques for Hopfield type synchronous discrete time neural networks with application to associative memory. IEEE Trans. CAS, 37, 1355-1366.
48. Michel, A. N. et al. (1990b). Associative memories via artificial neural networks. IEEE Control Systems Magazine, April, 6-17.
49. Michel, A. N. et al. (1991). Analysis and synthesis of a class of discrete-time neural networks described on hypercubes. IEEE Trans. Neural Networks, 2, 32-46.
50. Ogata, K. (1970). Modern control engineering (pp. 735-736). Englewood Cliffs, NJ: Prentice-Hall.
51. Pang, M. (1985). The determination of the generalized diagonally dominant matrix and its applications. Chinese Annals of Mathematics, 6A (3), 323-330.
52. Polzleitner, W., & Wechsler, H. (1990). Selective and focused invariant recognition using distributed associative memories (DAM). IEEE Trans. Pattern Analysis and Machine Intelligence, 12 (8), 809-814.
53. Renardy, M., & Rogers, R. C. (1993). An introduction to partial differential equations (pp. 5-7). New York: Springer.
54. Schultz, D. G. (1965). The generation of Lyapunov functions. In C. T. Leondes (Ed.), Advances in control systems (pp. 1-64). New York: Academic Press.
55. Schultz, D. G., & Gibson, J. E. (1962). The variable gradient method for generating Lyapunov functions. IEEE Trans., 81 (II), 203-209.
56. Schurmann, B. (1989a). Generalized adaptive bidirectional associative memory. In Proceedings of IEEE International Conference on Neural Networks (Vol. I, pp. 91-97), Washington, DC.
57. Schurmann, B. (1989b). Stability and adaptation in artificial intelligence neural systems. Phys. Rev., A40, 2681-2688.
58. Sun, Y., & Yang, Z. (1987). Some necessary and sufficient conditions for generalized diagonal dominant matrix. Journal of Mathematics, 7 (2), 143-146.
59. Tanaka, K. (1996). An approach to stability criteria of neural-network control system. IEEE Trans. Neural Networks, 7 (3), 629-642.
60. Tonkin, S. P. et al. (1996). Stability of a class of directionally sensitive asymmetric nonlinear neural networks. Neural Networks, 9 (4), 555-565.
61. Tu, B., & Li, J. (1987). Distribution of the characteristic roots of matrix and their applications to the stability theory. Chinese Annals of Mathematics, 8A (6), 659-663.
62. Vincent, T. L., & Grantham, M. J. (1997). Nonlinear and optimal control systems (pp. 221-239). New York: Wiley.
63. Yoshizawa, S., Worita, M., Doya, K., & Amari, S. (1991). Analysis of dynamics of an associative neural network with non-monotonic output function. In T. Kohonen et al. (Ed.), Artificial neural networks (pp. 255-260). Amsterdam: North-Holland/Elsevier Science.
64. You, Z., & Li, L. (1989). Distribution of the eigenvalues for conjugate generalized diagonally dominant matrices. Journal of Mathematical Research and Exposition, 9 (2), 309-310.
65. Zhen, G. (1985). The applications of Lyapunov direct method in automatic control. Shanghai: Shanghai Science Publishing House.
66. Zhou, Y.-T. et al. (1988). Image restoration using neural network. IEEE Trans. Acoustic, Speech, and Signal Processing, 36 (7), 1141-1151.