Effects of noise in training patterns on the memory capacity of the fully connected binary Hopfield neural network: Mean-Field theory and simulations

IEEE Transactions on Neural Networks

Volumn 9, Issue 4, 1998, Pages 697-704

  Wang, Lipo ^a  

^a DEAKIN UNIVERSITY   (Australia)

Author keywords

Associative memory; Hebbian learning; Hopfield; Mean field theory; Memory capacity; Neural network; Training noise

Indexed keywords

ASSOCIATIVE STORAGE; COMPUTER SIMULATION; LEARNING SYSTEMS; STORAGE ALLOCATION (COMPUTER);

FULLY CONNECTED BINARY HOPFIELD NEURAL NETWORKS; HEBBIAN LEARNING; MEAN FIELD THEORY;

NEURAL NETWORKS;

EID: 0032122765     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.701182     Document Type: Article

References (29)

1. S. Abe, "Global convergence and suppression of spurious states of the Hopfield neural networks," IEEE Trans. Circuits Syst. - II, vol. 40, pp. 246-257, Apr. 1993.
2. M. Abelles, Local Cortical Circuits. New York: Springer-Verlag, 1982, p. 21.
3. S.-I. Amari, "Learning patters and pattern sequences by self-organizing nets of threshold elements," IEEE Trans. Comput., vol. 21, p. 1197, 1972.
4. D. J. Amit, H. Gutfreund, and H. Sompolinsky, "Statistical mechanics of neural networks near saturation," Ann. Phys., vol. 173, pp. 30-67, 1987.
5. B. Derrida, E. Gardner, and A. Zippelius, "An exactly solvable asymmetric neural-network model," Europhys. Lett., vol. 4, no. 2, pp. 167-173, July 1987.
6. J. E. Freund, Mathematical Statistics, 5th ed. Englewood Cliffs, NJ: Prentice-Hall, 1992, p. 295.
7. T. Geszti, Physical Models of Neural Networks. Singapore: World Scientific, 1990.
8. D. O. Hebb, The Organization of Behavior. New York: Wiley, 1949, p. 62.
9. J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," in Proc. Nat. Academy Sci. USA, vol. 79, 1982, pp. 2554-2558.
10. _, "Neurons with graded response have collective computational properties like those of two-state neurons," in Proc. Nat. Academy Sci. USA, vol. 81, May 1984, pp. 3088-3092.
11. D. Kleinfeld, "Sequential state generation by model neural networks," in Proc. Nat. Academy Sci. USA, vol. 83, 1986, pp. 9469-9473.
12. B. Kosko, "Bidirectional associative memories," IEEE Trans. Syst., Man, Cybern., vol. 18, pp. 49-60, Jan./Feb. 1988.
13. W. A. Little, "The existence of persistent states in the brain," Math. Biosci., vol. 19, pp. 101-120, 1974.
14. W. S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Math. Biophys., vol. 5, pp. 115-133, 1943.
15. P. Peretto, "On learning rules and memory storage abilities of asymmetrical neural networks," J. Phys., France, vol. 49, pp. 711-726, 1988.
16. D. Schonfeld, "On the hysteresis and robustness of Hopfield neural networks," IEEE Trans. Circuits Syst. - II, vol. 40, pp. 745-748, Nov. 1993.
17. G. L. Shaw and R. Vasudevan, "Persistent states of neural networks and the random nature of synaptic transmission," Math. Biosci., vol. 21, pp. 207-217, 1974.
18. H. Sompolinsky and I. Kanter, "Temporal association in asymmetric neural networks," Phys. Rev. Lett., vol. 57, pp. 2861-2864, 1986.
19. L. Wang, "Processing spatio-temporal sequences with any static associative neural network," IEEE Trans. Circuit Syst. - II, vol. 5, May 1998.
20. _, "On the dynamics of discrete-time, continuous-state Hopfield neural networks," IEEE Trans. Circuits Syst. - II: Analog and Digital Signal Processing, vol. 5, May 1998.
21. L. Wang, "Noise injection into inputs in sparsely connected Hopfield and winner-take-all neural networks," IEEE Trans. Syst., Man, Cybern., vol. 27, pp. 868-870, October 1997.
22. L. Wang, "Discrete-time convergence theory and updating rules for neural networks with energy functions," IEEE Trans. Neural Networks, vol. 8, pp. 445-447, Mar. 1997.
23. _, "Oscillatory and chaotic dynamics in neural networks under varying operating conditions," IEEE Trans. Neural Networks, vol. 7, pp. 1382-1388, Nov. 1996.
24. _, "Suppressing chaos with hysteresis in a higher order neural network," IEEE Trans. Circuits Syst. - II, vol. 43, pp. 845-846, Dec. 1996.
25. L. Wang, E. E. Pichler, and J. Ross, "Oscillations and chaos in neural networks: an exactly solvable model," in Proc. Nat. Academy Sci. USA, vol. 87, Dec. 1990, pp. 9467-9471.
26. L. Wang and J. Ross, "Synchronous neural networks of nonlinear threshold elements with hysteresis," in Proc. Nat. Academy Sci. USA, vol. 87, Feb. 1990, pp. 988-992.
27. _, "Interactions of neural networks: models for distraction and concentration," in Proc. Nat. Academy Sci. USA, vol. 87, Sept. 1990, pp. 7110-7114.
28. _, "Chaos, multiplicity, crisis, and synchronicity in higher order neural networks," Phys. Rev. A, vol. 44, no. 4, pp. R2259-2262, Aug. 1991.
29. _, "Physical modeling of neural networks," in Methods in Neuroscience, Computers and Computations in the Neurosciences, vol. 10, P. M. Conn, Ed. San Diego, CA: Academic, 1992, pp. 549-567.