The Capacity of the Hopfield Associative Memory

IEEE Transactions on Information Theory

Volumn 33, Issue 4, 1987, Pages 461-482

  Mceliece, Robert J ^a   Posner, Edward C ^b   Rodemich, Eugene R ^a   Venkatesh, Santosh S ^a  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CODES, SYMBOLIC - ENCODING; INFORMATION THEORY - CHANNEL CAPACITY; PROBABILITY; SYSTEMS SCIENCE AND CYBERNETICS - NEURAL NETS;

EIGHT-NEURON NEURAL NETWORK; HAMMING DISTANCE; HOPFIELD ASSOCIATIVE MEMORY; QUANTIZED MEMORY CAPACITY;

DATA STORAGE, DIGITAL;

EID: 0023383370     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/TIT.1987.1057328     Document Type: Article

References (25)

1. J.J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Nat. Acad. Sci. USA, vol. 79, pp. 2554–2558, 1982.
2. S. Grossberg, Studies of Mind and Brain. Boston: Reidel, 1982.
3. S. Grossberg, The Adaptive Brain; Vol. I: Cognition, Learning, Reinforcement, and Rhythm; Vol. II: Vision, Speech, Language, and Motor Control. Amsterdam, The Netherlands: North-Holland, 1986.
4. G.E. Hinton and J.A. Anderson, eds., Parallel Models of Associative Memory, Hillsdale, NJ: Erlbaum, 1981.
5. F. Tanaka and S.F. Edwards, “Analytical theory of the ground state properties of a spin glass: I. Ising spin glass,” J. Phys. F. Metal Phys., vol. 10, pp. 2769–2778, 1980.
6. S. Wolfram, “Statistical mechanics of cellular automata,” Rev. Mod. Phys., vol. 55, pp. 601–644, 1983.
7. D.O. Hebb, The Organization of Behavior. New York: Wiley, 1949.
8. J.G. Eccles, The Neurophysiological Basis of Mind. Oxford: Clarendon, 1953.
9. K. Nakano, “Associatron-A model of associative memory,” IEEE Trans. Syst. Man, Cybern., vol. SMC-2, pp. 380–388, 1972.
10. T. Kohonen, Associative Memory: A System-Theoretic Approach. Berlin: Springer-Verlag, 1977.
11. S. Amari, “Neural theory of association and concept formation,” Biol. Cybern., vol. 26, pp. 175–185, 1977.
12. W.A. Little, “The existence of persistent states in the brain,” Math. Biosci., vol. 19, pp. 101–120, 1974.
13. G. Palm, “On associative memory,” Biol. Cybern., vol. 36, pp. 19–31, 1980.
14. J.J. Hopfield and D.W. Tank, “'Neural’ computation of decisions in optimization problems,” Biol. Cybern., vol. 52, pp. 141–152, 1985.
15. D.W. Tank and J.J. Hopfield, “Simple optimization networks: An A/D converter and a linear programming circuit,” IEEE Trans. Circuits Syst., vol. CAS-33, pp. 533–541, 1986.
16. W.A. Little and G.L. Shaw, “Analytic study of the memory storage capacity of a neural network,” Math. Biosci., vol. 39, pp. 281–290, 1978.
17. Y.S. Abu-Mostafa and J. St. Jacques, “Information capacity of the Hopfield Model,” IEEE Trans. Inform. Theory vol. IT-31, pp. 461–464, 1985.
18. S.S. Venkatesh, “Epsilon capacity of neural networks,” to be published.
19. S.S. Vankatesh and D. Psaltis, “Information storage and retrieval in two associative nets,” submitted to IEEE Trans. Inform. Theory.
20. W. Feller, An Introduction to Probability Theory and Its Applications, vol. I, 3rd ed. New York: Wiley, 1968.
21. R.J. McEliece, The Theory of Information and Coding, vol. 3 of Encyclopedia of Mathematics and Its Application. Reading, MA: Addison-Wesley, 1977.
22. J.M. Wozencraft and I.M. Jacobs, Principles of Communication Engineering. New York: Wiley, 1965.
23. E.C. Posner and E.R. Rodemich, “Linear capacity in the Hopfield model,” to be published.
24. D.J. Gross and M. Mezard, “The simplest spin glass,” Nuclear Phys., vol. B 240 [FS12], pp. 431–452, 1984.
25. R.J. McEliece and E.C. Posner, “The number of stable points of an infinite-range spin glass memory,” Telecommunications and Data Acquisition Progress Report, vol. 42-83, July-Sept. 1985, Jet Propulsion Lab., California Inst. Technol. Pasadena, Nov. 15, 1985, pp. 209–215.