Mutual information of sparsely coded associative memory with self-control and ternary neurons

Neural Networks

Volumn 13, Issue 4-5, 2000, Pages 455-462

  Bolle D ^a   Dominguez, D R C ^b,c   Amari, S ^d  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b UNIVERSIDAD REY JUAN CARLOS   (Spain)

^c INSTITUTO NACIONAL DE TÉCNICA AEROESPACIAL   (Spain)

^d RIKEN   (Japan)

Author keywords

Associative memory; Basin of attraction; Mutual information; Self control dynamics; Sparse coding; Storage capacity; Ternary neurons

Indexed keywords

ENCODING (SYMBOLS); MATHEMATICAL MODELS;

ASSOCIATIVE MEMORY;

NEURAL NETWORKS;

ASSOCIATIVE MEMORY; DYNAMICS; ENTROPY; MATHEMATICS; NERVE CELL; NERVE CELL NETWORK; NERVE CONDUCTION; PRIORITY JOURNAL; SELF CONTROL; SHORT SURVEY; SIMULATION;

EID: 0342756744     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(00)00027-7     Document Type: Article

References (29)

1. Amari S. Neural theory and association of concept information. Biological Cybernetics. 26:1977;175-185.
2. Amari S. Characteristics of sparsely encoded associative memeory. Neural Networks. 2:1989;451-457.
3. Amari S., Maginu K. Statistical neurodynamics of associative memory. Neural Networks. 1:1988;63-73.
4. Amit D., Gutfreund H., Sompolinsky H. Information storage in neural networks with low levels of activity. Physical Review A. 35:1987;2293-2303.
5. Arenzon J.J., Lemke N. Simulating highly diluted neural networks. Journal of Physics A. 27:1994;5161-5165.
6. Blahut R.E. Principles and practice of information theory. 1990;Addison-Wesley, Reading, MA.
7. Bollé D., Vinck B., Zagrebnov V.A. On the parallel dynamics of the Q-state Potts and Q-Ising neural networks. Journal of Statistical Physics. 70:1993;1099-1119.
8. Bollé D., Shim G.M., Vinck B., Zagrebnov V.A. Retrieval and chaos in extremely diluted Q-Ising neural networks. Journal of Statistical Physics. 74:1994;565-582.
9. Bouten M., Engel A. Basin of attraction in networks of multistate neurons. Physical Review A. 47:1993;1397-1400.
10. Buhmann J., Divko R., Schulten K. Associative memory with high information content. Physical Review A. 39:1989;2689-2692.
11. Derrida B., Gardner E., Zippelius A. An exactly solvable asymmetric neural network model. Europhysics Letters. 4:1987;167-173.
12. Dominguez D.R.C., Bollé D. Self-control in sparsely coded networks. Physical Review Letters. 80:1998;2961-2964.
13. Hebb D.O. The organization of behavior. 1949;Wiley, New York.
14. Hertz J., Krogh A., Palmer R.G. Introduction to the theory of neural computation. 1991;Addison-Wesley, Redwood City.
15. Horn D., Usher M. Neural networks with dynamical thresholds. Physical Review A. 40:1989;1036-1044.
16. Horner H. Neural networks with low levels of activity: Ising vs. McCulloch-Pitts neurons. Zeitschrift fur Physik B. 75:1989;133-136.
17. Kitano K., Aoyagi T. Retrieval dynamics of neural networks for sparsely coded sequential patterns. Journal of Physics A. 31:1998;L613-L620.
18. Nadal J.-P., Brunel N., Parga N. Nonlinear feedforward networks with stochastic outputs: infomax implies redundancy reduction. Network: Computation in Neural Systems. 9:1998;207-217.
19. Okada M. A hierarchy of macrodynamical equations for associative memory. Neural Networks. 8:1995;833-838.
20. Okada M. Notions of associative memory and sparse coding. Neural Networks. 9:1996;1429-1458.
21. Perez-Vicente C.J. Sparse coding and information in Hebbian neural networks. Europhysics Letters. 10:1989;621-625.
22. Rieger H. Storing an extensive number of grey-toned patterns in a neural network using multi-state neurons. Journal of Physics A. 23:1990;L1273-L1279.
23. Schultz S., Treves A. Stability of the replica-symmetric solution for the information conveyed by a neural network. Physical Review E. 57:1998;3302-3310.
24. Shannon C.E. A mathematical theory for communication. Bell Systems Technical Journal. 27:1948;379.
25. Shim G.M., Wong K.Y.M., Bollé D. Dynamics of temporal activity in multi-state neural networks. Journal of Physics A. 30:1997;2637-2652.
26. Schwenker F., Sommer F.T., Palm G. Iterative retrieval of sparsely coded associative memory patterns. Neural Networks. 9:1996;445-455.
27. Tsodyks M.V. Associative memory in asymmetric diluted network with low level of activity. Europhysics Letters. 7:1988;203-208.
28. Willshaw D.J., Buneman O.P., Longuet-Higgins H.C. Non-holographic associative memory. Nature. 222:1969;960-962.
29. Yedidia J.S. Neural networks that use three-state neurons. Journal of Physics A. 22:1989;2265-2273.