Storage capacity diverges with synaptic efficiency in an associative memory model with synaptic delay and pruning

IEEE Transactions on Neural Networks

Volumn 15, Issue 5, 2004, Pages 1215-1227

  Miyoshi, Seiji ^a   Okada, Masato ^b,c  

^a UNIVERSITY OF TOKYO   (Japan)

^b RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^c JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; DATA STORAGE EQUIPMENT; DIFFERENTIAL EQUATIONS; DISCRETE FOURIER TRANSFORMS; MATHEMATICAL MODELS; STATISTICAL METHODS;

ASSOCIATIVE MEMORY; MACRODYNAMICAL EQUATION; STATISTICAL NEURODYNAMICS; SYNAPTIC DELAY; SYNAPTIC PRUNING;

NEURAL NETWORKS;

ACTION POTENTIAL; ANIMAL; ARTICLE; ARTIFICIAL NEURAL NETWORK; ASSOCIATION; BIOLOGICAL MODEL; BRAIN; HUMAN; LEARNING; MEMORY; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NERVE TRACT; NONLINEAR SYSTEM; PHYSIOLOGY; REACTION TIME; SYNAPSE; SYNAPTIC TRANSMISSION; TIME;

ACTION POTENTIALS; ANIMALS; ASSOCIATION; BRAIN; HUMANS; LEARNING; MEMORY; MODELS, NEUROLOGICAL; NERVE NET; NEURAL NETWORKS (COMPUTER); NEURAL PATHWAYS; NEURONAL PLASTICITY; NONLINEAR DYNAMICS; REACTION TIME; SYNAPSES; SYNAPTIC TRANSMISSION; TIME FACTORS;

EID: 4644324648     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.832711     Document Type: Article

References (34)

1. G. Palm, "On associative memory," Biol. Cybern.., vol. 36, pp. 19-31, 1980.
2. G. Palm, "On the storage capacity of an associative memory with randomly distributed storage elements," Biol. Cybern., vol. 39, pp. 125-127, 1981.
3. M. Okada, "Notions of associative memory and sparse coding," Neural Netw., vol. 9, pp. 1429-1458, 1996.
4. P. Dayan and D. J. Willshaw, "Optimising synaptic learning rules in linear associative memories," Biol. Cybern., vol. 65, pp. 253-265, 1991.
5. F. Schwenker, F. T. Sommer, and G. Palm, "Iterative retrieval of sparsely coded associative memory patterns," Neural Netw., vol. 9, pp. 445-455, 1996.
6. H. Sompolinsky, "Neural networks with nonlinear synapses and a static noise," Phys. Rev. A, vol. 34, pp. 2571-2574, 1986.
7. H. Sompolinsky, "The theory of neural networks: The Hebb rule and beyond," in Heidelberg Colloquium on Glassy Dynamics, J. L. van Hemmen and I. Morgenstern, Eds. New York: Springer-Verlag, 1986, pp. 485-527.
8. W. G. Gibson and J. Robinson, "Statistical analysis of the dynamics of a sparse associative memory," Neural Netw., vol. 5, pp. 645-661, 1992.
9. M. Okada, T. Fukai, and M. Shiino, "Random and systematic dilutions of synaptic connections in a neural network with a nonmonotonic response function," Phys. Rev. E, vol. 57, pp. 2095-2103, 1998.
10. K. Mimura, T. Kimoto, and M. Okada, "Synapse efficiency diverges due to synaptic pruning following over-growth," Phys. Rev. E, vol. 68, p. 031 910, 2003.
11. G. Chechik, I. Meilijson, and E. Ruppin, "Synaptic pruning in development: A computational account," Neural Computat., vol. 10, pp. 1759-1777, 1998.
12. D. Horn, N. Levy, and E. Ruppin, "Memory maintenance via neuronal regulation," Neural Computat., vol. 10, pp. 1-18, 1998.
13. P. R. Huttenlocker, "Synaptic density in human frontal cortex. Development changes and effects of age," Brain Res., vol. 163, pp. 195-205, 1979.
14. P. R. Huttenlocker, C. De Courten, L. J. Garey, and H. Van der Loos, "Synaptogenesis in human visual cortex - Evidence for synapse elimination during normal development," Neurosci. Lett., vol. 33, pp. 247-252, 1982.
15. J. P. Bourgeois and P. Rakic, "Changing of synaptic density in the primary visual cortex of the rhesus monkey from fetal to adult age," J. Neurosci., vol. 13, pp. 2801-2820, 1993.
16. J. Takacs and J. Hamori, "Development dynamics of Purkinje cells and dendritic spines in rat cerebellar cortex," J. Neurosci. Res., vol. 38, pp. 515-530, 1994.
17. G. M. Innocenti, "Exuberant development of connections and its possible permissive role in cortical evolution," Trends Neurosci., vol. 18, pp. 397-402, 1995.
18. M. F. Eckenhoff and P. Rakic, "A quantative analysis of synaptogenesis in the molecular layer of the dentate gyrus in the resus monkey," Developmental Brain Res., vol. 64, pp. 129-135, 1991.
19. P. Rakic, J. P. Bourgeois, and P. S. Goldman-Rakic, "P. S. synaptic development of the cerebral cortex: Implementation for learning, memory and mental illness," Prog. Brain Res., vol. 102, pp. 227-243, 1994.
20. M. P. Stryker, "Binocular impulse blockade prevents the formation of ocular dominance columns in cat visual cortex," J. Neurosci., vol. 6, pp. 2117-2133, 1986.
21. A. W. Roe, S. L. Pallas, J. O. Hahm, and M. Sur, "A map of visual space induced in primary auditory cortex," Science, vol. 250, pp. 818-820, 1990.
22. J. R. Wolff, R. Laskawi, W. B. Spatz, and M. Missler, "Structural dynamics of synapses and synaptic components," Behav. Brain Res., vol. 66, pp. 13-20, 1995.
23. M. Shiino and T. Fukai, "Self-consistent signal-to-noise analysis and its application to analogue neural networks with asymmetric connections," J. Phys. A, Math. Gen., vol. 25, pp. L375-L381, 1992.
24. K. Fukushima, "A model of associative memory in the brain," Kybernetik, vol. 12, pp. 58-63, 1973.
25. H. F. Yanai and E. S. Kim, "Dynamics of Neural Nets With Delay-Synapses," (in Japanese), Technical report of the Institute of Electronics, Information and Communication Engineers, vol. NC92-116, 1993.
26. S. Miyoshi and K. Nakayama, "A recurrent neural network with serial delay elements for memorizing limit cycles," in Proc. ICNN'95, Perth, Australia, 1995, pp. 1955-1960.
27. S. Amari and K. Maginu, "Statistical neurodynamics of associative memory," Neural Netw., vol. 1, pp. 63-73, 1988.
28. S. Miyoshi, H. F. Yanai, and M. Okada, "Associative memory by recurrent neural networks with delay elements," in Proc. 9th Int. Conf. Neural Information Processing, vol. 1, Singapore, 2002, pp. 70-74.
29. S. Miyoshi, "Associative memory by recurrent neural networks with delay elements," Neural Netw., vol. 17, pp. 53-61, 2004.
30. S. Miyoshi and M. Okada, "Associative Memory by Neural Networks With Delays and Pruning" (in Japanese), in Trans. Inst. Electron. Inf. Commun. Eng. A, 2002, vol. J85, pp. 124-133.
31. M. Okada, "A hierarchy of macrodynamical equations for associative memory," Neural Netw., vol. 8, no. 6, pp. 833-838, 1995.
32. M. Kawamura and M. Okada, "Transient dynamics for sequence processing neural network," Journal of Physics A: Mathematical and General, vol. 35, pp. 253-266, 2002.
33. A. Düring, A. C. C. Coolen, and D. Sherrington, "Phase diagram and storage capacity of sequence processing neural networks," J. Phys. A, Math. Gen., vol. 31, pp. 8607-8621, 1998.
34. S. Amari, "Statistical neurodynamics of various versions of correlation associative memory," in Proc. ICNN'88, San Diego, CA, 1988, pp. 633-640.