Neural associative memory and the willshaw-palm probability distribution

SIAM Journal on Applied Mathematics

Volumn 69, Issue 1, 2008, Pages 169-196

  Knoblauch, Andreas ^a  

^a HONDA RESEARCH INSTITUTE EUROPE GMBH   (Germany)

Author keywords

Fault tolerance; Information retrieval; Neural network; Storage capacity; Willshaw model

Indexed keywords

ALGEBRA; ASSOCIATIVE PROCESSING; ASYMPTOTIC ANALYSIS; DATA STORAGE EQUIPMENT; DENDRIMERS; FAULT TOLERANCE; INFORMATION SERVICES; NUCLEAR MATERIALS SAFEGUARDS; POLYNOMIAL APPROXIMATION; PROBABILITY; PROBABILITY DISTRIBUTIONS; QUALITY ASSURANCE; RANDOM PROCESSES; RELIABILITY; VECTORS;

ASSOCIATIVE MEMORIES; BINOMIAL APPROXIMATIONS; CENTRAL MOMENTS; PALM DISTRIBUTIONS; PATTERN VECTORS; PERFORMANCE PARAMETERS; RANDOM PATTERNS; RETRIEVAL EFFICIENCIES; STORAGE CAPACITIES; STORAGE CAPACITY; STORED PATTERNS; SUBLINEAR; WILLSHAW MODEL;

NEURAL NETWORKS;

EID: 55149088346     PISSN: 00361399     EISSN: None     Source Type: Journal    
DOI: 10.1137/070700012     Document Type: Article

References (47)

1. D. J. AMIT, H. GUTFREUND, and H. SOMPOLINSKY, Information storage in neural networks with low levels of activity, Phys. Rev. A, 35 (1987), pp. 2293-2303.
2. D. J. AMIT, H. GUTFREUND, and H. SOMPOLINSKY, Statistical mechanics of neural networks near saturation, Ann. Phys., 173 (1987), pp. 30-67.
3. H. J. BENTZ, M. HAGSTROEM, and G. PALM, Information storage and effective data retrieval in sparse matrices, Neural Networks, 2 (1989), pp. 289-293.
4. H. BOSCH and F. KURFESS, Information storage capacity of incompletely connected associative memories, Neural Networks, 11 (1998), pp. 869-876.
5. V. BRAITENBERG, Cell assemblies in the cerebral cortex, in Theoretical Approaches to Complex Systems, R. Heim and G. Palm, eds., Lecture Notes in Biomath. 21, Springer-Verlag, Berlin, Heidelberg, New York, 1978, pp. 171-188.
6. J. T. BUCKINGHAM, Delicate Nets, Faint Recol lections: A Study of Partially Connected Associative Network Memories, Ph.D. thesis, University of Edinburgh, 1991.
7. J. T. BUCKINGHAM and D. J. WILLSHAW, Performance characteristics of associative nets, Network: Computation in Neural Systems, 3 (1992), pp. 407-414.
8. J. T. BUCKINGHAM and D. J. WILLSHAW, On setting unit thresholds in an incompletely connected associative net, Network: Computation in Neural Systems, 4 (1993), pp. 441-459.
9. T. M. COVER and J. A. THOMAS, Elements of Information Theory, Wiley, New York, 1991.
10. R. FAY, U. KAUFMANN, A. KNOBLAUCH, H. MARKERT, and G. PALM, Integrating object recognition, visual attention, language and action processing on a robot using a neurobiologically motivated associative architecture, in Proceedings of the NeuroRobotics Workshop at the 27th German GI Conference on Artificial Intelligence, University of Ulm, Germany, 2004.
11. W. GERSTNER and J. L. van HEMMEN, Associative memory in a network of "spiking" neurons, Network, 3 (1992), pp. 139-164.
12. B. GRAHAM and D. WILLSHAW, Improving recall from an associative memory, Biol. Cybernet., 72 (1995), pp. 337-346.
13. D. GREENE, M. PARNAS, and F. YAO, Multi-index hashing for information retrieval, in Proceedings of the 35th Annual Symposium on Foundations of Computer Science, IEEE Computer Society Press, 1994, pp. 722-731.
14. D. O. HEBB, The Organization of Behavior. A Neuropsychological Theory, Wiley, New York, 1949.
15. R. HECHT-NIELSEN, Confabulation Theory, Springer-Verlag, Heidelberg, 2007.
16. J. J. HOPFIELD, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA, 79 (1982), pp. 2554-2558.
17. P. KANERVA, Sparse Distributed Memory, MIT Press, Cambridge, MA, 1988.
18. A. KNOBLAUCH, Optimal matrix compression yields storage capacity 1 for binary Willshaw associative memory, in Artificial Neural Networks and Neural Information Processing-ICANN/ICONIP 2003.
19. O. KAYNAK, E. ALPAYDIN, E. OJA, and L. XU, eds., Lecture Notes in Comput. Sci. 2714, Springer-Verlag, Berlin, 2003, pp. 325-332.
20. A. KNOBLAUCH, Synchronization and Pattern Separation in Spiking Associative Memory and Visual Cortical Areas, Ph.D. thesis, Department of Neural Information Processing, University of Ulm, Germany, 2003.
21. A. KNOBLAUCH, Neural associative memory for brain modeling and information retrieval, Inform. Process. Lett., 95 (2005), pp. 537-544.
22. A. KNOBLAUCH, Statistical implications of clipped Hebbian learning of cell assemblies, Neuro-comput., 65-66 (2005), pp. 647-652.
23. A. KNOBLAUCH, On Compressing the Memory Structures of Binary Neural Associative Networks, Internal Report HRI-EU 06-02, Honda Research Institute Europe, Offenbach/Main, Germany, 2006.
24. A. KNOBLAUCH, Asymptotic Conditions for High-Capacity Neural Associative Networks, Internal Report HRI-EU 07-02, Honda Research Institute Europe, Offenbach/Main, Germany, 2007.
25. A. KNOBLAUCH, On the Computational Benefits of Inhibitory Neural Associative Networks, Internal Report HRI-EU 07-05, Honda Research Institute Europe, Offenbach/Main, Germany, 2007.
26. A. KNOBLAUCH, Closed-form expressions for the moments of the binomial probability distribution, SIAM J. Appl. Math., 69 (2008), pp. 197-204.
27. A. KNOBLAUCH and G. PALM, Pattern separation and synchronization in spiking associative memories and visual areas, Neural Networks, 14 (2001), pp. 763-780.
28. A. KNOBLAUCH and G. PALM, Scene segmentation by spike synchronization in reciprocally connected visual areas. II. Global assemblies and synchronization on larger space and time scales, Biol. Cybernet., 87 (2002), pp. 168-184.
29. T. KOHONEN, Associative Memory: A System Theoretic Approach, Springer-Verlag, Berlin, 1977.
30. P. E. LATHAM and S. NIRENBERG, Computing and stability in cortical networks, Neural Com-put., 16 (2004), pp. 1385-1412.
31. D. MARR, Simple memory: A theory for archicortex, Philos. Trans. Roy. Soc. London Ser. B, 262 (1971), pp. 24-81.
32. M. L. MINSKY and S. PAPERT, Perceptrons: An Introduction to Computational Geometry, MIT Press, Cambridge, MA, 1969.
33. X. MU, M. ARTIKLAR, P. WATTA, and M. H. HASSOUN, An RCE-based associative memory with application to human face recognition, Neural Process. Lett., 23 (2006), pp. 257-271.
34. J.-P. NADAL, Associative Memory: On the (puzzling) sparse coding limit, J. Phys. A, 24 (1991), pp. 1093-1101.
35. G. PALM, On associative memories, Biol. Cybernet., 36 (1980), pp. 19-31.
36. G. PALM, Neural Assemblies. An Alternative Approach to Artificial Intelligence, Springer-Verlag, Berlin, 1982.
37. G. PALM, Computing with neural networks, Science, 235 (1987), pp. 1227-1228.
38. G. PALM, Memory capacities of local rules for synaptic modification. A comparative review, Concepts in Neurosci., 2 (1991), pp. 97-128.
39. G. PALM and F. SOMMER, Associative data storage and retrieval in neural nets, in Models of Neural Networks III, E. Domany, J. L. van Hemmen, and K. Schulten, eds., Springer-Verlag, New York, 1996, pp. 79-118.
40. A. PAPOULIS, Probability, Random Variables, and Stochastic Processes, 3rd ed., McGraw-Hill, New York, 1991.
41. R. W. PRAGER and F. FALLSIDE, The modified Kanerva model for automatic speech recognition, Computer Speech and Language, 3 (1989), pp. 61-81.
42. D. A. RACHKOVSKIJ and E. M. KUSSUL, Binding and normalization of binary sparse distributed representations by context-dependent thinning, Neural Comput., 13 (2001), pp. 411-452.
43. M. REHN and F. T. SOMMER, Storing and restoring visual input with collaborative rank coding and associative memory, Neurocomput., 69 (2006), pp. 1219-1223.
44. F. T. SOMMER and G. PALM, Improved bidirectional retrieval of sparse patterns stored by Hebbian learning, Neural Networks, 12 (1999), pp. 281-297.
45. K. STEINBUCH, Die Lernmatrix, Kybernetik, 1 (1961), pp. 36-45.
46. H. WERSING and E. KÖRNER, Learning optimized features for hierarchical models of invariant object recognition, Neural Comput., 15 (2003), pp. 1559-1588.
47. D. J. WILLSHAW, O. P. BUNEMAN, and H. C. LONGUET-HIGGINS, Non-holographic associative memory, Nature, 222 (1969), pp. 960-962.