Optimal synaptic learning in non-linear associative memory

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2010, Pages

  Knoblauch, Andreas ^a  

^a HONDA RESEARCH INSTITUTE EUROPE GMBH   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ASSOCIATIVE STORAGE; BAYESIAN NETWORKS; BRAIN; HOPFIELD NEURAL NETWORKS; NEURONS;

ASSOCIATIVE MEMORY; BAYESIAN MODELLING; HEBBIAN LEARNING; HOPFIELD MODELS; NEURAL ACTIVITY PATTERNS; NEURAL ASSOCIATIVE MEMORY; NON LINEAR; SINGLE-LAYER PERCEPTRONS; STORAGE CAPACITY; WILLSHAW MODEL;

ASSOCIATIVE PROCESSING;

EID: 79959456580     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2010.5596604     Document Type: Conference Paper

References (65)

1. D. Hebb, The organization of behavior. A neuropsychological theory. New York: Wiley, 1949.
2. V. Braitenberg, "Cell assemblies in the cerebral cortex." in Lecture notes in biomathematics (21). Theoretical approaches to complex systems., R. Heim and G. Palm, Eds. Berlin Heidelberg New York: Springer-Verlag, 1978, pp. 171-188.
3. G. Palm, Neural Assemblies. An Alternative Approach to Artificial Intelligence. Berlin: Springer, 1982.
4. -, "Cell assemblies as a guideline for brain research." Concepts in Neuroscience, vol. 1, pp. 133-148, 1990.
5. E. Fransen and A. Lansner, "A model of cortical associative memory based on a horizontal network of connected columns." Network: Computation in Neural Systems, vol. 9, pp. 235-264, 1998.
6. A. Lansner, "Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations." Trends in Neuro-sciences, vol. 32(3), pp. 178-186, 2009.
7. D. Marr, "Simple memory: a theory for archicortex." Philosophical Transactions of the Royal Society of London, Series B, vol. 262, pp. 24-81, 1971.
8. E. Rolls, "A theory of hippocampal function in memory." Hippocam- pus, vol. 6, pp. 601-620, 1996.
9. R. Bogacz, M. Brown, and C. Giraud-Carrier, "Model of familiarity discrimination in the perirhinal cortex." Journal of Computational Neuroscience, vol. 10, pp. 5-23, 2001.
10. D. Marr, "A theory of cerebellar cortex." Journal of Physiology, vol. 202(2), pp. 437-470, 1969.
11. J. Albus, "A theory of cerebellar function." Mathematical Biosciences, vol. 10, pp. 25-61, 1971.
12. P. Kanerva, Sparse Distributed Memory. Cambridge, MA: MIT Press, 1988.
13. G. Laurent, "Olfactory network dynamics and the coding of multidimensional signals." Nature Reviews Neuroscience, vol. 3, pp. 884-895, 2002.
14. T. Kohonen, Associative memory: a system theoretic approach. Berlin: Springer, 1977.
15. H. Bentz, M. Hagstroem, and G. Palm, "Information storage and effective data retrieval in sparse matrices." Neural Networks, vol. 2, pp. 289-293, 1989.
16. R. Prager and F. Fallside, "The modified Kanerva model for automatic speech recognition." Computer Speech and Language, vol. 3, pp. 61-81, 1989.
17. D. Greene, M. Parnas, and F. Yao, "Multi-index hashing for information retrieval." Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pp. 722-731, 1994.
18. A. Knoblauch, "Neural associative memory for brain modeling and information retrieval." Information Processing Letters, vol. 95, pp. 537-544, 2005.
19. X. Mu, M. Artiklar, P. Watta, and M. Hassoun, "An RCE-based associative memory with application to human face recognition." Neural Processing Letters, vol. 23, pp. 257-271, 2006.
20. A. Wichert, "Cell assemblies for diagnostic problem-solving." Neuro-computing, vol. 69, pp. 810-824, 2006.
21. M. Rehn and F. Sommer, "Storing and restoring visual input with collaborative rank coding and associative memory." Neurocomputing, vol. 69, pp. 1219-1223, 2006.
22. G. Palm, "Memory capacities of local rules for synaptic modification. A comparative review." Concepts in Neuroscience, vol. 2, pp. 97-128, 1991.
23. K. Steinbuch, "Die Lernmatrix." Kybernetik, vol. 1, pp. 36-45, 1961.
24. D. Willshaw, O. Buneman, and H. Longuet-Higgins, "Non-holographic associative memory." Nature, vol. 222, pp. 960-962, 1969.
25. G. Palm, "On associative memories." Biological Cybernetics, vol. 36, pp. 19-31, 1980.
26. J.-P. Nadal, "Associative memory: on the (puzzling) sparse coding limit." J.Phys. A: Math. Gen., vol. 24, pp. 1093-1101, 1991.
27. F. Sommer and P. Dayan, "Bayesian retrieval in associative memories with storage errors." IEEE Transactions on Neural Networks, vol. 9, pp. 705-713, 1998.
28. F. Sommer and G. Palm, "Improved bidirectional retrieval of sparse patterns stored by Hebbian learning." Neural Networks, vol. 12, pp. 281-297, 1999.
29. A. Knoblauch, G. Palm, and F. Sommer, "Memory capacities for synaptic and structural plasticity." Neural Computation, vol. 22(2), pp. 289-341, 2010.
30. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities." Proceedings of the National Academy of Science, USA, vol. 79, pp. 2554-2558, 1982.
31. G. Palm, "On the asymptotic information storage capacity of neural networks." in Neural Computers, ser. NATO ASI Series F41, R. Eckmiller and C. von der Malsburg, Eds. Berlin, Heidelberg, New York: Springer Verlag, 1988, pp. 271-280.
32. M. Tsodyks and M. Feigel'man, "The enhanced storage capacity in neural networks with low activity level." Europhysics Letters, vol. 6, pp. 101-105, 1988.
33. P. Dayan and D. Willshaw, "Optimising synaptic learning rules in linear associative memory." Biological Cybernetics, vol. 65, pp. 253-265, 1991.
34. P. Dayan and T. Sejnowski, "The variance of covariance rules for associative matrix memories and reinforcement learning." Neural Computation, vol. 5, pp. 205-209, 1993.
35. G. Palm and F. Sommer, "Associative data storage and retrieval in neural nets." in Models of Neural Networks III, E. Domany, J. van Hemmen, and K. Schulten, Eds. New York: Springer-Verlag, 1996, pp. 79-118.
36. G. Chechik, I. Meilijson, and E. Ruppin, "Effective neuronal learning with ineffective hebbian learning rules." Neural Computation, vol. 13, pp. 817-840, 2001.
37. D. Sterratt and D. Willshaw, "Inhomogeneities in heteroassociative memories with linear learning rules." Neural Computation, vol. 20, pp. 311-344, 2008.
38. A. Lansner and O. Ekeberg, "An associative network solving the "4-bit adder problem"." in Proceedings of the IEEE First International Conference on Neural Networks, M. Caudill and C. Butler, Eds., San Diego, CA, 1987, pp. II-549.
39. -, "A one-layer feedback artificial neural network with a Bayesian learning rule." International Journal of Neural Systems, vol. 1(1), pp. 77-87, 1989.
40. I. Kononenko, "Bayesian neural networks." Biological Cybernetics, vol. 61(5), pp. 361-370, 1989.
41. -, "On Bayesian neural networks." Informatica (Slovenia), vol. 18(2), pp. 183-195, 1994.
42. A. Lansner and A. Holst, "A higher order Bayesian neural network with spiking units." International Journal of Neural Systems, vol. 7(2), pp. 115-128, 1996.
43. A. Sandberg, A. Lansner, K. Petersson, and O. Ekeberg, "A palimpsest memory based on an incremental Bayesian learning rule." Neurocomputing, vol. 32-33, pp. 987-994, 2000.
44. A. Knoblauch, "Neural associative networks with optimal bayesian learning." Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, HRI-EU Report 09-02, May 2009.
45. S. Waydo, A. Kraskov, R. Quiroga, I. Fried, and C. Koch, "Sparse representation in the human medial temporal lobe." Journal of Neuroscience, vol. 26(40), pp. 10 232-10 234, 2006.
46. A. Knoblauch, "Comparison of the lansner/ekeberg rule to optimal bayesian learning in neural associative memory." Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, HRI-EU Report 10-06, April 2010.
47. -, "Neural associative memory with optimal bayesian learning." submitted, pp. -, 2010.
48. -, "On the computational benefits of inhibitory neural associative networks." Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, HRI-EU Report 07-05, May 2007.
49. A. Knoblauch and G. Palm, "Pattern separation and synchronization in spiking associative memories and visual areas." Neural Networks, vol. 14, pp. 763-780, 2001.
50. T. Sejnowski, "Storing covariance with nonlinearly interacting neurons." Journal of Mathematical Biology, vol. 4, pp. 303-321, 1977.
51. -, "Statistical constraints on synaptic plasticity." Journal of Theoretical Biology, vol. 69, pp. 385-389, 1977.
52. D. Willshaw and P. Dayan, "Optimal plasticity in matrix memories: what goes up must come down." Neural Computation, vol. 2, pp. 85-93, 1990.
53. G. Palm and F. Sommer, "Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states." Network, vol. 3, pp. 177-186, 1992.
54. B. Graham and D. Willshaw, "Improving recall from an associative memory." Biological Cybernetics, vol. 72, pp. 337-346, 1995.
55. H. Markram, M. Toledo-Rodriguez, Y. Wang, A. Gupta, G. Silberberg, and C. Wu, "Interneurons of the neocortical inhibitory system." Nature Reviews Neuroscience, vol. 5, pp. 793-807, 2004.
56. H. Zhang, "The optimality of naive bayes." in Proceedings of the 17th Florida Artificial Intelligence Research Society Conference, V. Barr and Z. Markov, Eds. AAAI Press, 2004, pp. 562-567.
57. P. Domingos and M. Pazzani, "On the optimality of the simple Bayesian classifier under zero-one loss." Machine Learning, vol. 29, pp. 103-130, 1997.
58. G. Palm, "Local synaptic rules with maximal information storage capacity." in Neural and synergetic computers, ser. Springer Series in Synergetics, H. Haken, Ed. Berlin, Heidelberg, New York: Springer Verlag, 1988, vol. 42, pp. 100-110.
59. D. Golomb, N. Rubin, and H. Sompolinsky, "Willshaw model: Associative memory with sparse coding and low firing rates." Phys. Rev. A, vol. 41, pp. 1843-1854, 1990.
60. A. Holtmaat and K. Svoboda, "Experience-dependent structural synaptic plasticity in the mammalian brain." Nature Reviews Neuroscience, vol. 10, pp. 647-658, 2009.
61. A. Knoblauch, "Synchronization and pattern separation in spiking associative memory and visual cortical areas." PhD thesis, Department of Neural Information Processing, University of Ulm, Germany, 2003.
62. -, "On compressing the memory structures of binary neural associative networks," Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, HRI-EU Report 06-02, April 2006.
63. -, "Neural associative memory and the Willshaw-Palm probability distribution." SIAM Journal on Applied Mathematics, vol. 69(1), pp. 169-196, 2008.
64. -, "The role of structural plasticity and synaptic consolidation for memory and amnesia in a model of cortico-hippocampal interplay." in Connectionist Models of Behavior and Cognition II: Proceedings of the 11th Neural Computation and Psychology Workshop, J. Mayor, N. Ruh, and K. Plunkett, Eds. Singapore: World Scientific Publishing, 2009, pp. 79-90.
65. -, "Zip nets: Efficient associative computation with binary synapses." in Proceedings of the International Joint Conference on Neural Networks (IJCNN 2010), 2010.